Program, information processing method, and server

Abstract

To improve convenience or the like related to a chat.SOLUTION: A program to be executed by a server for executing processing related to a chat including a first user of a first terminal and a second user of a second terminal makes the server execute: receiving text information based on an input to the first terminal in which a chat room including the first user and the second user is displayed from the first terminal by a communication part of the server; generating an image set including one or more images on the basis of the text information; transmitting first information on the image set to the first terminal by the communication part; receiving second information on one or more images selected from the image set based on the input to the first terminal displaying images included in the image data displayed in the chat room on the basis of the first information from the first terminal by the communication part; and transmitting third information based on the second information to the second terminal by the communication part.SELECTED DRAWING: Figure 1-1

Description

[Technical field]

[0001] The present disclosure relates to a program, an information processing method, a server, and the like. [Background technology]

[0002] When users communicate using a chat application or the like, they may use stamp images or the like that can effectively express the user's emotions, etc. For example, Patent Document 1 discloses application software that conveys a message from the user to a recipient user through the picture or character of an electronic stamp in addition to the emotion of the picture or character of the electronic stamp. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-028728 A Summary of the Invention

[0004] According to a first aspect of the present invention, a program executed by a server that performs processing related to a chat including a first user of a first terminal and a second user of a second terminal includes the following steps executed by the server: receiving text information from the first terminal based on input to the first terminal displaying a chat room including the first user and the second user by a communication unit of the server; generating an image set including one or more images based on the text information; transmitting first information related to the image set to the first terminal by the communication unit; receiving second information from the first terminal by the communication unit related to one or more images selected from the image set based on input to the first terminal displaying an image in the chat room based on the first information; and transmitting third information based on the second information to the second terminal by the communication unit. According to a second aspect of the present invention, an information processing method of a server that performs processing related to a chat including a first user of a first terminal and a second user of a second terminal includes receiving text information from the first terminal by a communication unit of the server based on input to the first terminal on which a chat room including the first user and the second user is displayed, generating an image set including one or more images based on the text information, transmitting first information related to the image set to the first terminal by the communication unit, receiving second information from the first terminal by the communication unit related to one or more images selected from the image set based on input to the first terminal on which images included in the image set are displayed in the chat room based on the first information, and transmitting third information based on the second information to the second terminal by the communication unit. According to a third aspect of the present invention, a server that performs processing related to a chat including a first user of a first terminal and a second user of a second terminal includes a communication unit and a control unit, wherein the communication unit receives text information from the first terminal based on input to the first terminal on which a chat room including the first user and the second user is displayed, the control unit generates an image set including one or more images based on the text information, the communication unit transmits first information regarding the image set to the first terminal, receives second information from the first terminal regarding one or more images selected from the image set based on input to the first terminal on which images included in the image set are displayed in the chat room based on the first information, and transmits third information based on the second information to the second terminal. [Brief description of the drawings]

[0005] [Figure 1-1] FIG. 1 is a diagram showing an example of a system configuration of a communication system according to a first embodiment. [Figure 1-2] FIG. 2 is a diagram showing an example of functions realized by a control unit of the server according to the first embodiment. [Figure 1-3] FIG. 4 is a diagram showing an example of information stored in a storage unit of the server according to the first embodiment. [Figure 1-4] FIG. 4 is a diagram showing an example of account registration data according to the first embodiment. [Figure 1-5]FIG. 11 is a diagram showing an example of stamp image additional learning model management data according to the first embodiment. [Figure 1-6] FIG. 4 is a diagram showing an example of stamp image generation prompt management data according to the first embodiment. [Figure 1-7] FIG. 4 is a diagram showing an example of a stamp management database according to the first embodiment. [Figure 1-8] FIG. 2 is a diagram showing an example of functions realized by a control unit of the terminal according to the first embodiment. [Figure 1-9] FIG. 4 is a diagram showing an example of information stored in a storage unit of the terminal according to the first embodiment. [Figure 1-10] FIG. 4 is a diagram showing an example of a screen displayed on a display unit of the terminal according to the first embodiment. [Figure 1-11] FIG. 4 is a diagram showing an example of a screen displayed on a display unit of the terminal according to the first embodiment. [Figure 1-12] FIG. 4 is a diagram showing an example of a screen displayed on a display unit of the terminal according to the first embodiment. [Figure 1-13] 4 is a flowchart showing an example of the flow of processes executed by each device according to the first embodiment. [Figure 1-14] FIG. 11 is a diagram showing an example of a screen displayed on a display unit of a terminal according to a first modified example. [Figure 1-15] 10 is a flowchart showing an example of the flow of processes executed by each device according to a first modified example. [Figure 1-16] FIG. 11 is a diagram showing an example of a screen displayed on a display unit of a terminal according to a first modified example. [Figure 2-1] FIG. 11 is a diagram showing an example of functions realized by a control unit of a server according to a second embodiment. [Figure 2-2] FIG. 11 is a diagram showing an example of stamp image generation prompt management data according to the second embodiment. [Figure 2-3] FIG. 11 is a diagram showing an example of a screen displayed on a display unit of a terminal according to a second embodiment. [Figure 2-4] FIG. 11 is a diagram showing an example of a screen displayed on a display unit of a terminal according to a second embodiment. [Figure 2-5] FIG. 11 is a diagram showing an example of a screen displayed on a display unit of a terminal according to a second embodiment. [Figure 2-6]10 is a flowchart showing an example of the flow of processes executed by each device according to the second embodiment. [Diagram 2-7] FIG. 11 is a diagram showing an example of a screen displayed on a display unit of a terminal according to a second modified example. [Figure 2-8] FIG. 11 is a diagram showing an example of a screen displayed on a display unit of a terminal according to a second modified example. [Figure 2-9] FIG. 11 is a diagram showing an example of a screen displayed on a display unit of a terminal according to a second modified example. [Figure 2-10] 10 is a flowchart showing an example of the flow of processes executed by each device according to a second modified example. [Figure 3-1] FIG. 13 is a diagram showing an example of information stored in a storage unit of a server according to a third embodiment. [Figure 3-2] FIG. 11 is a diagram showing an example of a screen displayed on a display unit of a terminal according to a third embodiment. [Figure 3-3] FIG. 11 is a diagram showing an example of a screen displayed on a display unit of a terminal according to a third embodiment. [Diagram 3-4] 13 is a flowchart showing an example of the flow of processes executed by each device according to the third embodiment. [Figure 4-1] FIG. 13 is a diagram showing an example of information stored in a storage unit of a server according to the fourth embodiment. [Figure 4-2] FIG. 13 is a diagram showing an example of a screen displayed on a display unit of a terminal according to a fourth embodiment. [Figure 4-3] FIG. 13 is a diagram showing an example of a screen displayed on a display unit of a terminal according to a fourth embodiment. [Figure 4-4] 13 is a flowchart showing an example of the flow of processes executed by each device according to the fourth embodiment. [Figure 4-5] FIG. 13 is a diagram showing an example of a screen displayed on a display unit of a terminal according to a fourth modified example. [Figure 4-6] FIG. 13 is a diagram showing an example of a screen displayed on a display unit of a terminal according to a fourth modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0006] <Compliance with legal matters> It should be noted that the disclosures described herein are subject to compliance with the laws of the country of implementation, such as communications secrecy, as necessary for the implementation of the disclosure.

[0007] <Embodiment> In this specification, for ease of understanding, there are some places where the word "for example" is used, but please note that not only these places but also the entire embodiment described below are not limited to the contents of the description.

[0008] An embodiment for implementing a program or the like according to the present disclosure will be described with reference to the drawings.

[0009] The production of a terminal of the present invention (a terminal of the present invention) may include, for example, the concept that a program (for example, an application program) described in this specification is received (or received and stored in) on a terminal owned (possessed) by a user, thereby creating a state in which the functions of the present invention can be realized (a state in which the present invention can be executed) on the terminal.

[0010] Furthermore, the production of the system of the invention claimed in the present application (the system of the invention) may include the concept of creating a state in which the functions of the invention of the system claimed in the present application can be realized (creating a state in which the invention claimed in the present application can be executed) by, for example, receiving a program described in this specification (for example, an application program) transmitted from a server included in the system of the present application at a terminal included in the system of the present application (or by storing the received program in the terminal).

[0011] In this specification, a system can be, for example, configured to include a plurality of devices. The multiple devices may be a combination of devices of the same type, a combination of devices of different types, or a combination of devices of the same type and devices of different types. It should be noted that a system can be thought of as, for example, a number of devices working together to perform some kind of processing.

[0012] Moreover, a system relating to a client (client device) and a server can be considered to be, for example, at least one of the following: (1) Terminals and servers (2) Server (3) Terminal

[0013] (1) is, for example, a system including at least one terminal and at least one server. One example of this is a client-server system.

[0014] The server is, for example, configured with the following devices, and may be a single device or a combination of multiple devices.

[0015] Specifically, the server is configured to have at least one processor (for example, a CPU: Central Processing Unit, GPU: Graphics Processing Unit, APU: Accelerated Processing Unit, DSP: Digital Signal Processor (for example, an ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array), etc.), a computer device (processor + memory), a control device, an arithmetic device, a processing device, etc., and may be configured to have multiple of the same type of any one of the devices (for example, a CPU + CPU, a homogeneous multi-core processor, etc.), or multiple different types of any one of the devices (for example, a CPU + DSP, a heterogeneous multi-core processor, etc.), or may be a combination of multiple devices (for example, a processor + computer device, a processor + arithmetic device, a heterogeneous combination of multiple devices, etc.). The processor may be a virtual processor.

[0016] Also, when a server executes some processing, if the server is configured with a single device, the processing described in the embodiment is executed by the single device. If the server is configured with multiple devices, the server may be configured to execute some processing by one device and execute other processing by the other device. For example, if the server is configured with a processor and an arithmetic device, the server may be configured to execute a first processing by the processor and execute a second processing by the arithmetic device. Furthermore, when a configuration is made up of a plurality of devices, the devices may be arranged in locations physically separated from one another.

[0017] Furthermore, the functions of the server may be provided in the form of PaaS, IaaS, or SaaS in cloud computing, for example.

[0018] The control unit of the system can be at least one of the control unit of the terminal and the control unit of the server. That is, for example, the control unit of the system can be any one of (1A) only the control unit of the terminal, (1B) only the control unit of the server, or (1C) both the control unit of the terminal and the control unit of the server.

[0019] In addition, the control and processing (hereinafter collectively referred to as "control, etc.") performed by the control unit of the system may be performed by (1A) only the control unit of the terminal, (1B) only the control unit of the server, or (1C) both the control unit of the terminal and the control unit of the server. In addition, in (1C), for example, some of the controls performed by the control unit of the system may be performed by the control unit of the terminal, and the remaining controls may be performed by the control unit of the server. In this case, the allocation (allocation) of the controls may be equal or may be different.

[0020] Furthermore, when referring to a communication unit of a server, if the server is configured with a single device, it may be the communication unit itself that the single device has, or, if the server is configured with multiple devices, the communication unit of the server may be configured to include each communication unit that each device has. As an example, when a server includes a first device and a second device, the first device has a first communication unit and the second device has a second communication unit, the communication unit of the server may be conceptualized as including the first communication unit and the second communication unit.

[0021] (2) can be, for example, a system consisting of multiple servers (hereinafter referred to as a "server system"). In this case, the configuration of each server can be similarly applied to the configuration described above.

[0022] The control etc. performed by the server system may be performed by only one of the multiple servers (2A), by only the other servers (2B), or by both the one server and the other servers (2C). In addition, in (2C), for example, one server may perform part of the control etc. performed by the server system, and another server may perform the remaining control etc. In this case, the allocation (allocation) of the control etc. may be equal or may be allocated in different proportions.

[0023] (3) As an example, it may be a system composed of multiple terminals. This system can be, for example, the following system. A system that gives server functions to terminals (distributed system). This can be realized, for example, by using blockchain technology. A system in which terminals communicate wirelessly with each other. This can be realized, for example, by communicating in a peer-to-peer (P2P) manner using short-range wireless communication technology such as Bluetooth (registered trademark).

[0024] The above is not limited to the control unit, but also applies to each functional unit such as an input / output unit, a communication unit, a storage unit, and a clock unit that may be components of the system.

[0025] In the following embodiment, a system including a terminal and a server (a client-server system, for example) is illustrated as an example. It should be noted that the server system described above in (2) can also be used as the server.

[0026] Also, instead of a system including a terminal and a server, it is possible to apply a system that does not include a server, for example the system (3) above. In this case, the embodiment can be configured based on the above-mentioned blockchain technology. Specifically, for example, data stored and managed in a server described in the following embodiment is stored on the blockchain. Then, the terminal generates a transaction to the blockchain, and when the transaction is approved on the blockchain, the data stored on the blockchain can be updated.

[0027] It should be noted that even when the term "terminal" is used, this is not limited to the meaning of a terminal as a client device in a client server. That is, a terminal may include the concept of a device other than in a client-server context.

[0028] Furthermore, in this specification, the expression "through a communication I / F" is used as appropriate. This may, for example, indicate that a device transmits and receives various information and data through a communication I / F (through a communication unit) based on the control of a control unit (such as a processor).

[0029] In addition, in this specification, when the terms "related to" and "associated" are used, "B related to A" or "B related to A" may mean "B" having some kind of relationship with "A". Specific examples of this will be described later.

[0030] In addition, in this specification, when a device performs processing on two or more objects, such as "transmitting A and B" or "receiving A and B," this may include performing "A" and "B" in a synchronized manner (hereinafter referred to as "simultaneous"), and performing "A" and "B" with a different timing (hereinafter referred to as "non-simultaneous"). For example, when referring to transmitting first information and second information, this may include both concepts of transmitting the first information and the second information in a synchronized manner and transmitting the first information and the second information with a lag in timing. In addition, taking into account the lag (time lag), "simultaneous" may include "almost simultaneously."

[0031] Note that, even if "A" and "B" are performed at different times, this only needs to be the processing targeting "A" and "B," and the purpose does not necessarily have to be the same. For example, when the first information and the second information are transmitted as described above, it is sufficient to transmit the first information and the second information, and this may include cases where the first information and the second information are transmitted for the same purpose as well as cases where the first information and the second information are transmitted for different purposes.

[0032] In the following embodiment, a messaging service is illustrated as an example of a service that allows users to chat (hereinafter, referred to as a "chat service"). An application for realizing the chat service is referred to as a "chat application," and an application for realizing the messaging service is referred to as a "messaging application." A chat application may, as an example, allow users to chat in chat rooms.

[0033] Note that messaging services (MS) (including instant messaging services (IMS)) can be considered as one form of social networking services (SNS). Therefore, messaging services and social networking services may or may not be distinguished. In other words, social networking services may include messaging services.

[0034] In the following embodiment, an instant messaging service (IMS) that transmits and receives simple messages between a plurality of devices (terminals, for example) via a server is exemplified as an example of a messaging service. An instant messaging application, as an example, may allow users to talk in talk rooms.

[0035] A chat room (for example, a talk room) can be a UI (User Interface) or GUI (Graphical User Interface) that allows each user to view content transmitted and received between the terminals of multiple users.

[0036] In a messaging service, for example, a user can register any account as a "friend" and exchange messages with the account via a chat room.

[0037] Note that "friend" may refer to a relationship between a first user and a second user, for example, when the first user's account and the second user's account in a messaging application are unilaterally or bilaterally associated with each other. For example, if a first user's account and a second user's account are mutually associated, the first user and the second user may be said to be "friends." Also, for example, if a first user's account is associated with a second user's account, the first user and the second user may be said to be "friends" even if the second user's account is not associated with the first user's account.

[0038] In addition, talk rooms can include one-to-one user talk rooms (hereinafter referred to as "one-to-one talk rooms"), talk rooms for groups including multiple users (hereinafter referred to as "group talk rooms"), and talk rooms with OA businesses (for example, businesses that are affiliated with messaging service providers) (hereinafter referred to as "OA talk rooms").

[0039] In addition, an account of a messaging application that is an account of a business operator and not a general user may be referred to as an "OA (Official Account)," and a user of this official account may be referred to as an "official user." This may also be referred to as an "official account user" or an "official account business operator," etc. In contrast, an account of a messaging application that is a user who is not an OA business is called a "general account," and a user of a general account is called a "general user." This may also be called a "general account user," etc. In other words, messaging application accounts may include general accounts and official accounts.

[0040] In addition, for example, OA businesses may also be able to use terminals similar to those of users with general accounts to send and receive messages to and from other devices via the server.

[0041] Furthermore, a message (message information) may be, for example, information that defines a sender and a destination used in a messaging service, and may be information that is composed of identification information (message ID) for identifying the message and message content. Additionally, the message content may refer to the contents of the message excluding the message ID, for example. The message content may be one or more pieces of content.

[0042] Moreover, information set as identification information for identifying a message is referred to as a "message ID." It can also be considered that message content included in a message with the same message ID is identified by this message ID. Therefore, it can also be considered that identification information for identifying a message is substantially synonymous with identification information for identifying message content. Alternatively, individual identification information (message content ID) may be set for each message content, but this is not essential.

[0043] The content may include, for example, text content in text format, image content in image format (including at least one of still images and moving images), and sound content in sound format (including voice). In addition, the content may include operation content such as buttons and icons for user operations, and link content such as URIs (including URLs).

[0044] The text may include, for example, at least one of national characters represented by character codes, extended characters, machine-dependent characters, numbers, symbols, graphics, and codes. Note that the text does not have to include at least one of the above characters, extended characters, machine-dependent characters, numbers, symbols, figures, and signs, and may include other text.

[0045] The image may include at least one of various types of image information, such as an icon, a button, a stamp, an emoji, and a banner image.

[0046] Unlike the above definition, the superordinate concept of a message may be content, or content and message may be synonymous, or it is not necessary to define them in this way.

[0047] <Embodiment> For example, when creating stamps to be used as a means of facilitating communication in messaging applications, the user who creates them must carry out tedious tasks such as preparing and processing photographs, illustrations, etc. Also, for example, it is preferable to create multiple stamps based on a common character or concept, and there may be high barriers to entry for users to easily process and publish photos of their pets or illustrations they have drawn as stamps.

[0048] This embodiment relates to a service that allows a user to easily create a stamp based on an image provided by the user, as an example.

[0049] Moreover, this embodiment is an embodiment relating to a service that allows a user to easily create and send a stamp based on text input by the user, for example.

[0050] Here, a stamp is, by way of example and not limitation, an icon image or the like that can be used to add emotion or expression to a text message in a chat service, and may also be called a "sticker." It should be noted that a stamp may be a type of sticker.

[0051] In the following, for example, a collection of stamps that is composed of any number of stamps and has some kind of relationship, such as a common character or concept, will be referred to as a "stamp set." Also, for example, an image associated with an individual stamp will be referred to as a "stamp image." In contrast, for example, a collection of stamp images in a stamp set will be referred to as a "stamp set image." In the following, examples will be mainly given in which the stamp image is a still image, but the stamp image may be a moving image or animation.

[0052] In the following description, one or more image sets provided by a user for generating a stamp image are referred to as "base images." The base images may also be referred to as a "base image set." In the following description, characters, subjects, and the like commonly included in the base images are referred to as “objects.” An object may be said to be a concept common to the base images. An identifier for identifying an object is called an "instance prompt." An identifier for identifying a general property common to objects or a classification (class or category) of a learning subject is called a "class prompt." Specific examples of instance prompts and class prompts will be described later. In the following, an example will be given in which the base image is a still image, but the base image may be a moving image or animation.

[0053] Furthermore, an object may be considered as a type of element (image element) included in an image, and in the embodiments described below, an "object = element" may be considered. However, elements are not limited to objects, and other examples will be described later.

[0054] <First Example> The first embodiment is an embodiment in which a stamp set image is created based on a base image. The contents described in the first embodiment are similarly applicable to any of the other embodiments and other modified examples.

[0055] In the following embodiment, a service for realizing support for creating a stamp set image or a stamp set is referred to as a "stamp set creation support service" by way of example. Also, an application for realizing the stamp set creation support service is referred to as a "stamp set creation application" by way of example. In the stamp set creation application, for example, a user can easily create a stamp set image or a stamp set by registering a base image.

[0056] As a form for realizing the stamp set creation support service (stamp set creation application), for example, any of the following forms can be applied. (A1) A form in which a stamp set creation support service function is provided as one function of a messaging application (A2) A form in which a messaging service function is provided as one of the functions of a stamp set creation application (B1) A form constituting an application (integrated application) having a messaging service function and a stamp set creation support service function (C1) A form in which a stamp set creation application is configured as a separate application from a messaging application (C2) A form in which a messaging application is configured as a separate application from a stamp set creation application (D) A form in which a stamp set creation application is configured as a standalone application

[0057] In the cases of (A1), (A2) and (B1), for example, the messaging service provider may be the same provider as the stamp set creation support service provider. In this case, as one method, the user's account in the messaging application and the user's account in the stamp set creation application can be a common account. In this case, as an alternative method, the user's account in the messaging application and the user's account in the stamp set creation application can be automatically associated (linked).

[0058] In the embodiments (C1) to (C2), for example, the messaging service provider may be a provider different from the stamp set creation support service provider. In addition, in the embodiments (C1) to (C2), a process for associating a user's account in the messaging application with a user's account in the stamp set creation application (a linking process) can be performed.

[0059] In the following, as an example, a description will be given on the assumption that the server 10 is operated and managed by a messaging service provider in accordance with the form (A1).

[0060] In the following description, the user of terminal 20A communicating with server 10 is referred to as "user AA", the user of terminal 20B as "user BB", the user of terminal 20C as "user CC", . . . .

[0061] <System configuration> 1-1 is a diagram illustrating an example of a system configuration of a communication system 1 in this embodiment. In the communication system 1, for example, a server 10 and one or more terminals 20 (terminal 20A, terminal 20B, terminal 20C, ...) are connected via a network 30.

[0062] The server 10 has a function of providing, for example, a messaging service and a stamp set creation support service to a terminal 20 owned by a user via a network 30 . The server 10 can also be expressed as a messaging service server, a stamp set creation support server, an image generation server, etc. In this embodiment, the user of the server 10 is, for example, a company that provides a messaging service or a stamp set creation support service (a messaging service or stamp set creation support service provider). Note that the number of servers 10 and the number of terminals 20 connected to the network 30 are not limited to the above. In this embodiment, the "stamp set creation support service" is a service provided by a business operator (server 10) such as a company that provides stamp set creation support services, and may be provided to a user (user's terminal 20), for example.

[0063] The terminal 20 (terminal 20A, terminal 20B, terminal 20C, ...) may be any information processing terminal capable of implementing the functions described in each embodiment. Examples of the terminal 20 include a smartphone, a mobile phone (feature phone), a computer (such as, but not limited to, a desktop, laptop, tablet, etc.), a media computer platform (such as, but not limited to, a cable or satellite set-top box, digital video recorder), a handheld computer device (such as, but not limited to, a PDA (personal digital assistant), email client, etc.), a wearable terminal (such as a glasses-type device, a watch-type device, etc.), a VR (Virtual Reality) terminal, a smart speaker (a device for voice recognition), or other types of computers or communication platforms. The terminal 20 may also be expressed as an information processing terminal.

[0064] For example, the configurations of the terminals 20A, 20B, and 20C can be the same. In addition, if necessary, the terminal used by the user X may be expressed as the terminal 20X, and the user information in a predetermined service associated with the user X or the terminal 20X may or may not be expressed as the user information X. The user information is information of a user associated with an account used by the user in a specific service. The user information includes, but is not limited to, information associated with a user, such as the user's name, the user's icon image, the user's age, the user's sex, the user's address, the user's hobbies and interests, and the user's identifier, which is input by the user or given by a specific service, and may be any one of these, or a combination of these, or may not be the same.

[0065] The network 30 plays a role in connecting the various devices constituting the communication system 1. In other words, the network 30 refers to a communication network that provides connection paths so that the various devices described above can transmit and receive data after being connected to each other.

[0066] One or more portions of network 30 may or may not be wired or wireless networks. Network 30 may include, by way of example, an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless WAN (WWAN), a metropolitan area network (MAN), a portion of the Internet, a portion of the Public Switched Telephone Network (PSTN), a cellular network, integrated service digital networks (ISDN), wireless LAN, long term evolution (LTE), code division multiple access (CDMA), Bluetooth, satellite communications, or the like, or a combination of two or more thereof. Network 30 may include one or more networks 30.

[0067] The server 10 (not limited to this, but an example of a server, information processing device, or information management device) has a function of providing a predetermined service (messaging service or stamp set creation support service in this embodiment) to the terminal 20 or the like. The server 10 may be any device as long as it is an information processing device that can realize the functions described in each embodiment. The server 10 includes, for example, a server device, a computer (for example, a desktop, a laptop, a tablet, etc.), a media computer platform (for example, a cable or satellite set-top box, a digital video recorder), a handheld computer device (for example, a PDA, an email client, etc.), or other types of computers or communication platforms. The server 10 may also be expressed as an information processing device. When it is not necessary to distinguish between the server 10 and the terminal 20, the server 10 and the terminal 20 may or may not be expressed as information processing devices.

[0068] [Hardware (HW) configuration of each device] The hardware configuration of each device included in the communication system 1 will be described.

[0069] (1) Terminal hardware configuration FIG. 1-1 shows an example of the hardware configuration of the terminal 20. In FIG. The terminal 20 includes, for example, a control unit 21 (CPU: central processing unit), a storage unit 28, a communication I / F 22 (interface), an input / output unit 23, a clock unit 29A, and a position calculation information detection unit 29B. The HW components of the terminal 20 are connected to each other, for example, via a bus B. It is not essential that the HW configuration of the terminal 20 includes all of the components. For example, the terminal 20 may or may not be configured such that individual components or multiple components are removable.

[0070] The communication I / F 22 transmits and receives various data via the network 30. The communication may be performed either wired or wirelessly, and any communication protocol may be used as long as the communication between the two devices is possible. The communication I / F 22 has a function of communicating with various devices such as the server 10 via the network 30. The communication I / F 22 transmits various data to various devices such as the server 10 according to instructions from the control unit 21. The communication I / F 22 also receives various data transmitted from various devices such as the server 10 and transmits it to the control unit 21. The communication I / F 22 may also be simply referred to as a communication unit. When the communication I / F 22 is configured with a physically structured circuit, it may also be referred to as a communication circuit.

[0071] The input / output unit 23 includes a device for inputting various operations to the terminal 20, a device for outputting processing results processed by the terminal 20, etc. The input / output unit 23 may be an integrated input unit and an output unit, or may be separate from the input unit and the output unit, or may not be.

[0072] The input unit is realized by any one or a combination of all kinds of devices that can receive input from a user and transmit information related to the input to the control unit 21. Examples of the input unit include hardware keys such as a touch panel, a touch display, and a keyboard, a pointing device such as a mouse, a camera (operation input via moving images), and a microphone (operation input by voice).

[0073] The output unit is realized by any one or a combination of all kinds of devices capable of outputting the processing results processed by the control unit 21. Examples of the output unit include a touch panel, a touch display, a speaker (audio output), a lens (for example, 3D (three dimensions) output or hologram output), a printer, etc.

[0074] Although this is merely an example, the input / output unit 23 includes a display unit 24, a sound input unit 25, a sound output unit 26, and an imaging unit 27, for example.

[0075] The display unit 24 is realized by any one of all kinds of devices capable of displaying according to the display data written in the frame buffer, or a combination thereof. Examples of the display unit 24 include a touch panel, a touch display, a monitor (e.g., a liquid crystal display or an organic electroluminescence display (OLED)), a head mounted display (HDM), a projection mapping, a hologram, and a device capable of displaying images, text information, and the like in the air (which may or may not be a vacuum). Note that these display units 24 may or may not be capable of displaying display data in 3D.

[0076] The sound input unit 25 is used to input sound data (including voice data; the same applies below). The sound input unit 25 includes a microphone and the like. The sound output unit 26 is used to output sound data and includes a speaker and the like. The imaging unit 27 is used to obtain image data (including still image data and moving image data; the same applies below.) The imaging unit 27 includes a camera and the like.

[0077] When the input / output unit 23 is a touch panel, the input / output unit 23 and the display unit 24 may be disposed facing each other and have approximately the same size and shape.

[0078] The clock unit 29A is a built-in clock of the terminal 20, and outputs time information (timekeeping information). The clock unit 29A is configured to include, for example, a clock using a crystal oscillator. The clock unit 29A can also be expressed as, for example, a timekeeping unit or a time information detection unit.

[0079] The clock unit 29A may or may not have a clock that conforms to the Network Identity and Time Zone (NITZ) standard or the like.

[0080] The position calculation information detection unit 29B is a functional unit that detects (measures) information (hereinafter referred to as "position calculation information") necessary for the control unit 21 to calculate (measure) the position of its own terminal 20. The position calculation information detection unit 29B can also be expressed as a position calculation sensor unit, for example.

[0081] The position calculation information detection unit 29B includes, for example, a satellite positioning sensor (satellite positioning unit) which is a sensor or unit for calculating the position of the terminal 20 using a satellite positioning system such as GPS (Global Positioning System), an inertial measurement sensor (inertial measurement unit (IMU (Inertial Measurement Unit))) which is a sensor or unit for calculating the position of the terminal 20 using an inertial navigation system, a UWB positioning sensor (UWB positioning unit) which is a sensor or unit for calculating the position of the terminal 20 using UWB (Ultra Wide Band), and the like.

[0082] The satellite positioning unit, for example, includes an RF receiving circuit that converts RF (Radio Frequency) signals, including positioning satellite signals transmitted from positioning satellites and received by an antenna (not shown), into digital signals, and a baseband processing circuit that performs correlation calculation processing or the like on the digital signals output from the RF receiving circuit to capture the positioning satellite signals, and outputs information such as satellite orbit data and time data extracted from the positioning satellite signals as information for position calculation.

[0083] The inertial measurement unit has an inertial sensor that detects information required for calculating the position of the terminal 20 by inertial navigation calculation. The inertial sensor includes, for example, a three-axis acceleration sensor and a three-axis gyro sensor, and outputs the acceleration detected by the acceleration sensor and the angular velocity detected by the gyro sensor as information for position calculation.

[0084] The UWB positioning unit, for example, includes an ultra-wideband RF receiving circuit that converts an ultra-wideband RF (Radio Frequency) signal, including an ultra-wideband positioning pulse signal transmitted from a positioning beacon and received by an antenna not shown, into a digital signal, and a relative position calculation processing circuit that calculates the relative position between the terminal 20 and the positioning beacon based on the digital signal output from the ultra-wideband RF receiving circuit. As an example, the UWB positioning unit may or may not cause the terminal 20 to function as a positioning beacon by transmitting an ultra-wideband RF signal including an ultra-wideband pulse signal for positioning from an antenna not shown.

[0085] For example, the control unit 21 calculates the position of its own terminal 20 at regular timing or specific timing based on the position calculation information detected by the position calculation information detection unit 29B. The terminal position is referred to as the "terminal position", and the calculated terminal position is referred to as the "calculated terminal position". The control unit 21 may, but may not, associate the calculated terminal position with the date and time when the calculated terminal position is calculated and store it in the storage unit 28 as calculated terminal position history data.

[0086] The control unit 21 has a circuit physically structured to execute the functions realized by the codes or instructions included in the program, and is realized by a data processing device built into hardware, for example. Therefore, the control unit 21 may or may not be expressed as a control circuit.

[0087] The control unit 21 includes, for example, a central processing unit (CPU), a microprocessor, a processor core, a multiprocessor, an application-specific integrated circuit (ASIC), and a field programmable gate array (FPGA).

[0088] The storage unit 28 has a function of storing various programs and various data required for the operation of the terminal 20. The storage unit 28 includes various storage media such as a hard disk drive (HDD), a solid state drive (SSD), a flash memory, a random access memory (RAM), and a read only memory (ROM), for example. The storage unit 28 may or may not be expressed as a memory.

[0089] Terminal 20 stores program P in storage unit 28, and executes this program P, causing control unit 21 to execute processing as each unit included in control unit 21. In other words, program P stored in storage unit 28 causes terminal 20 to realize each function executed by control unit 21. Also, this program P may or may not be expressed as a program module.

[0090] (2) Server hardware configuration FIG. 1-1 shows an example of the HW configuration of the server 10. In FIG. The server 10 includes, for example, a control unit 11 (CPU), a storage unit 15, a communication I / F 14 (interface), an input / output unit 12, a display unit 13, and a clock unit 19. The components of the HW of the server 10 are connected to each other, for example, via a bus B. Note that it is not essential that the HW of the server 10 includes all components as the configuration of the HW of the server 10. For example, the HW of the server 10 may or may not be configured such that individual components or multiple components are removable.

[0091] The control unit 11 has circuits that are physically structured to execute functions realized by codes or instructions contained in a program, and is realized, for example, by a data processing device incorporated in hardware.

[0092] The control unit 11 is typically a central processing unit (CPU), but may also be a microprocessor, a processor core, a multiprocessor, an ASIC, or an FPGA, but is not limited to these in the present disclosure.

[0093] The storage unit 15 has a function of storing various programs and various data required for the operation of the server 10. The storage unit 15 is realized by various storage media such as an HDD, an SSD, and a flash memory. However, in the present disclosure, the storage unit 15 is not limited to these. Furthermore, the storage unit 15 may or may not be expressed as a memory.

[0094] The communication I / F 14 transmits and receives various data via the network 30. The communication may be performed either wired or wirelessly, and any communication protocol may be used as long as the communication between the two devices is possible. The communication I / F 14 has a function of communicating with various devices such as the terminal 20 via the network 30. The communication I / F 14 transmits various data to various devices such as the terminal 20 according to instructions from the control unit 11. The communication I / F 14 also receives various data transmitted from various devices such as the terminal 20 and transmits it to the control unit 11. The communication I / F 14 may also be simply referred to as a communication unit. When the communication I / F 14 is configured with a physically structured circuit, it may also be referred to as a communication circuit.

[0095] The input / output unit 12 includes a device for inputting various operations to the server 10, a device for outputting the results of processing by the server 10, etc. The input / output unit 12 may be an integrated input unit and an output unit, or may be separate input unit and output unit, or may not be the same.

[0096] The input unit is realized by any one or a combination of all kinds of devices that can receive input from a user and transmit information related to the input to the control unit 11. The input unit is typically realized by hardware keys such as a keyboard or a pointing device such as a mouse. Note that the input unit may or may not include, for example, a touch panel, a camera (operation input via moving images), or a microphone (operation input by voice).

[0097] The output unit is realized by any one or a combination of all kinds of devices capable of outputting the processing results processed by the control unit 11. Examples of the output unit include a touch panel, a touch display, a speaker (sound output), a lens (for example, 3D (three dimensions) output or hologram output), a printer, etc.

[0098] By way of example only, the input / output unit 12 includes a display unit 13, for example.

[0099] The display unit 13 is realized by a display or the like. The display is typically realized by a monitor (for example, a liquid crystal display or an organic electroluminescence display (OLED)). The display may or may not be a head mounted display (HDM) or the like. These displays may or may not be capable of displaying display data in 3D. In the present disclosure, the display is not limited to these.

[0100] The clock unit 19 is a built-in clock of the server 10, and outputs time information (timekeeping information). The clock unit 19 is configured to include, for example, an RTC (Real Time Clock) as a hardware clock, a system clock, etc. The clock unit 19 can also be expressed as, for example, a timekeeping unit or a time information detection unit.

[0101] (3)Other The server 10 stores a program P in the storage unit 15, and by executing this program P, the control unit 11 executes the processes of each unit included in the control unit 11. In other words, the program P stored in the storage unit 15 causes the server 10 to realize each function executed by the control unit 11. This program P may or may not be expressed as a program module. The same applies to other devices.

[0102] In each embodiment of the present disclosure, the description will be given assuming that the CPU of the terminal 20 and / or the server 10 executes the program P to realize the present invention. The same applies to other devices.

[0103] The control unit 21 of the terminal 20 and / or the control unit 11 of the server 10 may or may not realize each process by a CPU having a control circuit, but may also realize each process by a logic circuit (hardware) formed in an integrated circuit (IC (Integrated Circuit) chip, LSI (Large Scale Integration)) or a dedicated circuit. These circuits may or may not be realized by one or more integrated circuits, and the multiple processes shown in each embodiment may or may not be realized by one integrated circuit. Furthermore, an LSI may be called a VLSI, a super LSI, an ultra LSI, or the like, depending on the degree of integration. Therefore, the control unit 21 may or may not be expressed as a control circuit. The same applies to other devices.

[0104] In addition, the program P (e.g., a software program, a computer program, or a program module) of each embodiment of the present disclosure may or may not be provided in a state stored in a computer-readable storage medium. The storage medium is capable of storing the program P in a "non-transitory tangible medium." In addition, the program P may or may not be for realizing part of the functions of each embodiment of the present disclosure. Furthermore, the program P may or may not be a so-called difference file (difference program) that can realize the functions of each embodiment of the present disclosure in combination with a program P already recorded in a storage medium.

[0105] The storage medium may include one or more semiconductor-based or other integrated circuits (ICs) (such as, for example, a field programmable gate array (FPGA) or an application specific IC (ASIC)), a hard disk drive (HDD), a hybrid hard drive (HHD), an optical disk, an optical disk drive (ODD), a magneto-optical disk, a magneto-optical drive, a floppy diskette, a floppy disk drive (FDD), a magnetic tape, a solid state drive (SSD), a RAM drive, a secure digital card, or a drive, any other suitable storage medium, or a suitable combination of two or more of these. The storage medium may be volatile, non-volatile, or a combination of volatile and non-volatile, where appropriate. It should be noted that the storage medium is not limited to these examples and may be any device or medium capable of storing the program P. The storage medium may or may not be referred to as a memory.

[0106] The server 10 and / or the terminal 20 can realize the functions of the multiple functional units shown in each embodiment by reading out the program P stored in a storage medium and executing the read out program P. The same applies to other devices.

[0107] Furthermore, the program P of the present disclosure may or may not be provided to the server 10 and / or the terminal 20 via any transmission medium capable of transmitting a program (such as a communication network or broadcast waves). The server 10 and / or the terminal 20 executes the program P downloaded via the Internet or the like, for example, to realize the functions of the multiple functional units shown in each embodiment. The same applies to other devices.

[0108] In addition, each embodiment of the present disclosure may also be realized in the form of a data signal in which the program P is embodied by electronic transmission. At least a part of the processing in the server 10 and / or the terminal 20 may or may not be realized by cloud computing configured with one or more computers. At least a part or all of the processing in the terminal 20 may or may not be performed by the server 10. In this case, at least a part or all of the processing of each functional unit of the control unit 21 of the terminal 20 may or may not be performed by the server 10. At least a part or all of the processing in the server 10 may or may not be performed by the terminal 20. In this case, at least a part or all of the processing of each functional unit of the control unit 11 of the server 10 may or may not be performed by the terminal 20. Unless explicitly stated, the judgment configuration in the embodiments of the present disclosure is not essential, and a specified process may or may not be executed when the judgment condition is satisfied, or a specified process may or may not be executed when the judgment condition is not satisfied.

[0109] The programs disclosed herein are implemented using, for example, scripting languages ​​such as ActionScript and JavaScript (registered trademark), compiler languages ​​such as Objective-C and Java (registered trademark), and markup languages ​​such as HTML Living Standard.

[0110] [Functional configuration of each device] (1) Functional configuration of the server FIG. 1-2 is a diagram showing an example of functions realized by the control unit 11 of the server 10 in this embodiment. The control unit 11 includes, for example, an application management processing unit 111 and an image generation model unit 113 as functional units.

[0111] The application management processing unit 111 has a function of executing application management processing in accordance with an application management processing program 151 stored in the storage unit 15, for example.

[0112] The image generation model unit 113 has a function of generating and outputting an image based on, for example, an input keyword or sentence (which may be called a "prompt"). The image generation model unit 113 may be configured, for example, with a Text-to-Image model (trained neural network model) such as DALL-E, Stable Diffusion, or Imagen.

[0113] A prompt may be a command or text instruction that causes an image generation device to generate an image, or a factor that determines the content of an image generated by an image generation device. The prompt may or may not be entered by the user.

[0114] Note that the image generation model unit 113 (not limiting, but an example of an image generation device that generates an image based on a prompt) is not limited to being provided in the server 10. As a non-limiting example, the image generation device may be provided outside the server 10 (not limiting, but an example of an information processing device). And as a non-limiting example, the image generation device and the information processing device may communicate with each other to advance the processing in this embodiment.

[0115] FIG. 1-3 is a diagram showing an example of information stored in the storage unit 15 of the server 10 in this embodiment. The memory unit 15 stores, for example, an application management processing program 151 executed as application management processing, account registration data 153, stamp image additional learning model management data 155, stamp image generation prompt management data 157, and a stamp set management database 159.

[0116] The account registration data 153 is registration data related to the account of an application (messaging application in this embodiment), and an example of the data configuration is shown in FIG. 1-4. In the account registration data 153, for example, a user name, an application ID, and other registration information are stored in association with each other.

[0117] The user name is the name of the account of the terminal 20 that uses this application, and as an example, the name that the user of the terminal 20 registers when using the application is stored.

[0118] The application ID is information used to identify an application account, or the account itself. This application ID is preferably a value unique to each account, and for example, a unique value (proper value) is set and stored by the server 10 for each account. The application ID is information associated with the terminal 20 or the user of the terminal 20, and is an example of information related to the terminal or information related to the user of the terminal.

[0119] Other registration information may include various information, for example, identification information for identifying terminal 20, the telephone number (terminal telephone number) of terminal 20, an email address (terminal email address), and authentication information such as passwords (login password, authentication password, etc.) used for various authentications in applications.

[0120] The identification information for identifying the terminal 20 may be, for example, a terminal ID (for example, an IMEI (International Mobile Equipment Identity)).

[0121] For example, the other registration information may be stored in association with user information.

[0122] Note that the application ID may or may not be replaced with a "user ID." In addition, if the application allows only one account to be registered per terminal 20, for example, "identification information for identifying the terminal 20=identification information for identifying the user of the terminal 20=application ID" may be used.

[0123] Also, for example, it may be possible to assign multiple terminal IDs to one application ID, but this is not required. In this case, it may be possible to simultaneously start an application on multiple terminals 20 using one application ID as an identification (login) target, but this is not required.

[0124] Also, instead of various IDs such as application IDs, it is possible to apply a method of managing accounts using information such as terminal telephone numbers. In this case, instead of storing ID information such as an application ID in account registration data 153, information such as a terminal telephone number can be stored in account registration data 153. Note that, instead of replacing ID information such as an application ID with information such as a terminal telephone number, ID information such as an application ID may or may not correspond one-to-one to information such as a terminal telephone number.

[0125] In the following embodiment, for the sake of simplicity, it is assumed that one account is registered for one terminal 20. Also, in this case, since "identification information for identifying terminal 20 = identification information for identifying the user of terminal 20 = application ID" as described above, the term "user of the account" used in the following explanation may or may not be substantially synonymous with "terminal of the account."

[0126] The stamp image additional learning model management data 155 is management data regarding an image generation model (called an "additional learning model") that has undergone additional learning of a base image in the image generation model unit 113, and an example of the data configuration is shown in FIG. 1-5. In the stamp image additional learning model management data 155, for example, an additional learning model ID, an additional learning model URI, an instance prompt, a class prompt, and other model information are stored in association with each other.

[0127] The additional learning model ID is information used to identify the additional learning model. This additional learning model ID is preferably a unique value for each account, and as an example, a unique value (proper value) is set and stored by the server 10 for each model. In addition, an instance prompt may be used as the additional learning model ID.

[0128] The additional learning model URI is, for example, information indicating a storage location of a model (data file) in which the structure and parameters of the additional learning model are described. For example, the image generation model unit 113 may be configured to read the model specified by the additional learning model URI and drive the additional learning model that can generate the additional learning model for the base image that has been subjected to additional learning.

[0129] The instance prompt is a character string that serves as a keyword for generating an image that includes an object that is commonly included in the base image that has undergone additional learning in the additional learning model. The instance prompt may be referred to as an identifier for identifying an object, or as information for specifying the object of the image to be generated. The instance prompt may be referred to as an "instance token." This instance prompt is preferably a unique value (specific value) for each base image (objects such as characters and concepts commonly included in the base images) for which additional learning is performed, and for example, a unique value (specific value) is set and stored for each base image by the server 10. In addition, this instance prompt is preferably set to a character string (for example, random character examples such as "sts", "usu", "lvl", etc.) that does not match any previously learned prompt token (for example, a general word, etc.).

[0130] The class prompt is a character string for specifying the class (category) of an object in the base image that has been subjected to additional learning in the additional learning model. The class prompt may also be called a "class token." As the class prompt, for example, a character string corresponding to the category of the base image identified by an image identification unit (not shown) of the server 10 is set and stored.

[0131] For example, if a "rabbit character" appears as an object in the base image, the class prompt may be set to "rabbit," and the instance prompt may be set to "cpc," which is an identification character string that does not match general words, etc. For example, if a "bear character" appears as an object in the base image, the class prompt may be set to "bear", and the instance prompt may be set to "usu" as an identification character string.

[0132] It is possible to avoid storing class prompts in association with instance prompts.

[0133] Other model information may include, for example, the model name associated with the instance prompt, the name and description of the object that can be generated by the model (e.g., "rabbit character"), information about all or part of the base image used for additional learning, and example images generated by applying the instance prompt to the additional learning model.

[0134] The stamp image generation prompt management data 157 is data for managing a prompt template for generating a stamp set image, and an example of the data configuration of stamp image generation prompt management data 157A is shown in FIG. 1-6. In the stamp image generation prompt management data 157A, for example, a prompt template ID, a positive template prompt, and other information are stored in association with each other.

[0135] The prompt template ID is information used to identify a template of a prompt used for generating a stamp image, and, for example, a unique value (proper value) is set and stored by the server 10 for each template.

[0136] The positive template prompt is, for example, a character string that is a template for a prompt and is set by a user of the server 10. For example, the positive template prompt includes information for indicating the replacement position of the instance prompt and the class prompt (for example, in this figure, " <x>"). In addition, at least some of the prompts (tokens) may be set to be different in the positive template prompts of different prompt template IDs. In this way, different prompts may be generated from positive template prompts specified by different prompt template IDs, and as a result, different images may be generated.

[0137] For example, a negative template prompt may be further stored in association with the prompt template ID. The negative template prompt is, for example, a character string that is a template for a negative prompt, which is set by a user of the server 10. For example, the negative template prompt may include a prompt such as "text, lower, error, NSFW,...", which is an element that is preferably not included in order to generate an image suitable for a stamp image.

[0138] For example, when a stamp template ID, an instance prompt, and a class prompt are specified, the control unit 11 of the server 10 creates a prompt (which may be called a "positive prompt") that includes, for example, the instance prompt, the class prompt, and a positive template prompt. For example, the positive template prompt " <x>In the " " part, an instance prompt and a class prompt may be applied to generate a positive prompt. Also, a negative prompt including a negative template prompt may be generated.

[0139] For example, in Figure 1-6, if the instance prompt is specified as "cpc" and the class prompt is specified as "rabbit," when the prompt template ID "T0001" is set, the prompt created will be "illustration of cpc rabbit, best quality, full shot, ...".

[0140] The incremental learning model may then generate images based on the prompts.

[0141] The additional learning model may generate images based on positive and negative prompts.

[0142] Also, unlike this example, the positive template prompts and negative template prompts common to each stamp template may be stored as common positive prompts and common negative prompts without being associated with the stamp template ID. Template prompts with different contents and other registered information may be stored in association with the stamp template ID. Note that the common positive prompts and common negative prompts may not be stored. In this case, for example, when a stamp template ID, an instance prompt, and a class prompt are specified, the control unit 11 of the server 10 creates, for example, a positive prompt including the instance prompt, the class prompt, the template prompt, and a common positive prompt, and a negative prompt including the common negative prompt.

[0143] The stamp set management database 159 is, for example, a database for managing information related to stamp sets created by users, and an example of the data configuration thereof is shown in FIG. 1-7. Stamp set management database 159 stores stamp set management data as management data for each created stamp set.

[0144] In each stamp set management data, for example, a stamp set ID, a stamp set name, a creator application ID, other stamp set information, and stamp management data are stored in association with each other.

[0145] The stamp set ID is information for identifying a created stamp set, and for example, when a stamp set is registered, a unique value (proper value) is set and stored for each stamp set by the server 10.

[0146] The stamp set name is the name of the stamp set, and for example, the name set by the user when registering the stamp set is stored.

[0147] The creator application ID is information for identifying the user who created this stamp set, and as an example, the application ID of the user who created the stamp set, which is registered in the account registration data 153, is stored. If the user who created the stamp set is a user of the server 10 (the administrator of the service), identification information indicating the administrator may be stored in the creator application ID.

[0148] Other stamp set information may be capable of storing various information, such as, for example, a description of the stamp set, the selling price of the stamp set, identification information of the additional learning model used to create the stamp set, and information regarding the seed value used when generating the image of the additional learning model.

[0149] The stamp management data is data for managing information about each stamp included in the stamp set. For example, the stamp management data stores a stamp ID, stamp text, and stamp image URI in association with each other.

[0150] The stamp ID is information for identifying a stamp in a stamp set, and for example, a unique value (for example, a serial number) is set and stored for each stamp by the server 10. For example, an individual stamp (stamp image) can be specified by specifying the stamp set ID and the stamp ID.

[0151] The stamp text is a character string associated with the stamp image. The stamp text may be text that represents the meaning of the stamp image. The stamp text may be a character string that serves as a query key used when calling up the stamp image. The stamp text may be text contained in the stamp image.

[0152] The stamp image URI is information indicating the storage location of the created stamp image. For example, when a stamp image is created, the server 10 stores the location where the image file of the stamp image is saved.

[0153] (2) Functional configuration of the terminal FIG. 1-8 is a diagram showing an example of functions realized by a control unit 21 of a terminal 20 in this embodiment. The control unit 21 includes, as a functional unit, an application processing unit 211 for executing application processing according to an application processing program 281 stored in the storage unit 28, for example.

[0154] FIG. 1-9 is a diagram showing an example of data stored in the storage unit 28 of the terminal 20 in this embodiment. In the storage unit 28, for example, an application processing program 281 to be executed as application processing, and an application ID 283 corresponding to this terminal 20 or the account of the user of the terminal 20 are stored.

[0155] <Display screen> Examples of display screens will be described below. The transition of the display screens described below is merely one example of the transition of the display screens for realizing the method of the present disclosure. In the transition of the display screens exemplified below, the display of some of the display screens may be omitted, or other display screens may be added. Furthermore, the terms used on the display screens described below may differ from the terms used in the processes described thereafter (some terms may not be consistent).

[0156] In the following, as an example, the terminal 20 is a smartphone equipped with a display unit 24 of a vertically long display. In the smartphone, as an example, a touch panel functioning as an input unit is arranged opposite the display, forming a touch screen. When an element such as an icon, button, item, or input area is displayed on the display, and an area that is a part of the touch panel and faces the area where the element is displayed is operated by a user, a program associated with the element or a subroutine of the program may be executed.

[0157] In addition, in one drawing or across multiple drawings, a flow of screen transition may be explained by showing a specific screen displayed on a different terminal 20. In this case, for example, a specific screen may be displayed on a different terminal 20 based on at least one of the following (triggered by at least one of the following): Automatically displayed A notification (including a push notification, etc.; the same applies below) is sent to the terminal 20, and is displayed when the user performs an operation in response to the notification. No notification is sent to the terminal 20, but it is displayed when the user performs a specific operation on the application. A notification is sent to the terminal 20, and the message is displayed when the user performs a specific operation on the application that sent the notification.

[0158] In the following, a case will be illustrated in which the terminal 20A of user AA is used as an example of an account for creating a stamp set. In this example, the stamp set is created within a messaging application, and the name of the messaging application is shown as "Messaging App." It should be noted that the stamp set may be created on a stamp set creation application other than the messaging application, for example. Even in this case, the same screen configuration may be used.

[0159] 1-10 to 1-12 are diagrams showing an example of a screen displayed on the display unit 24 of the terminal 20A in this embodiment. In this example, a stamp set image is generated by a user AA of the terminal 20A using an image of a pet dog, a Pomeranian, captured in advance.

[0160] The left side of Fig. 1-10 is a home screen of a messaging application (not limited to this, but an example of a chat application), and the name of the messaging application is displayed in the center of the top of the screen, and the icon image and user name of the messaging application of the user of this terminal 20A are displayed in the right top of the screen.

[0161] Below that is an area showing the current position, page, etc. within the messaging application (hereinafter collectively referred to as the "in-app position display area"). In this example, the word "Home" is displayed because this is the home screen.

[0162] Below the in-app location display area is a friend / group list display area that displays a list of friends and groups in the messaging application. In this example, the friend / group list display area is configured to display a friend list for displaying a list of friends of the account of this terminal 20, and a group list for displaying a list of groups that include the account of this terminal 20 as a member.

[0163] Below the friend / group list display area is a service list display area that displays a list of services provided within the messaging application or that can be used in conjunction with the messaging application. In this example, the service list display area displays a "Stamp Set Creation" service for creating stamp sets, a "Dress Up" service for changing designs in chat rooms, etc.

[0164] For example, when the "Create stamp set" service button is tapped by the user, the display changes to the stamp set creation screen shown in the center of Figure 1-10.

[0165] As an example, this stamp set creation screen is configured to display a thumbnail image setting area for setting a thumbnail image for the stamp set to be created, and a stamp set name setting area for setting a title (stamp set name) below the in-app position display area. Below that, a stamp set description setting area for setting a description of the stamp to be created is displayed.

[0166] Below the stamp set description setting area, an individual stamp editing area STR1 is displayed for creating and editing the individual stamps (stamp images) that make up the stamp set. In this image, individual stamps have not been created in the individual stamp editing area STR1, so the frames representing the eight stamps are displayed with dotted lines. For example, you can create or edit individual stamps within a stamp set by tapping the boxes representing the individual stamps.

[0167] Below the individual stamp editing area STR1 are displayed an "Automatic stamp set generation" button SGB1 for generating stamp images in bulk and automatically creating a stamp set, and a "Stamp set creation complete" button SRB1 for registering the created stamp set. On this screen, for example, the information necessary for registering a stamp set has not been entered in the individual stamp editing area STR1 or the stamp set name setting area, so for example, the “Stamp set creation complete” button SRB1 is displayed in a grayed-out state and its function is disabled.

[0168] For example, when the "Automatic stamp set generation" button SGB1 is tapped by the user, the display changes to a stamp set automatic generation method selection screen shown on the right side of FIG. 1-10.

[0169] This stamp set automatic generation method selection screen displays, as an example, that the method of generating a stamp image can be selected by, for example, using a trained additional learning model prepared by the user or administrator, or by registering a new base image and creating an additional learning model. This screen displays that the available additional learning models can be selected from a "rabbit character" (as an example, an additional learning model with additional learning model ID "M0001") and a "bear character" (as an example, an additional learning model with additional learning model ID "M0002").

[0170] The additional learning model selectable on the stamp set automatic generation method selection screen may be set as follows. (M1) Additional learning model created by the service operator (M2) Additional learning model created by the user in the past (M3) Additional learning model created in the past by a user who belongs to the same group as the user (M4) Additional learning model created by any user in the past (M5). Any combination of (M1) to (M4)

[0171] Also, for example, a trained additional learning model may be made unselectable on the stamp set automatic generation method selection screen.

[0172] This screen also displays that the method of registering a base image can be selected from "select an image from an album" and "take a photo." When "select an image from an album", for example, image data stored in the storage unit 28 may be specified as the base image. In the case of "taking a photograph", for example, image data captured by the imaging unit 27 may be designated as the base image.

[0173] For example, when the user taps the "Select image from album" button, the display changes to the base image selection screen shown on the left side of Figure 1-11. This screen is configured to display an album image selection area below the in-app position display area for selecting an image to be used as a base image from images in an album. Also, below the album image selection area, a "Select Image" button SLB1 is displayed for selecting an image selected in the album image selection area (for example, an image that has been tapped to give it a check mark) as a base image.

[0174] For example, when nine images of Pomeranians are selected in the album image selection area and the "image selection" button SLB1 is tapped, the control unit 11 of the server 10 sets an instance prompt and a class prompt based on the selected base image. Then, for example, the image generation model unit 113 executes a stamp image generation model additional learning process so that an image close to the base image is generated in association with the instance prompt and the class prompt.

[0175] Then, for example, the display changes to the base image learning progress display screen shown in the center of Figure 1-11. On this screen, for example, a list of images selected in the album image selection area is displayed below the message "Learning base image." Also, below that, a progress bar showing the progress of additional learning is displayed. Below the progress bar, for example, a "Stop Learning" function button for stopping the additional learning process is displayed.

[0176] At the bottom of the base image learning progress display screen, a "stamp set image generation" button IGB1 for generating a stamp set image using the trained additional learning model is displayed. On this screen, since the additional learning process has not yet ended, the "Generate stamp set image" button IGB1 is displayed in a grayed-out state, and the function is disabled.

[0177] For example, when time has passed and the additional learning process is completed, the display changes to the base image learning progress display screen shown on the right side of FIG. 1-11. On this screen, the “Stop learning” function button disappears and the message “Learning completed” is displayed. Also, the display state of the “Create stamp set image” button IGB1 changes, and the function is enabled.

[0178] For example, the base image learning progress display screen may be configured to be able to exit to the main menu or the like during the additional learning process. Also, for example, the base image learning progress display screen may be configured to be able to return from the stamp set creation screen or the like.

[0179] Also, when the additional learning process is completed, an additional learning completion notification indicating that the learning is completed may be sent to the user who registered the base image from the OA of the stamp set creation application, for example. Then, for example, the screen may be returned to the base image learning progress display screen based on a tap operation on the additional learning completion notification.

[0180] For example, when the "Generate stamp set image" button IGB1 is tapped by the user, the display changes to, for example, the base image learning progress display screen shown on the left side of FIG. 1-12. In this screen, for example, a stamp number selection area NSR1 for selecting the number of stamp images to be generated is displayed in a raised manner, superimposed on the base image learning progress display screen after the additional learning process is completed. In the stamp number selection area NSR1, for example, selection buttons for "8" to "40" are displayed in a row as selectable numbers of stamp images to be generated. Note that, for example, in the stamp number selection area NSR1, the user may input and specify the number of stamp images to be generated.

[0181] For example, when the user taps the "N sheets (here, "N" = "8")" selection button for the number of sheets to be generated, for example, stamp image generation processing is executed based on a positive template prompt specified by any N prompt template IDs from the stamp image generation prompt management data 157A, and N stamp images are generated. Then, for example, the display changes to the generated stamp set image display screen shown in the center of FIG. 1-12.

[0182] On this screen, a specified number of generated stamp images are displayed side-by-side in a generated stamp image confirmation display area below the message "Stamp set images have been generated." Below the generated stamp image confirmation display area are displayed side by side a “Save trained AI model” button MSB1 for saving an additional learning model that has undergone additional learning processing based on the base image, a “Regenerate image” button RGB1 for regenerating the stamp image, and a “Confirm” button IAB1 for setting the stamp set image displayed in the generated stamp image confirmation display area as a stamp set.

[0183] For example, when the "Save trained AI model" button MSB1 is tapped, a record may be added to the stamp image additional learning model management data 155, and the additional learning model may be stored in association with the instance prompt and class prompt set during learning. When the additional learning process is completed, the additional learning model may be automatically stored in association with the stamp image additional learning model management data 155. By saving the additional learning model, users can generate stamp images based on their favorite base images by simply calling up the additional learning model, thereby saving the effort and time required for re-learning.

[0184] For example, when the "regenerate image" button RGB1 is tapped, for example, the seed value may be reset in the additional learning model, and the stamp image generation process may be executed again. Also, for example, when the "regenerate image" button RGB1 is tapped, for example, the stamp image generation process may be executed again based on a positive template prompt specified by a prompt template ID different from the one specified previously. This allows image generation to be repeated until an image desired by the user is generated.

[0185] As an example, it may be possible to individually select, by tapping or the like, a stamp image to be regenerated or a stamp image to be saved as is without being regenerated from among the multiple stamp images displayed in the generated stamp image confirmation display area.

[0186] Also, for example, when the "regenerate image" button RGB1 is tapped, the stamp number selection area NSR1 may be displayed again, allowing the user to re-specify the number of stamp images included in the stamp set.

[0187] In addition, in the stamp image generation process, "M" images, which is more than the designated "N", may be generated. For example, the user may be allowed to select "N" stamp images to be used as stamp set images from among the "M" stamp images displayed in the generated stamp image confirmation display area.

[0188] For example, when the "OK" button IAB1 is tapped by the user, the display changes to the stamp set creation screen shown on the right side of FIG. 1-12. In this screen, for example, the stamp set image confirmed in the generated stamp image confirmation display area is applied and displayed in the individual stamp editing area STR1. Also, for example, the first image of the generated stamp images is automatically set and displayed in the thumbnail image setting area.

[0189] In addition, the stamp set name setting area and the stamp set description setting area may be configured to automatically input title and description templates based on the results of image recognition (image description analysis) of the base image and the generated stamp set image, for example.

[0190] Also, by tapping an individual stamp image displayed in the individual stamp editing area STR1, it may be possible to correct or modify the stamp image or to input stamp text.

[0191] For example, when the "Stamp set creation complete" button SRB1 is tapped, stamp management data is added to the stamp set management database 159 and each data is stored.

[0192] In addition, in the stamp management data, a part or all of the positive template prompt used in generating the stamp image may be reflected and stored as the stamp text.

[0193] <Processing> 1-13 is a flowchart showing an example of the flow of processing executed by each device in this embodiment. From the left, this figure shows an example of processing executed by the control unit 21 of the terminal 20A of the user AA, and an example of processing executed by the control unit 11 of the server 10. Note that the processes described below are merely examples of processes for implementing the method of the present disclosure, and the present disclosure is not limited to these. Furthermore, another step may be added to the process described below, or some steps may be omitted (deleted) from the process described below.

[0194] First, the control unit 21 of the terminal 20A transmits stamp set image generation request information for generating a stamp set image to the server 10 via the communication I / F 22 based on, for example, an input to the input / output unit 23 of the terminal 20A (for example, a user input (user operation input, sound input, etc.); the same applies below) (A110).

[0195] When stamp set image generation request information is received from terminal 20A via communication I / F 14, the control unit 11 of the server 10 sends base image set transmission request information to terminal 20A via communication I / F 14, requesting the transmission of a base image including one or more additional learning images (S110).

[0196] When the base image set transmission request information is received from the server 10 via the communication I / F 22, the control unit 21 of the terminal 20A outputs the received base image set transmission request information. In the following, as an example of information output, an example of displaying the information on the display unit 24 is illustrated, but the control unit 21 of the terminal 20A may also, for example, output the received information as sound from the sound output unit 26.

[0197] Based on the user input in response to the output base image set transmission request information, the control unit 21 of the terminal 20A transmits base image set information including one or more pieces of image information to the server 10 via the communication I / F 22 (A120).

[0198] When the base image set information is received from each terminal 20 via the communication I / F 14, the control unit 11 of the server 10 executes an instance prompt setting process (S120). In the instance prompt setting process, the control unit 11 of the server 10 randomly generates a character string of a predetermined length (for example, three characters) and compares it with a natural language dictionary (not shown) (for example, an English word dictionary) stored in the storage unit 15. If the generated character string is not included in the natural language dictionary, the control unit 11 sets the generated character string as the instance prompt.

[0199] In the instance prompt setting process, the control unit 11 of the server 10 may select an instance prompt from preset instance prompt candidate character strings, for example.

[0200] In addition, in the instance prompt setting process, the control unit 11 of the server 10 may, for example, refer to the stamp image additional learning model management data 155 and set the instance prompt so that it does not overlap with an instance prompt that has already been stored.

[0201] Then, the control unit 11 of the server 10 executes a class prompt setting process (S130). In the class prompt setting process, the control unit 11 of the server 10 analyzes the base image received from the terminal 20 using an image identification unit (not shown), and sets a character string corresponding to the category of the base image as the class prompt. Note that the image recognition unit may be configured with an image classification model such as Vision Transformer or DeiT (Data-efficient image Transformers), for example.

[0202] When a class prompt is set, the control unit 11 of the server 10 generates a predetermined number of regularized images (for example, "the number of images of the base image x 10") using the image generation model unit 113 with the set class prompt as a prompt. As an example, if the class prompt is "dog", the regularized images will be images of dogs of any breed.

[0203] In addition, if the base prompt is not used in the stamp image generation model additional learning process or the stamp image generation process, the base prompt setting process may not be executed.

[0204] For example, when an instance prompt or a class prompt for a base image is set, the control unit 11 of the server 10 executes a stamp image generation model additional learning process (S140).

[0205] In the stamp image generation model additional learning process, the control unit 11 of the server 10 generates a pre-additional learning model in which the structure and parameters are copied based on the image generation model unit 113, for example. Then, the control unit 11 of the server 10 additionally learns a base image by DreamBooth, for example, using instance prompts, class prompts, and regularized images for the pre-additional learning model. DreamBooth may be realized by using the method described in "arXiv:2208.12242", for example. This makes it possible to create an additional learning model that can generate an image by reflecting objects such as characters and concepts common to the base image in the instance prompt.

[0206] In addition, in the stamp image generation model additional learning process, the control unit 11 of the server 10 may, for example, additionally learn the base image by performing fine tuning on the model before additional learning without using the regularized image.

[0207] In addition, in the stamp image generation model additional learning process, the control unit 11 of the server 10 may additionally learn the base image by applying LoRA (Low-Rank Adaptation) in DreamBooth or fine tuning, for example. LoRA may be realized by using the method described in "arXiv:2106.09685", for example. This can reduce the resources required for additional learning and shorten the learning time. In addition, when saving the additional learning model, all parameters of the additional learning model need to be saved when the entire model is fine-tuned, whereas in LoRA, only the difference parameters from the pre-learning model need to be saved. This also has the advantage of allowing the additional learning model to be saved efficiently.

[0208] For example, when the stamp image generation model additional learning process is completed, the control unit 11 of the server 10 transmits additional learning completion information indicating that an additional learning model reflecting (learning) the base image has been created to the terminal 20A via the communication I / F 14 (S150). Note that the control unit 11 of the server 10 may add the additional learning model that has learned the base image to the stamp image additional learning model management data 155 and store it.

[0209] When the additional learning end information is received from the server 10 via the communication I / F 22, the control unit 21 of the terminal 20A outputs (displays on the display unit 24, etc.) the received additional learning end information (A130).

[0210] For example, based on user input, the control unit 21 of the terminal 20A receives stamp set image number selection information, which is information regarding the number of stamp images included in a stamp set, and transmits the stamp set image number selection information to the server 10 via the communication I / F 22 (A140).

[0211] When the stamp set image number selection information is received from the terminal 20A via the communication I / F 14, the control unit 11 of the server 10 executes a stamp image generation prompt set setting process (S160). In the stamp image generation prompt set setting process, the control unit 11 of the server 10 randomly selects prompt template IDs for the number of stamp images to be generated, as specified in the stamp set image number selection information. Note that the prompt template IDs may be selected in ascending or descending order, as an example. The control unit 11 of the server 10 then generates prompts (positive prompts) for the number of stamp images to be generated, based on the positive template prompts linked to the prompt template IDs and the instance prompts and class prompts used in learning the additional learning model.

[0212] In the stamp image generation prompt set setting process, the control unit 11 of the server 10 may select a prompt template ID from a set of predetermined prompt template IDs for each number of stamp images to be generated, which is specified in the stamp set image number selection information. This makes it possible to prevent the generation of similar prompts.

[0213] In the stamp image generation prompt set setting process, the control unit 11 of the server 10 may set a prompt set based on, for example, a positive template prompt linked to a prompt template ID and a randomly generated or selected class prompt. <x>In the " portion, a pre-set common noun (such as "apple" or "banana") may be randomly selected and applied. In this case, the base image may include common elements such as common objects, or may not include common elements.

[0214] Then, the control unit 11 of the server 10 executes a stamp image generating process (S170). In the stamp image generation process, the control unit 11 of the server 10, for example, inputs a prompt to the additional learning model and acquires the generated image as a stamp image. This process is repeated the number of times corresponding to the number of stamp set images. The inference process in the incremental learning model may be executed, for example, by the image generation model unit 113. The inference process in the incremental learning model may be executed, for example, by the control unit 11 of the server 10.

[0215] Then, the control unit 11 of the server 10 transmits stamp set image information including the generated designated number of stamp images to the terminal 20A via the communication I / F 14 (S180).

[0216] When the stamp set image information is received from the server 10 via the communication I / F 22, the control unit 21 of the terminal 20A, for example, causes the display unit 24 to display the received stamp set image information (A150).

[0217] Then, the control unit 21 of the terminal 20A judges whether or not to end the process (A190). If it is judged that the process should be continued (A190: NO), the control unit 21 of the terminal 20A returns the process to A110, for example. On the other hand, if it is judged that the process should be ended (A190: YES), the control unit 21 of the terminal 20A ends the process.

[0218] The control unit 11 of the server 10 also determines whether or not to end the process (S190). If it is determined that the process should be continued (S190: NO), the control unit 11 of the server 10, for example, waits to receive stamp set image generation request information and returns the process to S110. On the other hand, if it is determined that the process should be ended (S190: YES), the control unit 11 of the server 10 ends the process.

[0219] As an example, when it is determined that regeneration of the stamp set image has been requested based on user input for the displayed stamp set image information, the control unit 21 of the terminal 20A may be configured to send stamp set image regeneration request information requesting regeneration of the stamp set image to the server 10 via the communication I / F 22.

[0220] When it is determined that stamp set image regeneration request information has been received from terminal A via communication I / F 14, the control unit 11 of the server 10 may, for example, re-execute the stamp image generation process and re-send the stamp set image information. When re-executing the stamp image generating process, the control unit 11 of the server 10 may change the seed value. Also, the stamp image generating process may be executed by a prompt based on a different prompt template ID.

[0221] In S180, the control unit 11 of the server 10 may transmit the stamp set image information to a terminal 20 different from the terminal 20A via the communication I / F 14. The terminal 20 different from the terminal 20A may be a terminal 20 linked to the account of the user AA, or may be a terminal 20 linked to the account of a user different from the user AA. Moreover, the stamp set image information may be transmitted to both the terminal 20A and the terminal 20 different from the terminal 20A. Furthermore, the destination of the stamp set image information including the generated stamp image may be set to any one or more terminals 20 based on user input at terminal 20A, for example and not by way of limitation.

[0222] <Effects of the First Embodiment> In this embodiment, the server 10 (not limited to any one example of an information processing device) acquires a base image (not limited to any one example of a first image set) for additional learning for the image generation model unit 113 (not limited to any one example of an image generation device that generates an image based on a prompt). Then, the server 10 causes the image generation device to learn the base image. The server 10 also sets a prompt set in a stamp image generation prompt set setting process. Then, the server 10 shows a configuration in which, in the stamp image generation process, the server 10 causes an additional learning model (not limited to any one example of an image generation device that generates an image based on a prompt) to generate a stamp set image (not limited to any one example of a second image set). As an example of an effect of the embodiment obtained by such a configuration, the information processing device can acquire a first image set and cause the image generating device to learn the first image set, and then generate a second image set by the image generating device that has learned the first image set.

[0223] Furthermore, the server 10 can receive the first image set from the terminal 20A (not limited to this, an example of a first information processing device) and transmit the generated second image set to a terminal 20 (not limited to this, an example of a second information processing device) other than the first information processing device or the terminal 20A. Therefore, the information processing device can transmit the second image set to any information processing device.

[0224] In addition, in this embodiment, the server 10 acquires a base image (not limited, but an example of a first image set) composed of an image including a Pomeranian (not limited, but an example of a first object). Then, the server 10 causes the image generating device to learn the base image based on an instance prompt or a class prompt (not limited, but an example of first information) associated with the base image. Also, the server 10 sets a prompt set, which is a plurality of prompts based on the first information, in a stamp image generation prompt set setting process. Then, the server 10 causes an additional learning model (not limited, but an example of an image generating device that generates an image based on a prompt) to generate a stamp set image (not limited, but an example of a second image set) in a stamp image generation process. Therefore, the stamp set image shows a configuration in which it is generated as an image set composed of images based on a Pomeranian. As an example of an effect of an embodiment obtained by such a configuration, an information processing device can acquire a first image set consisting of images including a first object, and cause an image generating device to generate a second image set consisting of images based on the first object.

[0225] This embodiment also shows an example of a configuration in which the second image set is a stamp image set, which allows a user of the information processing device to easily create a stamp image set composed of images based on the first object.

[0226] <First Modification (1)> In the above embodiment, the server 10 unconditionally accepts the base image transmitted by the user of the terminal 20, but is not limited to this. For example, if the base image is not suitable for the additional learning process, a different base image may be selected. Also, for example, if a stamp image generated by an additive learning model trained on a base image is likely to cause copyright infringement, a different base image may be selected.

[0227] FIG. 1-14 is a diagram showing an example of a screen displayed on the display unit 24 of the terminal 20A in this modified example.

[0228] The left side of Figure 1-14 is an example of a base image selection screen similar to the left side of Figure 1-11. Here, for example, it is assumed that three photos are selected as base images. For example, when the "Select Image" button SLB1 is tapped, the display changes to the base image reset request screen shown in the center of Figure 1-14.

[0229] This screen is configured so that, for example, a list of images selected in the album image selection area is displayed below a warning message saying "Please reselect base image." Below that, a base image condition display area is configured to display the predetermined conditions under which the base image can be accepted and whether or not the selected base image satisfies the predetermined conditions. In the base image condition display area, for example, if the predetermined conditions are not met, the characters are displayed in a grayed-out display mode and a round "OK" icon is added. Also, if the predetermined conditions are met, the characters are displayed in black and a cross "NG" icon is added.

[0230] In this screen, for example, three predetermined conditions are displayed in the base image condition display area. Also, it is displayed that, of the three predetermined conditions, the condition regarding the number of base images is met.

[0231] Here, the predetermined condition may be set, for example, as follows: The predetermined condition may be set by the server 10 based on a setting input by a user of the server 10, or may be set automatically by the server 10, for example. (C-1) When the number of base images required for additional learning is small. For example, if the number of base images is less than a set number (e.g., 8 images), there is a high possibility that additional learning will fail, so a warning may be displayed to encourage the user to increase the number of base images. Note that a specific number of images, such as "Please select X more base images," may be displayed in the base image condition display area. The process of determining this specified condition may, for example, determine whether the number of base images is less than or equal to a set number, and if the number is less than or equal to the set number, the specified condition is met.

[0232] (C-2) When different learning objects are included in the base image. For example, if one base image contains multiple objects (different characters or subjects) or if different images in the base image contain different objects, additional learning is likely to fail, so a warning may be displayed to encourage reselection of the base image. Note that an image containing multiple objects in the base image or multiple images containing different objects may be displayed in the base image condition display area. The process of determining this specified condition may, for example, be such that a base image is identified by an image identification unit, and if dissimilar objects of different classes (e.g., "dog" and "cat") are detected from the base image, the specified condition is satisfied.

[0233] (C-3) When the base image contains an image that infringes copyright. For example, if one or more images in the base image contain an illustration of a character or a subject that infringes copyright as an object, an image generated by the additional learning model is likely to contain a strong characteristic of an object that infringes copyright, so a warning may be displayed to encourage the user to reselect the base image. Images that may infringe copyright in the base image may be displayed in the base image condition display area. The process of determining the predetermined condition may, for example, use an image recognition model that has been trained on images of copyrighted works to construct a copyrighted work determination filter capable of determining whether or not a copyrighted work is included in an arbitrary image. Then, for example, when a base image is selected by the copyrighted work determination filter and determined to include a copyrighted work, the predetermined condition may be satisfied. Similarly, a predetermined condition may be set so that images that are offensive to public order and morals (for example, pornographic images) are not accepted as base images.

[0234] (C-4). Any combination of (C-1) to (C-3).

[0235] For example, when the "Reselect Image" button SLB2 at the bottom of the screen for reselecting the base image is tapped, the display changes to the base image selection screen shown on the right side of Figure 1-14, for example, and the base image can be reselected.

[0236] 1-15 is a flowchart showing an example of the flow of processing executed by each device in this modified example. From the left, the diagram shows an example of processing executed by the control unit 21 of the terminal 20A of the user AA, and an example of processing executed by the control unit 11 of the server 10.

[0237] For example, when base image set information is received from the terminal 20A via the communication I / F 14, the control unit 11 of the server 10 determines whether or not the base image satisfies a predetermined condition based on the received base image set information (S113). The method of determining whether or not the predetermined condition exists may be, for example, performed according to the above (C-1) to (C-4).

[0238] When it is determined that the images in the base image set satisfy the predetermined condition (S113: YES), the control unit 11 of the server 10 transmits base image set resetting request information for requesting reselection of a base image, etc. to the terminal 20 via the communication I / F 14 (S115). Then, the control unit 11 of the server 10 waits to receive the base image set information, for example.

[0239] When it is determined that the images in the base image set do not satisfy the predetermined condition (S113: NO), the control unit 11 of the server 10 executes the processes from S120 onwards.

[0240] When it is determined that base image set resetting request information has been received from the server 10 via the communications I / F 22 (A123: YES), the control unit 21 of the terminal 20A outputs (e.g., displays on the display unit 24) the received base image set resetting request information (A125).Then, the control unit 21 of the terminal 20A transmits base image set information including the reselected base image to the server 10 (A120).

[0241] When it is determined that the base image set resetting request information has not been received (A123: NO), the control unit 21 of the terminal 20A waits to receive additional learning end information, for example.

[0242] In addition, a part or all of the determination as to whether an image in the base image satisfies a predetermined condition may be executed by the control unit 21 of the terminal 20A. For example, the predetermined condition (C-1) may be determined by the terminal 20, and the conditions (C-2) and (C-3) may be determined by the server 10.

[0243] By making such a determination, it is possible to not accept base images that are difficult to appropriately perform additional learning on, and it is also possible to prevent images that are not suitable for presentation to an unspecified number of users from being generated from the additional learning model.

[0244] This modification shows an example of a configuration in which, when the first element satisfies a set condition, the information processing device performs control not to learn the first image set. This makes it possible to prevent the image generating device from learning the first element included in the first image set due to the set condition. As a result, it is possible to prevent the image generating device from generating an image including the first element that satisfies the set condition.

[0245] In addition, such a configuration can prevent images that include elements that are subject to copyright infringement or that are contrary to public order and morals from being generated in the second image set. Also, it can prevent the first image set from being improperly learned due to the first image set having an inappropriate number of images or the first image set including multiple elements.

[0246] In addition, this modification shows an example of a configuration in which, when the first element satisfies a set condition, the information processing device controls a notification to the first information processing device. In this way, when the first element satisfies the set condition, it is possible to notify the user of the first information processing device that the set condition is satisfied.

[0247] Furthermore, the information processing device can train the image generating device using the third image set by prompting the user of the first information processing device to transmit a third image set different from the first image set.

[0248] Furthermore, with this configuration, it is possible to notify the user of the first information processing device that the first image set contains images that are subject to copyright infringement or that contain elements that are contrary to public order and morals, and that additional learning of the first image set may not be performed properly.

[0249] <First Modification (2)> In the above embodiment, the class prompt is automatically set based on the base image, but the present invention is not limited to this. For example, when it is difficult to distinguish the class prompt from the base image, the class prompt may be set based on user input.

[0250] FIG. 1-16 is a diagram showing an example of a screen displayed on the display unit 24 of the terminal 20A in this modified example. The left side of Figure 1-16 is an example of a base image selection screen similar to the left side of Figure 1-11. When the "Select Image" button SLB1 is tapped, the display changes to the class prompt selection display screen shown in the center of Figure 1-16.

[0251] This screen is configured to display, for example, a list of images selected in the album image selection area below the message "Selected base image." Also, below that, a message "Please select an appropriate word that is common to the base images" is displayed, and a class prompt selection area is displayed to allow the user to select a class prompt from class prompt candidates.

[0252] In this screen, for example, a sofa is captured in the image of a Pomeranian, so the class prompt candidates "dog," "animal," and "sofa" are displayed side by side in the class prompt selection area. As an example, the class prompt candidates may be character strings that correspond to the categories of base images that have the top "Y" scores (for example, "Y" = "3") from the base image analysis results in the image recognition unit.

[0253] For example, when the class prompt "dog" is tapped and selected in the class prompt selection area, the display changes to the base image learning progress display screen shown on the right side of Figure 1-16. In addition to the message "Learning base image," this screen also displays that the class prompt being used in learning is "dog."

[0254] If the class prompt does not properly reflect the object of the base image (for example, the class prompt for "image of a Pomeranian" is set to "cat"), a regularized image is generated based on the class prompt, and the regularized image is likely to be an image far removed from the features common to the base image. This may result in the learning not converging easily in the stamp image generation model additional learning process, making it impossible to properly learn. In addition, there is a possibility that a stamp image that properly reflects the base image will not be generated from the additional learning model after learning (an image different from the object common to the base image will be generated). By making it possible to set class prompts appropriately, additional learning can be carried out smoothly, and more natural stamp set images can be generated.

[0255] The class prompt selection display screen may be displayed when the difference in the scores is small in the base image analysis results in the image recognition unit. Also, the class prompt may be input by the user.

[0256] <First Modification (3)> In the above embodiment, the process related to the additional learning of the base image is executed in the server 10, but this is not limited to the above. For example, a part or all of the process related to the additional learning of the base image may be executed in the terminal 20.

[0257] For example, when the control unit 21 of the terminal 20 receives a base image, the control unit 21 of the terminal 20 may execute an instance prompt setting process. Then, the control unit 21 of the terminal 20 may transmit base image set information including the instance prompt to the server 10.

[0258] Also, for example, when the control unit 21 of the terminal 20 receives a base image, the control unit 21 may execute an instance prompt setting process and a class prompt setting process. Then, the control unit 21 of the terminal 20 may transmit base image set information including the instance prompt and the class prompt to the server 10.

[0259] Also, for example, when the control unit 21 of the terminal 20 receives a base image, the control unit 21 may execute an instance prompt setting process, a class prompt setting process, and a stamp image generation model / additional learning process. Then, the control unit 21 of the terminal 20 may transmit additional learning model information including the additional learning model, the instance prompt, and the class prompt to the server 10. Then, the control unit 11 of the server 10 may execute a stamp image generation process based on the received additional learning model information.

[0260] For example, a part or all of the processing related to stamp image generation may be executed in the terminal 20.

[0261] For example, when the control unit 11 of the server 10 executes the stamp image generation model additional learning process, the control unit 11 may transmit additional learning model information including the additional learning model, the instance prompt, and the class prompt to the terminal 20. Then, the control unit 21 of the terminal 20 may execute the stamp image generation prompt set setting process and the stamp image generation process based on the received additional learning model information.

[0262] Also, for example, when the control unit 11 of the server 10 executes the stamp image generation model additional learning process, the control unit 11 may transmit additional learning model information including the additional learning model, the instance prompt, and the class prompt to the terminal 20. Then, the control unit 11 of the server 10 may execute the stamp image generation prompt set setting process and transmit the set prompt set information to the terminal 20. The control unit 21 of the terminal 20 may execute the stamp image generation process based on the received additional learning model information and prompt set information.

[0263] Furthermore, when the control unit 21 of the terminal 20 executes the stamp image generation prompt set setting process, the control unit 21 may transmit the set prompt set information to the server 10. Then, the control unit 11 of the server 10 may execute the stamp set image generation process based on the received prompt set information.

[0264] By executing part or all of the processing of the server 10 on the terminal 20, the load on the server 10 can be reduced.

[0265] <First Modification (4)> In the above embodiment, the stamp set image can be generated from a messaging application, but the present invention is not limited to this. For example, the stamp set image may be generated by using the stamp set creation application described above.

[0266] In this case, the server 10A providing the stamp set creation support service and the server 10B providing the messaging service may be separate servers.

[0267] The control unit 11A of the server 10A may include, for example, an image generation model unit 113. The storage unit 15A of the server 10A may store, for example, stamp image additional learning model management data 155 and stamp image generation prompt management data 157.

[0268] For example, a stamp set image may be created using a stamp set creation application, and when the stamp set is registered, stamp set information regarding the stamp set including the stamp set image may be transmitted from server 10A to server 10B. Upon receiving the stamp set information, server 10B may, for example, add a record to stamp set management database 159 and register the stamp set.

[0269] The generated stamp set images may be used as an image set for use in a stamp set, or may be exported from a stamp set creation application, for example, so that they can be used for other purposes.

[0270] The stamp set creation application may also be considered as an independent application that generates an image set based on a predetermined prompt set using an incremental learning model that has been incrementally trained on a base image.

[0271] <First Modification (5)> In the above embodiment, the additional learning model learns one base image (base image set), but is not limited to this. For example, different instance prompts may be set for multiple base images, and additional learning may be performed.

[0272] For example, set the instance prompt "cpc" to a base image that includes a "rabbit character" object, and set the instance prompt "usu" to a base image that includes a "bear character" object. Then, set a class prompt to each object, and execute the stamp image generation prompt set setting process. For example, when a stamp image generation prompt set setting process is executed using the instance prompts "cpc and usu" for an additional learning model that has been additionally trained with two types of base images, and then a stamp image generation process is executed, an image including a "rabbit character and a bear character" is generated as a stamp image.

[0273] That is, a stamp set image including a plurality of objects may be generated by assigning different instance prompts to a plurality of base images including different objects and performing additional learning.

[0274] When a plurality of base images are additionally learned, for example, an instance prompt (instance token) assigned to each object included in the instance prompt may be specified based on a user input of the terminal 20. Also, an instance token may be indirectly specified by selecting a base image associated with the instance token or a stamp image generated from the instance token.

[0275] <First Modification (6)> In the above embodiment, the stamp image is a still image, but the present invention is not limited to this. For example, the stamp image may be a moving image. When generating a moving image as a stamp image, the image generation model unit 113 may be configured, for example, with a Text-to-Video model (trained neural network model) such as Imagen Video or Make-A-Video.

[0276] In addition, the base image is not limited to a still image. For example, when the base image is a moving image, the control unit 11 of the server 10 acquires the base image, executes a scene analysis process, and acquires a still image of an important scene in the moving image. Then, for example, the acquired still image may be used as a learning image (which may be a new base image) for an additional learning model to execute an instance prompt setting process, etc. When the image generation model unit 113 is a Text-to-Video model, an additional learning model may be created based on a base image that is a moving image.

[0277] <Second Example> The second embodiment is an embodiment in which a stamp image is generated based on the stamp text, for example. The contents described in the second embodiment are similarly applicable to any of the other embodiments and other modified examples.

[0278] FIG. 2-1 is a diagram showing an example of functions realized by the control unit 11 of the server 10 in this embodiment. The control unit 11 includes, for example, an application management processing unit 111, an image generation model unit 113, and a language model unit 115 as functional units.

[0279] For example, the language model unit 115 has a function of generating a prompt for generating a stamp image in the image generation model unit 113 based on an input keyword (such as a stamp text). For example, the language model unit 115 may be configured with a large-scale language model such as HyperCLOVA (registered trademark), GPT-4, or PaLM. Note that the language model unit 115 may be configured with a small-scale language model with a smaller number of parameters, specialized for learning related to prompt generation.

[0280] Note that language model unit 115 (not limiting, but an example of a language generation device that generates a prompt based on the first prompt) is not limited to being provided in server 10. As a non-limiting example, the language generation device may be provided outside server 10 (not limiting, but an example of an information processing device). And as a non-limiting example, the processing in this embodiment may proceed by communication between the language generation device and the information processing device.

[0281] FIG. 2-2 is a diagram showing an example of the data configuration of the stamp image generation prompt management data 157B in this embodiment. In the stamp image generation prompt management data 157B, for example, a prompt template ID, a stamp text template, and other information are stored in association with each other.

[0282] The prompt template ID and other information may be, for example, similar to the stamp image generation prompt management data 157A.

[0283] The stamp text template is, for example, a character string that is a template for stamp text and is set by a user of the server 10, or the stamp text itself that is set in the stamp. Note that different text may be set for stamp text of different prompt template IDs.

[0284] For example, when a stamp template ID, an instance prompt, and a class prompt are specified, the control unit 11 of the server 10 generates a prompt to be input to the image generation model unit 113 by the language model unit 115, for example. In the following, as an example, a prompt for input to language model unit 115 is referred to as a "prompt for creating an image generation prompt", and a prompt for input to image generation model unit 113 is referred to as a "prompt for image generation".

[0285] For example, the control unit 11 of the server 10 first generates a prompt for creating a prompt for image generation using a prompt template for creating a prompt for image generation based on the stamp template ID, the instance prompt, and the class prompt.

[0286] The prompt template for creating a prompt for image generation is, for example, a template set by a user of the server 10, and may be, for example, the following template: " <Rules> Create a prompt that instructs the image generation model to generate an illustration. Please create a list of English keywords for the prompt from the text you will enter. Use only adjectives, verbs, and nouns in your keyword list. The first keyword is "*Instance Prompt* *Class Prompt*". <Output> *Instance prompt* *Class prompt*, keyword, keyword,(loop) <Input> *Stamp text* "

[0287] For example, *instance prompt* and *class prompt* are replaced with the specified instance prompt and class prompt, and *stamp text* is replaced with the stamp text associated with the stamp template ID, which may be determined based on user input.

[0288] When the prompt for generating a prompt for image generation is generated, the control unit 11 of the server 10 generates a prompt for image generation based on the prompt for generating a prompt for image generation, for example, by using the language model unit 115. If the instance prompt is specified as "cpc", the class prompt is specified as "rabbit", and the stamp text is "thanks", the generated image generation prompt will be, for example, "cpc rabbit, expressing thanks, best quality, ...".

[0289] For example, a common positive prompt or a common negative prompt may be stored, and a prompt template for creating a prompt for image generation may be set to generate a prompt for image generation incorporating the common positive prompt or the common negative prompt.

[0290] <Display screen> 2-3 to 2-5 are diagrams showing examples of screens displayed on the display unit 24 of the terminal 20A in this embodiment.

[0291] The left side of Figure 2-3 is, for example, a stamp set creation screen similar to the center of Figure 1-10. When the "Automatically generate stamp set" button SGB2 is tapped on this screen, for example, the display changes to the stamp set creation screen in the center of Figure 2-3.

[0292] On this screen, a stamp number selection area NSR2 for selecting the number of stamp images to be generated is configured to be displayed by rising from the bottom of the screen. Note that, for example, the number of stamp images to be generated may be input and specified by the user in the stamp number selection area NSR2.

[0293] For example, when the user taps the "N" (here, "N" = "8") button for selecting the number of stamps to be generated in the stamp number selection area NSR2, stamp text templates designated by any N prompt template IDs are selected from the stamp image generation prompt management data 157B. Then, for example, the display changes to the stamp set creation screen shown on the right side of FIG. 2-3.

[0294] For example, this screen is configured so that stamp text assigned in the stamp text template to the number of stamps specified in the stamp number selection area NSR2 is displayed in the individual stamp editing area STR1. For example, since no stamp image has been set for each stamp, the "Stamp set creation complete" button SRB1 is still displayed in a grayed-out state, and its function is disabled. Also, for example, since stamp text has been set for each stamp, the "Automatic stamp set generation" button SGB2 is displayed as a "Generate stamp set image" button SGB3.

[0295] Unlike this example, when the number of stamps to be generated selection button is tapped in the number of stamps selection area NSR2, for example, a stamp text setting area may be displayed in which a list of stamp texts including some or all of the stamp text templates is displayed. In the stamp text setting area, for example, stamp text may be set in the individual stamp editing area STR1 based on an arbitrary stamp text template selected by the user.

[0296] For example, when a user taps a stamp with the stamp text "Bye-bye" set in the individual stamp editing area STR1, the display changes to the stamp set creation screen shown on the left side of FIG. 2-4.

[0297] In this screen, for example, a stamp editing area SER1 for specifying editing operations for individual stamps is configured to be displayed rising from the bottom. As an example, the stamp editing area SER1 is configured to display a "Use Album Image" button for specifying the method for setting the stamp image, a "Take Photo" button, and a "Draw Illustration" button below which is a "Text Edit" button for setting (editing) the stamp text.

[0298] For example, when the “Edit Text” button is tapped by the user, the display changes to the stamp text editing screen shown in the center of Figure 2-4. In this screen, for example, a character string "Leave it to me", which is changed by the user from the character string "Bye-bye" set by the stamp text template, is input and displayed in a text editing area TER1 for setting the character string of the stamp text to be set in the stamp. Also, a stamp text placement setting area for setting the position where the stamp image and stamp text are superimposed is displayed above the text editing area TER1. In the stamp text placement setting area, for example, the position where the stamp text and stamp image are superimposed may be set by dragging the stamp text "Leave it to me". The size, color, font, etc. of the characters of the stamp text displayed in the stamp text layout setting area may be set.

[0299] As an example, if a word included in the NG word list related to a prompt that generates an image inappropriate for a stamp image is entered into the text editing area TER1, a display may be displayed requesting the user to re-enter the stamp text, and a different string may be accepted again.

[0300] For example, when the stamp text is edited in the stamp editing area SER1 and the placement of the stamp text is set in the stamp text placement setting area, when the “Confirm” button SEB1 is tapped, for example, the edited content on the stamp text editing screen is reflected, and for example, the display changes to the stamp set creation screen shown on the right side of Figure 2-4.

[0301] This screen displays that the stamp text of the stamp that was set to "Bye-bye" in the individual stamp editing area STR1 has been changed to "Leave it to me."

[0302] For example, when the "Generate stamp set image" button SGB3 is tapped by the user, the display changes to the stamp set automatic generation method selection screen shown on the left side of FIG. 2-5. This screen is, for example, the same as the screen shown on the right side of FIG. 1-10. When the "Select image from album" button is tapped by the user on this screen, a base image is accepted, and a stamp image generation model additional learning process is executed based on the base image, and an additional learning model is generated. When the additional learning model is generated, for example, a prompt for image generation is generated based on the stamp text set for each stamp, the instance prompt set for the base image, and the class prompt. Then, a stamp image is generated by inputting the prompt for image generation into the additional learning model. When the stamp images for the number of sheets selected in the stamp number selection area NSR2 are generated, the display changes to the generated stamp set image display screen shown in the center of FIG. 2-5.

[0303] On this screen, a generated stamp image confirmation display area displays a specified number of stamp images generated by an additional learning model in response to a prompt based on the stamp text.

[0304] In addition, in the generated stamp image confirmation display area, the set stamp text may be displayed so as to overlap with the generated stamp image.

[0305] For example, when the "OK" button IAB1 is tapped by the user, the display changes to the stamp set creation screen shown on the right side of FIG. 2-5. In this screen, for example, the stamp set image confirmed in the generated stamp image confirmation display area is applied and displayed in the individual stamp editing area STR1. For example, the set stamp text is superimposed on each stamp image and displayed. In the following, an image in which the stamp text is superimposed on the stamp image generated by the additional learning model (an image rendered by superimposing the stamp text on the generated image of the additional learning model) may be referred to as a "stamp image with text".

[0306] The rendering position and character color of the stamp text in the stamp image with text may be determined based on information set by the user in the stamp text placement setting area. Also, the character color of the stamp text may be determined by LayoutGAN or the like based on the stamp image generated by the additional learning model and the set stamp text, and the stamp text may be appropriately placed on the stamp image with text.

[0307] <Processing> 2-6 is a flowchart showing an example of the flow of processing executed by each device in this embodiment. In this figure, from the left, an example of processing executed by the control unit 21 of the terminal 20A of the user AA and an example of processing executed by the control unit 11 of the server 10 are shown.

[0308] For example, when the stamp set image number selection information is sent to the server 10 via the communication I / F 22, the control unit 21 of the terminal 20A accepts stamp text for the number of stamp images based on a user input, for example. Then, the control unit 21 of the terminal 20A transmits stamp text set information including the accepted stamp text to the server 10 via the communication I / F 22 (A210). The stamp text may be one word, or may be composed of two or more words.

[0309] For example, when the control unit 11 of the server 10 receives stamp set image number selection information from the terminal 20A via the communication I / F 14, the control unit 11 of the server 10 selects prompt template IDs for the number of stamp images to be generated specified in the stamp set image number selection information, for example, randomly. Then, the control unit 11 of the server 10 may transmit default stamp text set information including a stamp text template linked to the prompt template ID to the terminal 20A via the communication I / F 14. When receiving the default stamp text set information from the server 10, the control unit 21 of the terminal 20A may be configured to display the received default stamp text set information on the display unit 24. Then, the control unit 21 of the terminal 20A may be configured to accept stamp text based on a user input for the displayed default stamp text set information.

[0310] The control unit 11 of the server 10 may be configured not to transmit the default stamp text set information to the terminal 20. In other words, each stamp text may be determined by the server 10.

[0311] When the stamp text set information is received from the terminal 20A via the communication I / F 14, the control unit 11 of the server 10 executes a stamp image generation prompt generation process (S210).

[0312] In the stamp image generation prompt generation process, the control unit 11 of the server 10 generates, for example, a prompt for creating an image generation prompt for the number of stamp images to be generated based on the stamp text contained in the received stamp text set information and the instance prompt and class prompt used in learning the additional learning model.

[0313] Then, the control unit 11 of the server 10 inputs, for example, a prompt for creating an image generation prompt to the language model unit 115, and generates image generation prompts for the number of stamp images to be generated.

[0314] Then, the control unit 11 of the server 10 executes a stamp image generating process (S170). In the stamp image generation process, the control unit 11 of the server 10, for example, inputs a prompt for image generation to the additional learning model, and acquires the generated image as a stamp image. This process is repeated the number of times corresponding to the number of stamp set images.

[0315] Then, the control unit 11 of the server 10 executes a stamp text synthesis process (S220). In the stamp text synthesis process, the control unit 11 of the server 10 outputs, for example, a stamp image with text obtained by synthesizing the generated stamp image with a character string image of the stamp text used in generating the image generation prompt.

[0316] Then, the control unit 11 of the server 10 transmits stamp set image information including the generated images of the specified number to the terminal 20A via the communication I / F 14 (S180). The image included in the stamp set image information may be a stamp image with text, or may be a stamp image generated by applying an image generation prompt to an additional learning model. As an example, the stamp set image information may include a stamp image and a stamp image with text. As an example, the stamp set image information may include stamp text used to generate the image generation prompt, linked to the stamp image or the stamp image with text.

[0317] <Effects of the second embodiment> This embodiment shows an example of a configuration in which an information processing device acquires a stamp text template or stamp text set information (not limited to, but an example of first text information) and sets a prompt set based on an instance prompt or class prompt (not limited to, but an example of first information) and the first text information. This allows the acquired first text information to be reflected in the prompt set, and as a result, the first text information can be reflected in the second image set generated by the image generating device.

[0318] Furthermore, by receiving the first image set and the first text information from the first information processing device, a second image set including the first object of the first image set and generated based on the first text information can be transmitted to the first information processing device or the second information processing device, so that a user of the first information processing device or the second information processing device can easily obtain a second image set including the first object desired by the user and reflecting the first text information intended by the user.

[0319] Furthermore, by setting a prompt for creating an image generation prompt (not limited to, but an example of the first prompt) based on the first information and the first text information, and having the language model unit 115 (not limited to, but an example of a language generation device) generate an image generation prompt (not limited to, but an example of a prompt set) based on the first prompt, the information processing device can cause the image generation device to generate images based on a prompt set that is more effective. Therefore, it can be expected that the generated second image set will be images of higher quality.

[0320] In addition, by synthesizing character images based on the first text information with the images that make up the second image set, the user can easily understand what text each image in the second image set reflects.

[0321] In addition, in this embodiment, the image generation model unit 113 (not limited to this, but an example of an image generation device that generates an image based on a prompt) shows an example of a configuration that generates a stamp image (not limited to this, but an example of an image) without receiving a prompt input from a user. That is, this embodiment shows the configuration of an image generation device that generates an image without receiving a prompt input from a user. This allows the image generating device to generate an image without accepting a prompt input from the user. As a result, it is possible to prevent the image generating device from accepting an inappropriate prompt from the user. This makes it possible to prevent the user from using the image generating device in an inappropriate manner, thereby improving the robustness of the image generating device.

[0322] <Second Modification Example (1)> As an example, a stamp text template in the stamp image generation prompt management data 157B may store a collection of stamp texts (stamp text set) corresponding to the number of stamps, and the stamp text for generating different stamp images may be set so as not to include words that would result in the generation of similar stamps. That is, when a prompt template ID is specified and a stamp text template is determined, a stamp text set for generating a stamp set image may be determined. Then, a prompt for creating a prompt for generating each image may be generated based on the stamp text set, and then each image generating prompt may be generated.

[0323] This makes it possible to generate stamp set images with less overlap and richer variation.

[0324] For example, the purpose, etc. of the stamp set generated for each stamp text template (for example, a "greeting stamp set" or an "apology stamp set") may be defined, and a stamp text set may be set according to the purpose, etc. Then, for example, a prompt template ID may be designated based on the result of selection of the purpose, etc. by the user of the terminal 20.

[0325] <Second Modification Example (2)> In the above embodiment, the prompt for creating each image generation prompt is generated based on the stamp text, etc., but the present invention is not limited to this. For example, a feature prompt that indicates the features of the image to be generated may be set in the prompt for creating each image generation prompt.

[0326] FIG. 2-7 is a diagram showing an example of a screen displayed on the display unit 24 of each terminal 20 in this modified example. The left side of Figure 2-7 is an example of a base image learning progress display screen similar to the right side of Figure 1-11. For example, when the "Generate stamp set image" button IGB2 is tapped, the display changes to the base image learning progress display screen in the center of Figure 2-7.

[0327] As an example, this screen is configured to superimpose on the base image learning progress display screen after the additional learning process is completed, and to display a stamp image style selection area TSR1 that allows the user to select the style of the stamp image to be generated. In the stamp image style selection area TSR1, for example, selection buttons such as "Realistic", "SF", and "Manga" are displayed side by side as selectable stamp image styles.

[0328] For example, the user may be allowed to input and specify the style of the stamp image in the stamp image style selection area TSR1. In this case, for example, if the input style is a word included in the NG word list that is not suitable for a stamp image, a display indicating that the style cannot be accepted may be displayed on the terminal 20A.

[0329] For example, when the selection buttons for "real" and "cute" are selected and the "OK" button is tapped, for example, in the stamp image generation prompt generation process, the following is input to the prompt template for creating the image generation prompt: " Create a prompt that instructs the image generation model to generate an image. Please create a prompt so that the generated image reflects the style indicated in the *Feature Prompt*. " The template may be used such that the *feature prompt* portion is replaced with one or more styles selected in the stamp image style selection area TSR1.

[0330] For example, a prompt for creating a prompt for image generation is generated based on the above template, and a prompt for image generation is also generated. Then, a stamp image generation process is executed based on the prompt for image generation, and N stamp images are generated. Then, for example, the display changes to the generated stamp set image display screen shown on the right side of Figure 2-7. On this screen, a stamp image reflecting the style selected in the stamp image style selection area TSR1 is generated and displayed in the generated stamp image confirmation display area.

[0331] For example, when the "regenerate image" button RGB2 is tapped, the stamp image style selection field TSR1 may be displayed again so that a different style can be selected.

[0332] Also, the feature prompt may be directly written as a positive prompt in the image generation prompt.

[0333] In addition, the feature prompt may allow the selection of a background (for example, a "flower field" or a "night sky"), an object the character possesses (for example, a "bouquet" or a "hat"), an object the character rides on (for example, a "car" or a "rocket"), etc. Also, information indicating the season or time (for example, "spring" or "summer", "morning" or "night", etc.) for determining the overall impression of the stamp image may be made selectable as a feature prompt.

[0334] In this way, by allowing the user to specify the style and the like in the prompt for creating the image generation prompt or the image generation prompt, it is possible to help the user generate the stamp image he or she desires more easily. It is also possible to prevent the user from directly embedding a positive prompt in the image generation prompt, which may result in the generation of an image that is not suitable as a stamp image.

[0335] <Second Modification Example (3)> In the above embodiment, an example of generating a stamp set image consisting of a plurality of stamp images has been described, but the present invention is not limited to this. For example, the stamp image to be generated may be an image of a single stamp, or a single image itself.

[0336] 2-8 and 2-9 are diagrams showing examples of screens displayed on the display unit 24 of the terminal 20A in this modified example. On the stamp set creation screen on the left side of Figure 2-8, for example, when the user taps the "+" icon for creating a single stamp in the individual stamp editing area STR1, the display changes to the stamp set creation screen in the center of Figure 2-8.

[0337] In this screen, for example, a stamp editing area SER2 for specifying editing operations for individual stamps is configured to be displayed rising from the bottom. When the user taps the "Generate Image" button in the stamp editing area SER2, the display changes to the stamp text editing screen shown on the right side of Figure 2-8, for example. On this screen, the character string "OK" entered by the user is displayed in the text editing area TER2. When the "OK" button SEB2 is tapped, the edited content on the stamp text editing screen is reflected, and the display changes to the stamp set automatic generation method selection screen shown on the left side of FIG. 2-9.

[0338] When the user taps the "Select image from album" button on this screen, a base image is accepted, and a stamp image generation model additional learning process is executed based on the base image, and an additional learning model is generated. When the additional learning model is generated, a prompt for image generation is generated based on, for example, the stamp text set in the stamp, and the instance prompt and class prompt set in the base image. Then, a stamp image is generated by inputting the prompt for image generation into the additional learning model. Then, for example, the display changes to the generated stamp image display screen shown in the center of Figure 2-9.

[0339] On this screen, a stamp image (in this example, a stamp image with text) generated by the additional learning model is displayed in a generated stamp image confirmation display area in response to a prompt based on the input stamp text. For example, when the "OK" button IAB2 is tapped by the user, the display changes to the stamp set creation screen shown on the right side of Figure 2-9. On this screen, for example, the stamp image confirmed in the generated stamp image confirmation display area is applied and displayed in the individual stamp editing area STR1.

[0340] 2-10 is a flowchart showing an example of the flow of processing executed by each device in this modified example. In this figure, from the left, an example of processing executed by the control unit 21 of the terminal 20A of the user AA and an example of processing executed by the control unit 11 of the server 10 are shown.

[0341] First, the control unit 21 of the terminal 20A transmits stamp image generation request information for generating a stamp image to the server 10 via the communication I / F 22 based on, for example, a user input (A200).

[0342] For example, when the additional learning end information is displayed (A130), the control unit 21 of the terminal 20A acquires stamp text that is the basis of the stamp image to be generated based on a user input, for example, and transmits stamp text information including the acquired stamp text to the server 10 via the communication I / F 22 (A220).

[0343] If the stamp text information includes a word included in the NG word list that is not suitable for a stamp image, the control unit 21 of the terminal 20A may cause the display unit 24 to display stamp text re-input request information that requests re-input of the stamp text. Then, the stamp text acquisition process may be executed again.

[0344] When the stamp text information is received from the terminal 20A via the communication I / F 14, the control unit 11 of the server 10 executes a stamp image generation prompt generation process based on the received stamp text (S210). Then, the control unit 11 of the server 10 executes a stamp image generation process based on the generated image generation prompt (S170).

[0345] For example, when the stamp text information includes a word included in the NG word list that is not suitable for a stamp image, the control unit 11 of the server 10 may transmit stamp text re-input request information to the terminal 20A, which requests re-input of the stamp text. Then, the control unit 21 of the terminal 20A may display the received stamp text re-input request information on the display unit 24, and execute step A220 again.

[0346] For example, when the control unit 11 of the server 10 executes the stamp text synthesis process (S220), the control unit 11 transmits stamp image information including the generated stamp image and the stamp image with text to the terminal 20A via the communication I / F 14 (S230).

[0347] When the stamp image information is received from the server 10 via the communication I / F 22, the control unit 21 of the terminal 20A causes the display unit 24 to display the received stamp image information (A230).

[0348] In this modification, the server 10 (not limited to, but an example of an information processing device) acquires a base image (not limited to, but an example of a first image set) consisting of an image including a Pomeranian (not limited to, but an example of a first object) for additional learning for the image generation model unit 113 (not limited to, but an example of an image generation device that generates an image based on a prompt). Then, the server 10 makes the image generation device learn the first image set based on an instance prompt or a class prompt (not limited to, but an example of the first information) associated with the first image set. In addition, when the server 10 acquires a stamp text (not limited to, but an example of the first text information), the server 10 sets a stamp image generation prompt (not limited to, but an example of a second prompt) based on the first information and the first text information. Then, in the stamp image generation process, the server 10 makes the additional learning model (not limited to, but an example of an image generation device that generates an image based on a prompt) generate a stamp image (not limited to, but an example of a second image). Therefore, the second image shows a configuration in which an image based on a Pomeranian is generated. As an example of an effect of a modified example obtained by such a configuration, an information processing device can acquire a first image set consisting of images including a first object and first text information, and cause an image generating device to generate a second image, which is an image based on the first object and reflects the first text information.

[0349] <Second Modification Example (4)> In the above embodiment, the stamp image generation prompt generation process is executed in the server 10. However, the present invention is not limited to this. For example, the stamp image generation prompt generation process may be executed in the terminal 20.

[0350] For example, the control unit 11 of the server 10 may transmit additional learning end information including the instance prompt and the class prompt used in the stamp image generation model additional learning process to the terminal 20A. Then, when the control unit 21 of the terminal 20A receives stamp texts for the number of stamp images, the control unit 21 may generate image generation prompt creation prompts for the number of stamp images to be generated based on the received stamp texts and the instance prompts and the class prompts. Then, the control unit 21 of the terminal 20A may generate a prompt for image generation based on the generated prompt for creating an image generation prompt, for example, by a language model unit (not shown). Then, the control unit 21 of the terminal 20A may transmit image generation prompt set information including the generated image generation prompt to the server 10.

[0351] The control unit 21 of the terminal 20A may transmit prompt set information for creating a prompt for image generation, including a prompt for creating a prompt for image generation, to the server 10. The control unit 11 of the server 10 may input a prompt for creating a prompt for image generation based on the prompt set information for creating a prompt for image generation to the language model unit 115, and generate a prompt for image generation.

[0352] As described above, part or all of the learning of the base image and the image generation using the additional learning model may be executed in the terminal 20A.

[0353] <Second Modification (5)> In the above embodiment, the control unit 11 of the server 10 outputs a stamp image with text by combining the generated stamp image with a character string image of the stamp text in the stamp text combining process, but this is not limited to this. As a non-limiting example, the control unit 11 of the server 10 may combine a character string image different from the stamp text with the stamp image in the stamp text combining process.

[0354] By way of example and not limitation, the control unit 11 of the server 10 inputs a prompt to the language model unit 115 as follows: " Please rephrase the *stamp text* in different words. " Then, the character string image of the output sentence obtained may be combined with the stamp image.

[0355] <Third Example> The third embodiment is an embodiment in which, as an example, in a messaging service, a stamp image is generated in real time based on input text and sent to a chat room. The contents described in the third embodiment are similarly applicable to any of the other embodiments and other modified examples.

[0356] In this embodiment, an example of the functions realized by the control unit 11 of the server 10 may be configured similarly to that shown in FIG. 2-1.

[0357] FIG. 3-1 is a diagram illustrating an example of information stored in the storage unit 15 of the server 10 in this embodiment. In the storage unit 15, for example, an application management processing program 151 and account registration data 153 are stored.

[0358] <Display screen> 3-2 to 3-3 are diagrams showing examples of screens displayed on the display unit 24 of the terminal 20A in this embodiment. In this example, it is assumed that images including a rabbit character, a bear character, and the like have been learned by the image generation model unit 113. However, it is assumed that the learned rabbit character and bear character have not been associated with an instance prompt. The left side of FIG. 3-2 shows an example of a chat room screen between user AA and user BB, who is a friend of user AA.

[0359] On this talk room screen, the current location display area indicates that this is a one-to-one talk room with "user BB." In addition, below that, a talk room content display area TR1 is configured to be displayed for displaying content sent and received within this talk room.

[0360] In the talk room content display area TR1, for example, comments from user AA of the terminal 20A, which is the terminal itself, are displayed in balloons on the right side, and comments from other users are displayed in balloons on the left side as content.

[0361] The chat room content display area TR1 displays a text message CT1 sent by user AA about looking forward to camping, and a text message CT2 sent by user BB about reserving a campsite.

[0362] Below the chat room content display area TR1, a content input area for inputting and selecting content to send to this chat room is displayed. In this screen, for example, the text message input area MIR1 in the content input area is selected by the user, and the character string "Thank you" is entered and displayed.

[0363] For example, when a user taps the stamp image generation button SCB1 indicated by a face icon displayed on the right side of the text message input area MIR1, for example, a stamp image generation process is executed based on the character string entered in the text message input area MIR1. Then, for example, the display changes to the chat room screen shown on the right side of Figure 3-2.

[0364] This screen is configured to display a stamp image generation progress indicator display indicating that a stamp image is being generated in the talk room content display area TR1. For example, by checking the stamp image generation progress indicator display, the user of the terminal 20A can understand that a stamp image is being generated and the progress of the generation process. For example, the stamp image generation process may be interrupted by tapping an image generation cancel button indicated by an "x" mark on the upper right side of the stamp image generation progress indicator. This makes it possible to prevent the conversation from being delayed due to the time it takes to generate the stamp image.

[0365] For example, when the stamp image generation process is completed, the display changes to the chat room screen shown on the left side of Figure 3-3. In this screen, for example, a generated stamp list information display area SSR1 for displaying a stamp list based on a generated stamp image is displayed above the content input area. The generated stamp list information display area SSR1 is configured to display a predetermined number (for example, "five") of stamps including stamp images generated by the image generation model unit 113 based on the character string "Thank you", for example. Note that the character string "Thank you" input in the text message input area MIR1 may be superimposed on each stamp displayed in the generated stamp list information display area SSR1.

[0366] For example, when a user taps a rabbit character stamp in the generated stamp list information display area SSR1, the display changes to the chat room screen shown on the right side of Figure 3-3. On this screen, the stamp content SCT1 selected in the generated stamp list information display area SSR1 is transmitted to and displayed in the talk room content display area TR1.

[0367] Although a one-to-one talk room has been illustrated in this example, stamps may be sent in a similar manner in a group talk room or an OA talk room.

[0368] <Processing> FIG. 3-4 is a flowchart showing an example of the flow of processes executed by each device in this embodiment. In this figure, from the left, an example of a process executed by the control unit 21 of the terminal 20A, an example of a process executed by the control unit 21 of the terminal 20B, and an example of a process executed by the control unit 11 of the server 10 are shown.

[0369] First, the control unit 21 of the terminal 20A transmits chat request information for requesting to chat with the user of the terminal 20B via a talk room to the server 10 via the communication I / F 22 based on a user input, for example. When the chat request information is received from the terminal 20A via the communication I / F 14, the control unit 21 of the server 10 executes a chat start process for conducting a chat between the terminal 20A and the terminal 20B via the talk room (S310). In the chat start process, the control unit 11 of the server 10 generates, for example, a talk room ID for identifying the talk room, and transmits the generated talk room ID to each terminal 20 participating in the talk room via the communication I / F 14.

[0370] For example, when a talk room ID is received from server 10 via communication I / F 22, control unit 21 of terminal 20A and control unit 21 of terminal 20B execute chat start processing to conduct a chat via the talk room between terminal 20A and terminal 20B (A310-B310). In the chat start process, the control unit 21 of each terminal 20 causes the display unit 24 to display a talk room screen, for example.

[0371] The chat request information may be transmitted from the terminal 20B to the server 10.

[0372] For example, when sending a message to a talk room is selected based on an input to the input / output unit 23 of the terminal 20B, the control unit 21 of the terminal 20B may transmit content transmission request information including, for example, the talk room ID and the text of the message to be sent to the server 10 via the communication I / F 22. Note that the control unit 21 of the terminal 20A may transmit the content transmission request information to the server 10.

[0373] When content transmission request information is received from the terminal 20 via the communication I / F 14, the control unit 11 of the server 10 may transmit content information (e.g., text content information, image content information, stamp content information, etc.) based on the content transmission request information to each terminal 20 in the talk room. When the content information is received from the server 10 via the communication I / F 22, the control unit 21 of the terminal 20 may, for example, display the received content information in the talk room content display area.

[0374] For example, the control unit 21 of the terminal 20A accepts text content (text message) to be sent to the talk room based on an input to the input / output unit 23 (A320). Then, for example, when generation of a stamp based on the text content (character string) is selected based on an input to the input / output unit 23, the control unit 21 of the terminal 20A sends stamp text information including the input text content to the server 10 via the communication I / F 22 (A220). The text content in the stamp text information may be referred to as stamp text.

[0375] When the stamp text information is received from the terminal 20A via the communication I / F 14, the control unit 11 of the server 10 executes a stamp image generation prompt generation process (S330). In the stamp image generation prompt generation process, the control unit 11 of the server 10 generates a prompt for creating an image generation prompt based on the following prompt template for creating an image generation prompt, for example. " <Rules> Create a prompt that instructs the image generation model to generate an illustration. Please create a list of English keywords for the prompt from the text you will enter. Use only adjectives, verbs, and nouns in your keyword list. <Output> keyword, keyword, (loop) <Input> *Stamp text* "

[0376] For example, the stamp text included in the stamp text information (for example, "Thank you") is substituted for *stamp text*.

[0377] Then, the control unit 11 of the server 10 generates a prompt for image generation, for example, by using the language model unit 115, based on the prompt for creating a prompt for image generation.

[0378] Then, the control unit 11 of the server 10 uses the generated image generation prompt to repeat the stamp image generation process (S170) and the stamp text synthesis process (S220) a predetermined number of times (a set number of times) while changing the seed value, for example. In the stamp image generation process, the server 10 may, for example, execute an inference process based on the image generation prompt by the image generation model unit 113 that has not undergone additional learning. Note that the inference process in the image generation model that has not undergone additional learning may, for example, be executed by the control unit 11 of the server 10.

[0379] The control unit 11 of the server 10 may generate different prompts for image generation by repeating the stamp image generation prompt generation process, and repeat the stamp image generation process (S170) and the stamp text synthesis process (S220) a predetermined number of times. Also, the stamp text synthesis process does not have to be executed.

[0380] When a predetermined number of stamp images are generated by repeating the process a predetermined number of times, the control unit 11 of the server 10 transmits generated stamp list information including the generated predetermined number of stamp images to the terminal 20A via the communication I / F 14 (S350). The generated stamp list information may include stamp images with text. The generated stamp list information may also include information for identifying each stamp image (e.g., a temporary ID, etc.) and stamp text.

[0381] When the created stamp list information is received from the server 10 via the communication I / F 22, the control unit 21 of the terminal 20A causes the display unit 24 to display the received created stamp list information (A330). For example, when a specific stamp included in the generated stamp list information is selected based on user input to the generated stamp list information displayed on the display unit 24 (A340), the control unit 11 of the terminal 20A transmits generated stamp selection information including the temporary ID of the selected stamp, etc., to the server 10 via the communication I / F 22 (A350).

[0382] When the control unit 11 of the server 10 receives the generated stamp selection information from the terminal 20A via the communication I / F 14, the control unit 11 transmits stamp content information including the stamp image (or stamp image with text) specified in the received generated stamp selection information to each terminal 20 via the communication I / F 14 (S370). The stamp content information may include stamp text. The stamp content information may not include a stamp image or a stamp image with text. The stamp content information may include a prompt for image generation generated in the stamp image generation prompt generation process.

[0383] When the stamp content information is received from the server 10 via the communication I / F 22, the control unit 21 of the terminal 20 causes the display unit 24 to display the received stamp content information (A370 and B370).

[0384] Then, the control unit 21 of the terminal 20A judges whether or not to end the process (A390). If it is judged that the process should be continued (A390: NO), the control unit 21 of the terminal 20A returns the process to A310, for example. On the other hand, if it is judged that the process should be ended (A390: YES), the control unit 21 of the terminal 20A ends the process. The same may be applied to the terminal 20B.

[0385] The control unit 11 of the server 10 also determines whether or not to end the process (S390). If it is determined that the process should be continued (S390: NO), the control unit 11 of the server 10 returns the process to S310, for example. On the other hand, if it is determined that the process should be ended (S390: YES), the control unit 11 of the server 10 ends the process.

[0386] <Effects of the third embodiment> In this embodiment, the server 10, which executes processing related to a chat including user AA (not limited to, an example of a first user) of terminal 20A (not limited to, an example of a first terminal) and user BB (not limited to, an example of a second user) of terminal 20B (not limited to, an example of a second terminal), generates a predetermined number of stamp images (not limited to, an example of an image set) when it receives stamp text information (not limited to, an example of text information based on an input to the first terminal in which a chat room including the first user and the second user is displayed) from the first terminal. Then, the server 10 transmits generated stamp list information (not limited to, an example of the first information) to the first terminal. Then, the server 10 receives generated stamp selection information (not limited to, an example of the second information) and transmits stamp content information (not limited to, an example of the third information) based on the second information to the second terminal. This allows the first user to input text information into the chat room and check the image set generated based on the text information on the first terminal. Then, the first user can select an image that the first user desires from the image set on the first terminal and transmit the third information based on the selected image to the second terminal of the second user who is the other party in the chat room. This allows for an improvement and enhancement of the user experience when participating in a chat.

[0387] Furthermore, by generating an image set using an image generating device that generates images based on a prompt based on text information, the first user can easily express the image he or she wants to generate using the text information.

[0388] Furthermore, by having the language processing device generate the prompt, it is possible to obtain a prompt that enables the image generating device to generate a high-quality image.

[0389] <Third Modification (1)> In the above embodiment, the prompt for image generation is generated by the language model unit 115 based on the prompt for creating a prompt for image generation, but this is not limited to this. For example, the control unit 11 of the server 10 may generate a prompt for image generation with the stamp text in the stamp text information from the terminal 20A (which may be called a character string acquired by the terminal 20A) as a positive prompt.

[0390] For example, if the stamp text is "Thank you," the image generation prompt may be "Thanks" or "Thanks."

[0391] The image generation prompt may be generated by combining a common positive prompt or a common negative prompt with stamp text.

[0392] Also, a positive template prompt including information for indicating the replacement position of the stamp text may be set, and the stamp text may be applied to the positive template prompt to generate a prompt for generating an image.

[0393] In other words, the control unit 11 of the server 10 may not include the language model unit 115 .

[0394] <Third Modification (2)> For example, if the language model unit 115 determines that the image generation prompt is an inappropriate prompt based on the prompt for creating an image generation prompt, the control unit 11 of the server 10 may, for example, execute the stamp image generation prompt generation process again to generate a different image generation prompt.

[0395] Note that the determination of whether or not a prompt is inappropriate may be made, for example, by determining whether or not a word included in the NG word list is included as a positive prompt in the generated image generation prompt. Also, for example, a stamp image generated in an image generation prompt may be applied to a suitable image determination filter or the like to determine whether or not an inappropriate image may be generated.

[0396] <Third Modification (3)> In the above embodiment, when a prompt for image generation is generated in the stamp image generation prompt generation process, the stamp image generation process is executed, but this is not limited to this. For example, the storage unit 15 of the server 10 may store a combination of a previously generated prompt for image generation and a stamp image generated using the prompt for image generation. For example, when a prompt for image generation similar to a previously generated prompt for image generation is generated, the control unit 11 of the server 10 may not execute the stamp image generation process, but may use the previously generated stamp image as a generation result of the additional learning model.

[0397] The similarity of the image generation prompts may be determined based on, for example, the cosine similarity between the embedded expressions of the prompts, and the image generation prompts may be determined to be similar if the cosine similarity is greater than or equal to a threshold value (e.g., 0.7).

[0398] This makes it possible to omit the stamp image generation process using an additional learning model for similar prompts, thereby reducing resources and processing time, and by minimizing the time the user has to wait for operations, allowing users to use the service more comfortably.

[0399] <Third Modification (4)> In the above embodiment, the stamp image generation prompt generation process and the stamp image generation process are executed in the server 10, but this is not limited to the above. For example, the stamp image generation prompt generation process, the stamp image generation process, and some or all of the stamp text synthesis process may be executed in the control unit 21 of the terminal 20A.

[0400] For example, when the control unit 21 of the terminal 20A receives text content, the control unit 21 may execute a stamp image generation prompt generation process based on the received text content. Then, when the image generation prompt is generated, stamp image generation request information including the image generation prompt may be transmitted to the server 10. For example, when the control unit 11 of the server 10 receives stamp image generation request information from the terminal 20A, the control unit 11 may execute a stamp image generation process based on the received image generation prompt.

[0401] By having the terminal 20A execute some or all of the processing in the server 10, the load on the server 10 can be reduced.

[0402] <Third Modification (5)> In the above embodiment, in the generated stamp selection information, one stamp is selected on the terminal 20A based on the stamp image included in the generated stamp list information, and stamp content information is transmitted to the terminals 20 participating in the talk room, but this is not limiting. For example, in the generated stamp selection information, multiple stamp images may be selected from among the stamp images included in the generated stamp list information.

[0403] For example, when a first stamp image and a second stamp image are selected from the generated stamp list information, the control unit 11 of the server 10 may sequentially transmit first stamp content information including the first stamp image and second stamp content information including the second stamp image to the terminal 20. That is, for example, in a talk room, after a stamp based on the first stamp image is displayed, a stamp based on the second stamp image may be displayed.

[0404] The control unit 11 of the server 10 may transmit stamp content information including the first stamp image and the second stamp image to the terminal 20. That is, for example, a plurality of stamp images may be displayed at once in a talk room.

[0405] <Fourth Example> The fourth embodiment is an embodiment in which, for example, a stamp image is generated using an additive learning model created using a base image provided by the server 10 or a user of the terminal 20, and is sent to a talk room. The contents described in the fourth embodiment are similarly applicable to any of the other embodiments and other modified examples.

[0406] In this embodiment, an example of the functions realized by the control unit 11 of the server 10 may be configured similarly to that shown in FIG. 2-1.

[0407] FIG. 4-1 is a diagram illustrating an example of information stored in the storage unit 15 of the server 10 in this embodiment. The storage unit 15 stores, for example, an application management processing program 151, account registration data 153, and stamp image additional learning model management data 155.

[0408] <Display screen> 4-2 and 4-3 are diagrams showing examples of screens displayed on the display unit 24 of the terminal 20A in this embodiment. In this example, it is assumed that an additional learning model has already been created in which images including a rabbit character, a bear character, etc. have been additionally learned in the image generation model unit 113. It is also assumed that the learned rabbit character is associated with the instance prompt "cpc", the bear character is associated with the instance prompt "usu", and the pet Pomeranian is associated with the instance prompt "sks".

[0409] Each additional learning model may be independent for each object (instance prompt), or a plurality of objects may be additionally learned collectively in one additional learning model.

[0410] The left side of FIG. 4-2 shows an example of a chat room screen between user AA and user BB, who is a friend of user AA. For example, when the stamp image generation button SCB2 is tapped by the user, the display changes to the chat room screen shown in the center of Figure 4-2.

[0411] For example, this screen is configured so that an instance prompt selection area ISR1 for specifying an instance prompt in the stamp image generation process is displayed above the content input area. The instance prompt selection area ISR1 is configured to display, below the text "Please select an example stamp to generate," instance prompts that have been learned in the additional learning model and corresponding images.

[0412] Note that the image corresponding to the instance prompt may be, for example, one of the base images associated with the instance prompt, or one of the stamp images generated based on the instance prompt.

[0413] Furthermore, in the instance prompt selection area ISR1, for example, images corresponding to the following instance prompts may be displayed so as to be selectable. -An instance prompt of an additional learning model created by the stamp sender himself (in this case, user AA). -An instance prompt of the additional learning model created by the participants in the chat room (in this case, user AA and user BB). · An instance prompt of an additional learning model created by an operator of a messaging service (for example, a user of server 10). Any combination of the above.

[0414] For example, when the image of a Pomeranian showing the instance prompt "sks" is tapped by the user, a stamp image generation process is executed based on the character string entered in the text message input area MIR1 and the instance prompt selected in the instance prompt selection area ISR1. Then, for example, the display changes to the chat room screen shown on the right side of FIG. 4-2.

[0415] On this screen, the stamp image generation progress indicator progresses, and when the stamp image generation process is completed, the display changes to the chat room screen shown on the left side of Figure 4-3, for example. In this screen, the generated stamp list information display area SSR2 is configured to display a predetermined number of stamps (for example, "5 stamps") including stamp images generated by an additional learning model using the instance prompt "sks" based on the character string "Thank you". Note that the character string "Thank you" entered in the text message input area MIR1 may not be superimposed on each stamp displayed in the generated stamp list information display area SSR2.

[0416] For example, when the user taps the rightmost stamp in the generated stamp list information display area SSR2, the display changes to the chat room screen shown on the right side of FIG. 4-3. On this screen, the stamp content SCT2 selected in the generated stamp list information display area SSR2 is transmitted to and displayed in the talk room content display area TR1.

[0417] <Processing> FIG. 4-4 is a flowchart showing an example of the flow of processes executed by each device in this embodiment. In this figure, from the left, an example of a process executed by the control unit 21 of the terminal 20A, an example of a process executed by the control unit 21 of the terminal 20B, and an example of a process executed by the control unit 11 of the server 10 are shown.

[0418] First, the control unit 21 of the terminal 20A and the control unit 11 of the server 10 execute a stamp image additional learning process based on a user input to the terminal 20A, for example (A410-S410). The stamp image additional learning process may be executed, for example, in accordance with each step of A110 to A130 and S110 to S150 in FIG. 1-13.

[0419] In addition, in the stamp image additional learning process, the control unit 11 of the server 10 may, for example, read the base image set information according to the image set stored in the storage unit of the server 10 based on a user input to the server 10. In other words, the base image to be learned by the additional learning model may be provided by a service operator such as a user of the server 10.

[0420] In addition, in the stamp image additional learning process, the control unit 11 of the server 10 may cause the additional learning model to learn a plurality of base images and a plurality of instance prompts or class prompts associated with the base images.

[0421] As an example, when stamp text information is received from terminal 20A via communication I / F 14, the control unit 11 of the server 10 refers to the stamp image additional learning model management data 155 and sends stamp instance selection request information to terminal 20A via communication I / F 14 to select a registered instance prompt (S430).

[0422] When the stamp instance selection request information is received from the server 10 via the communication I / F 22, the control unit 21 of the terminal 20A causes the display unit 24 to display the stamp instance selection request information, for example. Then, for example, based on a user input in response to the displayed stamp instance selection request information, the control unit 21 of the terminal 20A transmits stamp instance information for specifying an instance prompt to the server 10 via the communication I / F 22. Note that the stamp instance information may include the instance prompt, or may include information for identifying the instance prompt.

[0423] When the stamp instance information is received from the terminal 20A via the communication I / F 14, for example, the control unit 11 of the server 10 executes a stamp image generation prompt generation process based on the stamp text given in the stamp text information and the instance prompt and class prompt specified in the stamp instance information (S210). Note that the control unit 11 of the server 10 may, for example, refer to the third modified example (1) and generate a prompt for image generation without using the language model unit 115.

[0424] Then, the control unit 11 of the server 10 uses the generated image generation prompt to repeat the stamp image generation process (S170) and the stamp text synthesis process (S220) a predetermined number of times while changing the seed value, for example.

[0425] The control unit 11 of the server 10 may generate different prompts for image generation by repeating the stamp image generation prompt generation process, and repeat the stamp image generation process (S170) and the stamp text synthesis process (S220) a predetermined number of times. Also, the stamp text synthesis process does not have to be executed.

[0426] Then, the control unit 11 of the server 10 transmits generated stamp list information including the generated predetermined number of stamp images to the terminal 20A via the communication I / F 14 (S350).

[0427] <Effects of the Fourth Embodiment> In this embodiment, the server 10 obtains a base image (not limiting, an example of a first image set) composed of images including a rabbit character and a Pomeranian (not limiting, an example of a first element), and the image set is composed of images based on the first element. This allows the first user to select an image from an image set that is configured based on the first element, thereby improving convenience for the first user.

[0428] Furthermore, by using the first image set as an image set for additional learning for the image generation model unit 113 (not limiting, but an example of an image generation device), and having the image generation device learn the first image set based on an instance prompt or class prompt (not limiting, but an example of fourth information) associated with the first image set, the server 10 can generate an image based on the first element using a prompt based on the fourth information.

[0429] Also, when the first element satisfies the set condition, the server 10 performs control not to learn the first image set. This makes it possible to prevent the image generating device from learning the first element included in the first image set according to the set condition. As a result, it is possible to prevent the image generating device from generating an image including the first element that satisfies the set condition.

[0430] In addition, such a configuration can prevent images that include elements that are subject to copyright infringement or that are contrary to public order and morals from being generated in the image set. Also, it can prevent the first image set from being improperly learned due to the first image set having an inappropriate number of images or the first image set including multiple elements.

[0431] In addition, in this embodiment, the server 10 shows an example of a configuration in which the server 10 receives a first image set from a first terminal. This allows the server 10 to generate an image set based on the first image set that is configured with images including a first element desired by the first user. Therefore, the first user can express and include the desired first element by an image. Then, the first user can select an image from the image set that includes the first element.

[0432] Also, when the first element satisfies the set condition, the server 10 controls a notification to the first terminal. This allows the first user of the first terminal that transmitted the first image set to be notified that the set condition is satisfied when the first element satisfies the set condition.

[0433] Furthermore, the server 10 can generate an image set using the second image set by prompting the first user to transmit a second image set that is different from the first image set.

[0434] In addition, the server 10 can notify the first user that the first image set contains images that are subject to copyright infringement or that contain elements that are contrary to public order and morals, and that additional learning of the first image set may not be performed properly.

[0435] <Fourth modified example (1)> In the above embodiment, as an example, one instance prompt related to the image generation prompt is selected by the user of the terminal 20A, but this is not limited to this. When multiple objects are additionally learned together in one additional learning model, as an example, images corresponding to multiple instance prompts may be selectable in the instance prompt selection area ISR1, and a stamp image based on the multiple instance prompts may be generated.

[0436] This allows the stamp image to include multiple elements selected by the user, enabling users to send expressive stamps to their chat partners that fit the flow of the chat room.

[0437] <Fourth Modification (2)> In the above embodiment, as an example, the instance prompt or class prompt related to the image generation prompt is selected by the user of the terminal 20A, but this is not limited to this. For example, in the stamp image generation prompt generation process, the control unit 11 of the server 10 may randomly select an instance prompt that is available to the user of the terminal 20A from among the instance prompts stored in the stamp image additional learning model management data 155, and generate a prompt for image generation.

[0438] In this case, for example, when the stamp image generation button SCB1 on the left side of Figure 4-2 is tapped by the user, the instance prompt selection area ISR1 may not be displayed, and a stamp image including multiple objects may be displayed in the generated stamp list information display area SSR2. This allows the user to easily select a desired stamp from a wide variety of stamp images.

[0439] In addition, when multiple objects are additionally learned together in one additional learning model, for example, the control unit 11 of the server 10 may randomly select multiple instance prompts and generate a prompt for image generation. Furthermore, an instance prompt based on a base image registered by a business operator and an instance prompt based on a base image registered by a user of the terminal 20 may be used to generate a prompt for generating an image. This allows for the creation of stamp images with even greater variety.

[0440] <Fourth modified example (3)> In the above embodiment, the prompt for generating an image is generated based on the text message (character string) input in the text message input area MIR1, but the present invention is not limited to this. For example, the prompt for generating an image may be generated based on past content information transmitted and received in the chat room during the conversation.

[0441] 4-5 and 4-6 are diagrams showing examples of screens displayed on the display unit 24 of the terminal 20A in this modified example. The left side of FIG. 4-5 shows an example of a chat room screen between user AA and user BB, who is a friend of user AA. In this screen, the chat room content display area TR1 displays, for example, text message CT1, text message CT2, as well as a text message CT3 sent by user AA regarding confirming the location of the campsite, and a text message CT4 sent by user BB regarding confirming whether or not to bring their pet Pomeranian. Moreover, the text message input area MIR1 is selected by the user, and the character string "Of course!" is input and displayed.

[0442] For example, when the stamp image generation button SCB3 is tapped by the user, the display changes to the chat room screen shown in the center of FIG. 4-5. On this screen, for example, when the user taps on the image of a Pomeranian showing the instance prompt "sks" in the instance prompt selection area ISR2, for example, a stamp image generation process is executed based on the text messages CT1 to CT4 sent and received in this chat room, the character string entered in the text message input area MIR1, and the instance prompt selected in the instance prompt selection area ISR2. Then, for example, the display changes from the chat room screen on the right side of Figure 4-5 to the chat room screen on the left side of Figure 4-6.

[0443] On this screen, the generated stamp list information display area SSR3 is configured to display a predetermined number of stamps (for example, "5") including stamp images generated by an additive learning model using the instance prompt "sks" based on the chat content in this chat room and the character string "Of course!".

[0444] For example, when the user taps the rightmost stamp containing an image of a Pomeranian walking through a campsite in the generated stamp list information display area SSR3, the display changes to the chat room screen shown on the right side of Figure 4-6. On this screen, the stamp content SCT3 selected in the generated stamp list information display area SSR3 is transmitted to and displayed in the talk room content display area TR1.

[0445] In the process of this modified example, for example, in the stamp image generation prompt generation process, the following template may be used as a prompt template for creating a prompt for image generation. " <Rules> Create a prompt that instructs the image generation model to generate an illustration. Please create a list of English keywords for the prompt from the text you will enter. Use only adjectives, verbs, and nouns in your keyword list. The first keyword is "*Instance Prompt* *Class Prompt*". <Output> *Instance prompt* *Class prompt*, keyword, keyword,(loop) <Input> *Dialogue 1* *Dialogue 2* ... *Dialogue N* *Stamp text* "

[0446] For example, from *Dialogue 1* to *Dialogue N*, content information (which may also be called text content or character strings) transmitted and received within a talk room may be replaced in chronological order.

[0447] Here, the N pieces of content information transmitted and received within the talk room to be applied to the prompt template for creating a prompt for image generation may be selected as follows, for example. Content information sent or received within a specified period (for example, within 5 minutes) before the stamp text information is sent. A certain number of sent and received content information (e.g., '4 messages') before the stamp text information is sent. Content information selected by the user of the terminal 20A. This content information can be said to be the context of the chat in the talk room.

[0448] In this example, the stamp text may be left blank (for example, no input is made in the text message input area MIR1). In this case, the stamp image may be generated based on the content information sent and received in the chat room.

[0449] This configuration makes it possible to generate stamp images that correspond to the content information sent and received in a chat room. This allows users to send stamps that correspond to the content of the conversation, further enhancing communication.

[0450] This modified example shows an example of a configuration in which the server 10 generates an image set based on the context of the chat in the chat room. This allows each image included in the image set to be generated as an image that reflects the context of the chat room, and the first user can select an image that matches the chat content with the second user from the image set. Then, the second user can check the third information based on the image that matches the chat content.

[0451] <Fourth Variation (4)> In the above embodiment, the instance prompt selection area ISR1 can select, for example, an instance prompt of an additional learning model created by an operator of a messaging service or an additional learning model created by a user of the terminal 20 that sends the stamp, but is not limited to this. For example, a user who sends a stamp can select an instance prompt in the instance prompt selection area ISR1, etc., by purchasing a usage right for an additional learning model created by a general user or an official user, and a stamp image can be generated based on the additional learning model for which the usage right has been purchased.

[0452] In the process of this modification, for example, the control unit 11 of the server 10 executes a payment process for purchasing a stamp when it receives stamp purchase request information from the terminal 20. Then, the control unit 11 of the server 10 may store the other model information of the stamp image additional learning model management data 155 in association with the application ID of the user who can use the additional learning model (instance prompt). Then, when receiving stamp text information from the terminal 20A, the control unit 11 of the server 10 may refer to other model information in the stamp image additional learning model management data 155 and search for an additional learning model (instance prompt) in which the application ID of the terminal 20A is registered. Then, the control unit 11 of the server 10 may transmit stamp instance selection request information to the terminal 20A to select the searched instance prompt.

[0453] The purchase of the usage right may include, for example, the following concepts: Purchase of permanent usage rights -Purchase of usage rights with a set number of uses -Purchase of usage rights with a fixed usage period

[0454] <Other> In the above embodiment, the elements included in the image (image elements) may include, for example, the image style, style, etc. in addition to the above-mentioned objects. Furthermore, the image based on the elements may include an image of the style (an image reflecting the style), an image of the style (an image reflecting the style), etc. In this case, too, an image set (second image set) based on elements such as style and style can be generated by the image generating device using a method similar to that of the above embodiment.

[0455] Also, as an example, (1) 1st element: object, 2nd element: style (2) First element: object, second element: style (3) Other combinations For example, the information processing device may acquire a first image set including a plurality of elements (a first element, a second element, etc.), and then have an image generation device learn the first image set based on information associated with the first image set, so that a second image set is generated by the image generation device as an image set composed of images based on the above-mentioned plurality of elements.

[0456] At least a part of the processing that is to be performed by the server 10 in the above embodiment and the like may be performed by the terminal 20. Conversely, at least a part of the processing that is to be performed by the terminal 20 in the above embodiment may be performed by the server 10.

[0457] The operator of the server 10 may also be an image generation service provider in cooperation with a messaging service provider. In this case, the processing described in the above embodiments may be realized by one server, or the processing described in the above embodiments may be shared and realized by a server system consisting of multiple servers.

[0458] For example, a system configured with one or more servers may be defined as a server system, and the server of the present invention may be considered as a server system. Also, for example, the server system may be configured to include an information processing device, an image generating device, and a language generating device.

[0459] In addition, in the above embodiments, a server for distributing various applications (a server from which the terminal 20 downloads applications) may be configured as a server different from a server for providing the corresponding service (application). In other words, a server for distributing applications and a server for performing application management processing, etc., described in the above embodiments, etc., may be configured as physically separated servers, or may be configured as a single server.

[0460] Furthermore, the application is not limited to various application programs, but may include, for example, a program for providing a function of another service as one function of a base application (for example, a program for providing a function of an advertisement delivery service as one function of a messaging application, or vice versa), a program for updating the base application, etc. Also, data used in the application program (which may include data for updating the application, etc.) may be included.

[0461] In the above embodiment, the present invention is realized by a client-server system, but the present invention is not limited to this. As described above, the present invention may be realized by a system such as a distributed system that provides the functions of a server or a server system to the terminal 20. For example, the processes described in the flowcharts of the above embodiment as being performed by the server or the server system may be performed by the terminal.

[0462] As described above, the contents described in the above embodiments, the modifications, other embodiments, and the like can be applied in combination with each other. [Explanation of symbols]

[0463] 1. Communication Systems 10 Server 20 Terminals 30 Network< / x> < / x> < / x>

Claims

1. A program executed by a server that executes a process related to a chat including a first user of a first terminal and a second user of a second terminal, receiving, from the first terminal, text information based on an input to the first terminal in which a chat room including the first user and the second user is displayed, by a communication unit of the server; generating an image set including one or more images based on the text information; transmitting first information regarding the image set to the first terminal by the communication unit; receiving, from the first terminal by the communication unit, second information regarding one or more images selected from the image set based on an input to the first terminal, in which images included in the image set are displayed in the chat room based on the first information; The server transmits third information based on the second information to the second terminal via the communication unit.

2. The program according to claim 1, the image set is generated by an image generation device that generates images based on a prompt; The prompt is based on the text information.

3. The program according to claim 2, The prompts are generated by a language processor.

4. The program according to claim 1, Obtaining a first image set, the first image set being comprised of images including a first element, is performed by the server; The image set is generated based on the first image set as an image set composed of images based on the first element.

5. The program according to claim 4, the image set is generated by an image generation device that generates images based on a prompt; the first image set is an image set for additional learning of the image generating device, Training the image generation device to the first set of images based on fourth information associated with the first set of images is performed by the server.

6. The program according to claim 5, When the first element satisfies a set condition, the server performs control to prevent the first image set from being learned.

7. The program according to claim 6, The set conditions include at least one of the following: the first element is an element that is subject to copyright infringement, the first element is an element that is contrary to public order and morals, the first element includes multiple elements, and the number of images constituting the first image set is less than or equal to a set number.

8. The program according to claim 4, Receiving the first image set from the first terminal by the communication unit is performed by the server.

9. The program according to claim 8, When the first element satisfies a set condition, the server controls a notification to the first terminal.

10. The program according to claim 9, the notification is a notification that prompts transmission of a second image set different from the first image set, receiving the second image set from the first terminal by the communication unit, the second image set being executed by the server; The set of images is generated based on the second set of images.

11. The program according to claim 9, The set conditions include at least one of the following: the first element is an element that is subject to copyright infringement, the first element is an element that is contrary to public order and morals, the first element includes multiple elements, and the number of images constituting the first image set is less than or equal to a set number.

12. The program according to claim 1, Generating the set of images based on a chat context of the chat room is performed by the server.

13. 1. An information processing method for a server that executes processing related to a chat including a first user of a first terminal and a second user of a second terminal, comprising: receiving, from the first terminal, text information based on an input to the first terminal in which a chat room including the first user and the second user is displayed, by a communication unit of the server; generating an image set including one or more images based on the text information; transmitting first information regarding the image set to the first terminal by the communication unit; receiving, from the first terminal by the communication unit, second information regarding one or more images selected from the image set based on an input to the first terminal, in which images included in the image set are displayed in the chat room based on the first information; and transmitting third information based on the second information to the second terminal by the communication unit.

14. A server that executes a process related to a chat including a first user of a first terminal and a second user of a second terminal, A communication unit and a control unit are provided, The communication unit receives text information from the first terminal based on an input to the first terminal on which a chat room including the first user and the second user is displayed, The control unit generates an image set including one or more images based on the text information; The communication unit is transmitting first information about the set of images to the first terminal; receiving second information from the first terminal regarding one or more images selected from the image set based on an input to the first terminal, the images included in the image set being displayed in the chat room based on the first information; Third information based on the second information is transmitted to the second terminal.

Images