Information processing device, control method, and server device

Abstract

To provide an information processing system, a control method, and a server device which improve a user interface that decides information to be provided to another user without making a user perform a complicated operation.SOLUTION: One or a plurality of computer processors included in an information processing system executes the steps of: transmitting information about display of a moving image including a first object indicating a first user to a terminal device of a user different from the first user; setting a user group including the first user and one or a plurality of second users; transmitting information about display of a relevant image selected by the first user from the group including the plurality of relevant images about the first object to the terminal device of the second user included in the user group; and including a new relevant image including the first object indicating the first user and the second object indicating at least one second user of the one or plurality of second users, in the group according to an instruction of the first user.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The embodiments disclosed in this specification relate to an information processing system, a control method, and a server device. [Background technology]

[0002] Conventionally, instant messengers have been known that realize a function of transmitting input information, such as text information or image information, input by a user operating a terminal device to a terminal device of another user. In recent instant messengers, image information using a character object, such as an avatar, corresponding to a user can be transmitted as input information from a user's terminal device to a terminal device of another user.

[0003] For example, Patent Document 1 describes an information processing system in which an avatar stamp is used as an example of image information that can be transmitted from a user's terminal device to another user's terminal device. In the information processing system described in Patent Document 1, a user can use image information (such as a self-made avatar stamp) corresponding to an object such as an avatar that the user has created and registered. In this way, in the information processing system described in Patent Document 1, a user can appeal to other users about an object corresponding to the user by communicating with the other users using an image corresponding to an object that represents the user. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-91363 A Summary of the Invention [Problem to be solved by the invention]

[0005] In the information processing system described in Patent Document 1, in order to enrich the candidates of information corresponding to the user, the user himself needs to perform a cumbersome operation of generating information, which makes it difficult to encourage the user to use the information processing system. Also, in the information processing system described in Patent Document 1, when there are many candidates of information corresponding to the user, the operation of selecting information to be provided to other users becomes cumbersome for the user, which makes it difficult to encourage the user to use the information processing system.

[0006] The information processing system, control method, and server device disclosed in this specification make it possible to improve the user interface that determines the information to be provided to other users, without requiring the user to perform complicated operations. [Means for solving the problem]

[0007] The disclosed information processing system is an information processing system having one or more computer processors, in which the one or more computer processors perform the following: transmitting information regarding the display of a video including a first object representing a first user to a terminal device of a user other than the first user; setting a user group including the first user and one or more second users; and transmitting information regarding the display of a related image selected by the first user from a group including a plurality of related images related to the first object to a terminal device of a second user included in the user group; and the one or more computer processors perform the following in response to an instruction from the first user: including in the group a new related image including a first object representing the first user and a second object representing at least one second user of the one or more second users.

[0008] In addition, in the disclosed information processing system, in setting user groups, it is preferable that one or more computer processors set multiple user groups, and in transmitting information regarding the display of related images, the one or more computer processors transmit related images including at least one second user only to the user group including the at least one second user.

[0009] In the disclosed information processing system, it is preferable that the new related image includes a second object indicating one or more second users selected by the first user.

[0010] In addition, in the disclosed information processing system, it is preferable that one or more computer processors select one or more second users, and that the new related image includes a second object representing the selected one or more second users.

[0011] In addition, in the disclosed information processing system, it is preferable that in selecting one or more second users, one or more computer processors select one or more second users who have sent information most recently from the information sent by each second user in a chat between users included in a user group, in order of the closest sending time to the current time.

[0012] In addition, in the disclosed information processing system, it is preferable that in selecting one or more second users, one or more computer processors select one or more second users who have a chat screen between users included in the user group displayed on a terminal device at a time close to the current time.

[0013] In addition, in the disclosed information processing system, it is preferable that in selecting one or more second users, one or more computer processors select one or more second users who have a predetermined relationship with the first user.

[0014] In the disclosed information processing system, it is preferable that the second user having a predetermined relationship with the first user is a user who has had one-to-one information exchange with the first user.

[0015] In addition, in the disclosed information processing system, it is preferable that a second user having a predetermined relationship with the first user is a user of a terminal device that receives information regarding the display of a video including a first object representing the first user.

[0016] Furthermore, in the disclosed information processing system, it is preferable that the second user having a predetermined relationship with the first user is a user who is followed by the first user, or a user who has a mutual follow relationship with the first user.

[0017] In addition, in the disclosed information processing system, it is preferable that a second user having a predetermined relationship with a first user is a user who participated in the same event as the first user or a user who participated in the same game together with the first user.

[0018] In addition, in the disclosed information processing system, it is preferable that one or more computer processors select a predetermined number of second users when the number of second users participating in information exchange between users included in a user group reaches a predetermined number, and generate a new related image including a second object representing the selected predetermined number of second users.

[0019] In addition, in the disclosed information processing system, it is preferable that the new related image includes a third object related to the design of the character-related image that is different from the first object and the second object, and includes the first object and the second object in a pose corresponding to the third object.

[0020] The disclosed control method is a control method for an information processing system having one or more computer processors, comprising: the one or more computer processors transmitting information regarding the display of a video including a first object representing a first user to a terminal device of a user other than the first user; setting a user group including the first user and one or more second users; and transmitting information regarding the display of a related image selected by the first user from a group including a plurality of related images relating to the first object to a terminal device of a second user included in the user group; and the one or more computer processors including, in response to an instruction from the first user, a new related image including a first object representing the first user and a second object representing at least one second user of the one or more second users into the group.

[0021] The disclosed server device is a server device that communicates with a first terminal device of a first user and a second terminal device of one or more second users, and has a processing unit that receives information regarding the display of a video including a first object representing the first user transmitted from the first terminal device, transmits the information regarding the display of the video received from the first terminal device to a terminal device of a user different from the first user, sets a user group including the first user and one or more second users, receives information regarding the display of a related image selected by the first user from a group including a plurality of related images related to the first object transmitted from the first terminal device, and transmits information regarding the display of the related image to the second terminal device, and in response to an instruction from the first user, a new related image including a first object representing the first user and a second object representing at least one second user of the one or more second users is included in the group. Effect of the Invention

[0022] The disclosed information processing system, control method, and server device make it possible to improve the user interface that determines the information to be provided to other users, without requiring the user to perform complicated operations. [Brief description of the drawings]

[0023] [Figure 1] FIG. 1 is a schematic diagram illustrating an example of an overview of an information processing system. [Diagram 2] FIG. 1 is a schematic diagram illustrating an example of an overview of an information processing system. [Diagram 3] FIG. 1 is a diagram illustrating an example of a schematic configuration of an information processing system. [Figure 4] FIG. 2 is a diagram illustrating an example of a schematic configuration of a terminal device. [Diagram 5] 3A to 3C are diagrams illustrating examples of data structures of various tables. [Figure 6] FIG. 2 illustrates an example of a schematic configuration of a server device. [Figure 7] 5A to 5C are diagrams illustrating examples of various screens displayed on the terminal device. [Figure 8] 5A to 5C are diagrams illustrating examples of various screens displayed on the terminal device. [Figure 9] 5A to 5C are diagrams illustrating examples of various screens displayed on the terminal device. [Figure 10] 5A to 5C are diagrams illustrating examples of various screens displayed on the terminal device. [Figure 11] 5A to 5C are diagrams illustrating examples of various screens displayed on the terminal device. [Figure 12] 5A to 5C are diagrams illustrating examples of various screens displayed on the terminal device. [Figure 13] FIG. 13 is a diagram illustrating an example of a screen displayed on a terminal device. [Figure 14] 5A to 5C are diagrams illustrating examples of various screens displayed on the terminal device. [Figure 15] FIG. 13 is a diagram illustrating an example of a screen displayed on a terminal device. [Figure 16] FIG. 11 is a diagram illustrating an example of an operation sequence of the information processing system. [Figure 17] FIG. 11 is a diagram illustrating an example of an operation flow of a candidate information display process. [Figure 18] FIG. 11 is a diagram illustrating an example of an operational flow of an additional generation process. [Figure 19] 5A to 5C are diagrams illustrating examples of various screens displayed on the terminal device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0024] Various embodiments of the present invention will be described below with reference to the drawings. However, it should be noted that the technical scope of the present invention is not limited to these embodiments, but extends to the inventions described in the claims and their equivalents.

[0025] (Outline of Information Processing System) 1 and 2 are schematic diagrams for explaining an example of an overview of an information processing system. The information processing system includes a terminal device operated by each of a plurality of users and a server device. For example, the terminal device is an information processing device such as a multi-function mobile phone (a so-called "smartphone") owned by the user. For example, the server device is a computer for providing a video distribution service and an information exchange service to the terminal device via a communication network. The server device may provide services (such as online games) other than the video distribution service and the information exchange service to the terminal device.

[0026] The video distribution service is a service in which an image showing a user operating a specific terminal device (such as a distribution image including a character object showing the user, as described below) is distributed from the specific terminal device to one or more other users' terminal devices. The information exchange service is a service in which various information such as text information and image information is transmitted from one terminal device to another terminal device between the terminal devices of multiple users running instant messengers or the like. The information exchange service may be a service realized by a so-called chat function.

[0027] A character object representing each user may be used in conjunction with at least two or more services among the video distribution service, the information exchange service, and other services. The use of a character object in conjunction with the other services may mean, for example, that a character object whose appearance has been changed in the video distribution service is also displayed with the changed appearance in the information exchange service. The use of a character object in conjunction with the other services may mean that a character object whose use is permitted in the information exchange service can also be used in the video distribution service. The use of a character object in conjunction with the other services may mean that a character object whose use is no longer permitted in the information exchange service can also not be used in the video distribution service.

[0028] The terminal device stores a control program such as an application program, and loads the control program into memory in response to a start operation from a user. The terminal device starts at least one of a video distribution service and an information exchange service by inputting an instruction from the user and executing an instruction included in the loaded control program. When the video distribution service is started, the terminal device executes an instruction included in the control program to realize multiple functions related to the video distribution service. When the information exchange service is started, the terminal device executes an instruction included in the control program to realize multiple functions related to the information exchange service. When a user wants to use various services provided by the server device, the user uses a terminal device capable of communicating with the server device to perform a known login operation related to the various services, thereby starting a session between the server device and the terminal device related to the various services. This allows the user to participate in and use various services.

[0029] When a video distribution service is started, for example, the user's terminal device realizes a generation function that generates character video data including motion data based on various input data input by the user. For example, the input data is a plurality of imaging data acquired at predetermined sampling time intervals by an imaging device provided in the user's terminal device, and in this case, the user is depicted in each imaging data. The character video data is an example of information related to the display of a distribution image including a character object representing the user. The character video data is generated at predetermined time intervals during the video distribution service.

[0030] Also, for example, the user's terminal device displays a delivery image based on the character video data. In addition to displaying the delivery image, the user's terminal device transmits the generated character video data to the other user's terminal device via the server device, thereby realizing an output function of transmitting the generated character video data to the server device at predetermined time intervals. Also, for example, the user's terminal device realizes a function of displaying a delivery image including a specific object based on information indicating a predetermined instruction of another user transmitted from the server device at predetermined time intervals.

[0031] Hereinafter, a user who uses at least the function of a terminal device to transmit character video data to a server device may be referred to as a broadcasting user. A terminal device operated by a broadcasting user may be referred to as a broadcasting terminal device. Also, a user who uses at least the function of a terminal device to receive character video data from a broadcasting terminal device and display a broadcast image of the broadcasting user may be referred to as a viewing user. A terminal device operated by a viewing user may be referred to as a viewing terminal device. Note that one terminal device may have the functions of both a broadcasting terminal device and a viewing terminal device.

[0032] When the video distribution service is started, as shown in Fig. 1(a), the distribution terminal device displays a distribution screen including a distribution image related to the distribution user's distribution event. One or more viewing terminal devices receive character video data from the distribution terminal device via the server device, and display a distribution screen including a distribution image related to the distribution user's distribution event based on the character video data. The distribution image of the distribution user includes a character object representing the distribution user. The character object is displayed to move according to the motion data of the distribution user included in the character video data.

[0033] When the broadcasting terminal device is equipped with a microphone, the broadcasting terminal device can acquire voice uttered by the broadcasting user and transmit the acquired voice data of the broadcasting user to one or more viewing terminal devices. In this case, the broadcasting terminal device transmits the character video data and voice data of the broadcasting user to the server device in order to transmit the voice data of the broadcasting user together with the character video data to one or more viewing terminal devices via the server device. In addition, the one or more viewing terminal devices receive from the server device the character video data and voice data of the broadcasting user transmitted to the server device by the broadcasting terminal device.

[0034] When the voice generated by the broadcast user is acquired by the broadcast terminal device, the broadcast user's motion data included in the character video data includes face motion data indicating the movement of the broadcast user's lips. In this case, the broadcast terminal device displays a broadcast image including a character object representing the broadcast user whose lips move in approximate synchronization with the voice generated by the broadcast user. When one or more viewing terminal devices receive the broadcast user's voice data together with the broadcast user's character video data, they output the broadcast user's voice and display a broadcast image including a character object representing the broadcast user whose lips move in approximate synchronization with the output voice.

[0035] The broadcasting terminal device generates a character-related image including a character object representing the broadcasting user in response to an instruction from the broadcasting user. For example, the character-related image is a so-called "stamp image" that can be transmitted by the broadcasting terminal device to one or more other users' terminal devices in an information exchange service. The character-related image is not limited to a stamp image, and may be any image that can be used in the service provided by the server device. The other users may be limited to viewing users, or may be users other than the broadcasting user including the viewing user. The other users may be users other than the broadcasting user not including the viewing user.

[0036] The character-related image may be used in a service other than the information exchange service provided by the server device. For example, in a video distribution service, a distribution terminal device that generates a character-related image may be configured to transmit the character-related image to one or more viewing terminal devices. Also, in an online game, a distribution terminal device that generates a character-related image may be configured to transmit the character-related image to one or more terminal devices of other users participating in the online game.

[0037] In the example shown in Fig. 1(a), the broadcasting terminal device generates three types of character-related images. The character-related images generated by the broadcasting terminal device are not limited to the three types shown in Fig. 1(a), and two types of character-related images or four or more types of character-related images may be generated.

[0038] Next, the broadcasting terminal device sets an available group including character-related images available in at least one service provided by the server device. The character-related images belonging to (included in) the available group set by the broadcasting terminal device are not limited to the two types shown in Fig. 1(a). The available group may be set so that one type of character-related image belongs to the available group, or the available group may be set so that three or more types of character-related images belong to the available group.

[0039] Character-related images that do not belong to (are not included in) the available group set by the distribution terminal device are not limited to one type shown in Fig. 1(a). There may be two or more types of character-related images that do not belong to the available group. There may be no character-related images that do not belong to the available group. In this case, each time a character-related image is generated, the available group may be set so that the generated character-related image belongs to the available group.

[0040] Hereinafter, with reference to FIG. 1(b), an example of the use of character-related images in the information exchange service will be described. FIG. 1(b) is a diagram showing an example of an information exchange screen displayed by a broadcasting terminal device using the information exchange service. In the example shown in FIG. 1(b), input information input by a broadcasting user is displayed in chronological order from the top to the bottom of the screen in the user display area of ​​the information exchange screen, starting from the oldest input information. For example, the user display area is an area near the right side of the information exchange screen. The input information is text information, still image information, moving image information, stamp images, and the like. In addition, input information of other users transmitted from one or more terminal devices of other users is displayed in chronological order from the top to the bottom of the screen in the other user display area of ​​the information exchange screen, starting from the oldest input information. For example, the other user display area is an area near the left side of the information exchange screen.

[0041] As shown in Fig. 1(b), the broadcasting terminal device, in response to an instruction from the broadcasting user, selectably displays candidate information indicating all or some of the character-related images belonging to the broadcasting user's available group. The candidate information is an image identical to the character-related image, a thumbnail image that is a reduced version of the character-related image, a simplified image that is a deformed version of the character-related image, etc. The candidate information may include text information, a mark image, etc. indicating the character-related image.

[0042] The broadcasting terminal device displays a character-related image corresponding to candidate information selected by the broadcasting user from the displayed candidate information, below the most recently input information in the user display area of ​​the information exchange screen (Figure 1 (b)). Along with displaying the character-related image, the broadcasting terminal device transmits information relating to the display of the character-related image to one or more terminal devices of other users via the server device. In this case, the terminal device of one or more other users that receives the information relating to the display of the character-related image displays the character-related image below the most recently input information in the other user display area of ​​the information exchange screen. The information relating to the display of the character-related image is identification information for identifying the character-related image. The information relating to the display of the character-related image may be the character-related image itself.

[0043] The broadcasting terminal device executes an additional generation process to generate a new related image including a character object representing the broadcasting user and a character object representing at least one of the one or more other users. The one or more other users are users who are participating in the started information exchange service and are able to exchange information with the broadcasting user.

[0044] An example of the additional generation process will be outlined below with reference to FIG.

[0045] First, the distribution terminal device selects one or more users from a selection target group consisting of one or more other users, either in response to an instruction from the distribution user or automatically. In the example shown in (1) of FIG. 2, other user X is selected from a selection target group to which two other users (other user X and other user Y) belong.

[0046] Users belonging to the selection target group are not limited to other users who participate in the information exchange service and can exchange information with the broadcast user. The selection target group may include one or more users who have a predetermined relationship with the broadcast user, or the selection target group may only include one or more users who have a predetermined relationship with the broadcast user. In addition, the selection target group may include users who participate in services other than information exchange, or the selection target group may only include users who participate in services other than information exchange.

[0047] When another user X is selected, the broadcasting terminal device generates a new character-related image including both a character object representing the broadcasting user and a character object representing the selected other user X ((2) in FIG. 2). The new character-related image may include various objects other than the character object (objects representing text, background objects, etc.). The character objects included in the new character-related image may be controlled to automatically perform a predetermined action.

[0048] Next, the broadcasting terminal device causes the new character-related image to belong (be included) in the available group, and selectably displays candidate information indicating the new character-related image, as shown in (3) of Fig. 2. Then, when a piece of candidate information is selected by the broadcasting user from the displayed candidate information, the broadcasting terminal device displays the character-related image corresponding to the piece of candidate information below the most recently input information in the user display area of ​​the information exchange screen.

[0049] The distribution terminal device may include the new character-related image in the available group, and may delete other character-related images included in the available group. The deleted other character-related images may be all character-related images included in the available group other than the new character-related image. As a result, the character-related images already included in the available group are changed to the new character-related images.

[0050] The terminal device that generates the character-related image is not limited to the distribution terminal device, and any terminal device that can use a character object representing a user may generate a character-related image including the character object.

[0051] As described above with reference to Figures 1 and 2, the information processing system, the terminal device, and the server device increase the types of character-related images that can be used based on information about character objects representing users other than the user who instructed the generation or increase of the character-related images. In this way, the information processing system, the terminal device, and the server device make it possible to improve the user interface that determines the information to be provided to other users without requiring the user to perform complicated operations.

[0052] Furthermore, the information processing system, the terminal device, and the server device change the character-related image based on the information on the character object representing the other user other than the user who instructed the generation or increase of the character-related image to at least one of the available character-related images. In this way, the information processing system, the terminal device, and the server device make it possible to change the information provided to other users, without requiring the user to perform complicated operations, thereby preventing the user from getting bored with the selection.

[0053] Furthermore, in the conventional information processing system, in order for a user to perform an operation or give an instruction to increase, decrease, or change the candidates for character-related images, the user needs to interrupt or end the service that uses the character-related image. Thus, in the conventional information processing system, when a user performs an operation or gives an instruction to increase, decrease, or change the candidates for character-related images, multiple communications are generated regarding the interruption or end of the service and the resumption of the interrupted service, which may cause a problem of an increase in the processing load of the terminal device and / or the communication load between the terminal device and the server device.

[0054] In contrast, in the information processing system, terminal device, and server device disclosed in this embodiment, by performing an additional generation process of generating a new character-related image based on information about a character object indicating a user other than the user who instructed the generation of the character-related image, it is possible to diversify the information provided to the other users and / or change the information provided to the other users to prevent the user from becoming bored with the selection, thereby reducing the processing load of the terminal device and / or the communication load between the terminal device and the server device. In addition, the information processing system, terminal device, and server device disclosed in this embodiment can diversify the character-related images provided to the other users, thereby improving the user's willingness to use various services in which the character-related images can be used. In addition, the information processing system, terminal device, and server device disclosed in this embodiment can realize automatic change of the character-related image with a simple interface while continuing the execution of the service.

[0055] In the example shown in FIG. 1 and FIG. 2, the user and other users who use the information exchange service may be users who have a predetermined relationship. For example, other users who have a predetermined relationship with the user are mutual followers of the user. In addition, when the user follows other users and the other users also follow the user (the user and the other users follow each other), the user and the other users become mutual followers. In addition, other users who have a predetermined relationship with the user may be other users who have a mutual follow relationship with a predetermined user who is a mutual follower of the user. In addition, other users who have a predetermined relationship with the user may be other users who have a friend relationship with the user, or other users whose specific information (such as a phone number, an email address, or a predetermined ID (identification)) is stored in the user's terminal device. In addition, when the user is a distribution user, other users who have a predetermined relationship with the user may be viewing users who have viewed a distribution image distributed by the user. In this case, when the user and other users who have a predetermined relationship perform an information exchange service as shown in FIG. 1 and FIG. 2, a predetermined information exchange group composed of the user and other users may be temporarily created by the terminal device of the user or the other users or the server device.

[0056] 1 and 2 are merely for the purpose of deepening understanding of the contents of the present invention. The present invention is specifically embodied in the following embodiments, and may be embodied in various modified forms without substantially departing from the principles of the present invention. All such modified forms are included in the scope of the present invention and the disclosure of this specification.

[0057] (Information Processing System 1) FIG. 3 is a diagram showing an example of a schematic configuration of an information processing system 1. The information processing system 1 includes a terminal device 2 operated by each of a plurality of users, and a server device 3. The terminal device 2 and the server device 3 are connected to each other via a communication network such as a base station 4, a mobile communication network 5, a gateway 6, and the Internet 7. Communication between the terminal device 2 and the server device 3 is performed based on a communication protocol such as Hypertext Transfer Protocol (HTTP). In addition, a connection may be first established between the terminal device 2 and the server device 3 by HTTP communication, and thereafter, communication may be performed based on WebSocket, which is a two-way communication with a lower cost (lower communication load and processing load) than HTTP communication. Note that the communication method between the terminal device 2 and the server device 3 is not limited to the above-mentioned method, and any communication method technology may be used between the terminal device 2 and the server device 3 as long as it can realize this embodiment.

[0058] The communication network may be a Local Area Network (LAN) or a Wide Area Network (WAN), or any combination of LANs and WANs. The communication network may include a Public Switched Telephone Network (PSTN), an Integrated Services Digital Network (ISDN), and / or a sub-network. The communication network may be a wired communication network, such as an Ethernet network cable or a Universal Serial Bus (USB) cable. The communication network may be a wireless communication network, such as a cellular network, including EDGE, 3G, 4G, and / or 5G wireless cellular systems. The wireless communication network may include Wi-Fi, wireless LAN, Bluetooth, or other known wireless forms of communication.

[0059] The terminal device 2 is an information processing device such as a smartphone. The terminal device 2 may be a mobile phone, a notebook PC, a tablet terminal, a tablet PC, a head mounted display (HMD), an electronic book reader, a wearable computer, or the like. The terminal device 2 may be a portable game device, a game console, or the like. The terminal device 2 may be any information processing device as long as it is capable of displaying and outputting character objects representing the user and other users, outputting the voices of other users, and processing information exchange with other users.

[0060] 3, one server device 3 is illustrated as a component of the information processing system 1, but the server device 3 may be a collection of multiple physically separate server devices 3. In this case, each of the multiple server devices 3 may have the same function, or may have the function of a single server device 3 in a distributed manner. For example, a server device 3a for realizing a video distribution service and a server device 3b for realizing an information exchange service may be components of the information processing system 1.

[0061] (Terminal device 2) FIG. 4 is a diagram showing an example of a schematic configuration of the terminal device 2. The terminal device 2 is connected to the server device 3 via a communication network such as a base station 4, a mobile communication network 5, a gateway 6, and the Internet 7, and communicates with the server device 3. The terminal device 2 generates character video data including motion data according to various data (image capture data, etc.) input by a user, and transmits the generated character video data and / or the user's voice data to the server device 3. The terminal device 2 also receives character video data and / or voice data of other users transmitted from the server device 3, and displays character objects of other users and / or outputs voices of other users based on the received character video data and / or voice data. The terminal device 2 also displays input information such as text information, still image information, video information, and stamp images input by the user, and input information input by other users transmitted from the terminal devices 2 of other users in chronological order. To realize such functions, the terminal device 2 includes a terminal communication I / F 21, a terminal storage device 22, a display device 23, an input device 24, an imaging device 25, a microphone 26, and a terminal processing device 27.

[0062] The terminal communication I / F 21 is implemented as hardware, firmware, or communication software such as a TCP / IP (Transmission Control Protocol / Internet Protocol) driver or a PPP (Point-to-Point Protocol) driver, or a combination of these. The terminal device 2 can transmit data to and receive data from other devices such as a server device 3 via the terminal communication I / F 21. The terminal communication I / F 21 may include a network controller such as an Ethernet PRO network interface card for interfacing with a communication network. The network controller may also be compliant with other direct wireless communication standards such as Bluetooth (registered trademark), Near Field Communication (NFC), or infrared.

[0063] The terminal storage device 22 is, for example, a semiconductor memory device such as a ROM (Read Only Memory) or a RAM (Random Access Memory). The terminal storage device 22 stores an operating system program, a driver program, a control program, data, and the like used for processing in the terminal processing device 27. The driver programs stored in the terminal storage device 22 are, for example, an output device driver program for controlling the display device 23, and an input device driver program for controlling the input device 24. The control programs stored in the terminal storage device 22 are, for example, application programs for realizing various functions related to various services including a video distribution service and an information exchange service. The control programs may be those transmitted from the server device 3 or other devices.

[0064] The terminal storage device 22 stores, as data, identification information (e.g., a user ID) for uniquely identifying a user who operates the terminal device 2. The terminal storage device 22 also stores, as data, background data, model data, and the like. The terminal storage device 22 also stores, as data, a user table T1, an object table T2, and a group table T3. The terminal storage device 22 may also temporarily store data related to a predetermined process.

[0065] The background data is asset data for constructing a virtual space in which the user's character object exists in the delivered image. The background data includes data for drawing the background of the virtual space, data for drawing various objects included in the delivered image, and data for drawing various other background objects displayed in the delivered image. The background data may include object position information indicating the positions of various background objects in the virtual space.

[0066] The terminal storage device 22 also stores a plurality of types of motion data for character-related images as data. The motion data for character-related images includes body motion data that is a digital representation of the position and orientation of each part (head, arms, hands (including fingers), chest, torso, legs, or other parts) for making the character object perform a predetermined action. The motion data for character-related images may also include face motion data for making the character object show a predetermined facial expression.

[0067] The motion data for a character-related image may be a set of multiple continuous motion data to express a character object moving for a predetermined time (e.g., 3 seconds). The motion data for a character-related image may be a single piece of motion data. When the motion data for a character-related image is a single piece of motion data, the character object included in the character-related image generated by the motion data for that character-related image does not move.

[0068] The terminal storage device 22 may also store object data for character-related images, such as object data for expressing various background objects, gift objects, and characters, together with motion data for character-related images. By using object data for character-related images relating to the design of such character-related images, it becomes possible to diversify the expression of the character-related images. In this case, the terminal storage device 22 may store an object ID associated with the object data for character-related images.

[0069] The display device 23 is a liquid crystal display. The display device 23 may be an organic EL (Electro-Luminescence) display or the like. The display device 23 displays on a display screen a moving image corresponding to moving image data and / or a still image corresponding to still image data supplied from the terminal processing device 27. The display device 23 does not need to be a component of the terminal device 2. In this case, the display device 23 may be a display of an HMD capable of communicating with the server device 3, a projection device for projection mapping capable of communicating with the terminal device 2 via wired or wireless communication, a retinal projection type projection device, or the like. The terminal device 2 may include a display controller such as a graphics card or a graphics adapter for interfacing with the display device 23.

[0070] The input device 24 is a pointing device such as a touch panel. When the input device 24 is a touch panel, the input device 24 can detect various touch operations such as tapping, double tapping, and dragging by the user. The touch panel may be equipped with a capacitance-type proximity sensor and configured to be capable of detecting a non-contact operation by the user. The input device 24 may be an input key or the like. The user can use the input device 24 to input characters, numbers, and symbols, or positions on the display screen of the display device 23, etc. When operated by the user, the input device 24 generates a signal corresponding to the operation. Then, the input device 24 supplies the generated signal to the terminal processing device 27 as an instruction from the user. The terminal device 2 may be equipped with an I / O interface for interfacing with the input device 24. The I / O interface may be connected to various peripheral devices other than the input device 24.

[0071] The imaging device 25 is a camera having an imaging optical system, an imaging element, an image processing unit, and the like. The imaging optical system is, for example, an optical lens, and forms an image of a light beam from a subject on the imaging surface of the imaging element. The imaging element is, for example, a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), and outputs an image of the subject image formed on the imaging surface. The image processing unit creates moving image data in a predetermined file format from images continuously generated by the imaging element at predetermined intervals, and outputs the moving image data as imaging data. Alternatively, the image processing unit creates still image data in a predetermined file format from images generated by the imaging element, and outputs the moving image data as imaging data.

[0072] The microphone 26 is a sound collecting device configured to acquire voice uttered by the user and convert it into voice data. The microphone 26 is configured to acquire voice input of the user, converts the user's voice input acquired by the microphone 26 into voice data, and outputs the voice data to the terminal processing device 27.

[0073] The terminal processing device 27 is a processor that loads the operating system program, the driver program, and the control program stored in the terminal storage device 22 into the memory and executes the instructions included in the loaded programs. The terminal processing device 27 is, for example, an electronic circuit such as a central processing unit (CPU), a micro processing unit (MPU), a digital signal processor (DSP), or a graphics processing unit (GPU), or a combination of various electronic circuits. The terminal processing device 27 may be realized by integrated circuits such as application specific integrated circuits (ASICs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), and micro controller units (MCUs). In FIG. 4, the terminal processing device 27 is illustrated as a single component, but the terminal processing device 27 may be a collection of multiple physically separate processors. For example, multiple processors that operate in parallel in a cooperative manner to execute instructions may be implemented.

[0074] It should be noted that the functions performed by the components disclosed herein (e.g., the terminal processing device 27 and / or the server processing device 33 (described below)) may be implemented in "circuitry" or "processing circuitry" including general-purpose processors, application-specific processors, integrated circuits, ASICs, CPUs, conventional circuits, and / or combinations thereof, programmed to perform the described functions. Processors include transistors and other circuits and are considered to be "circuitry" or "processing circuitry." A processor may be a "programmed processor" that executes programs stored in memory.

[0075] As used herein, a "circuitry," "unit," or "means" refers to hardware that is programmed to implement or performs the described functions, which may be any hardware disclosed herein or any hardware known to be programmed to implement or perform the described functions.

[0076] Where the hardware is a processor considered to be a type of "Circuitry", the "Circuitry", "unit", or "means" is the combination of the hardware and software used to configure the hardware and / or processor.

[0077] The "Processing Circuitry" may be used to control any computer-based and cloud-based control process. In the operational sequences, operational flows, and flowcharts described below, descriptions or blocks may be understood as indicating modules, segments, or portions of code that include one or more executable instructions for implementing specific logical functions or steps of the control process. These alternative implementations are also included within the scope of the exemplary embodiments of the present advancements. For example, specific logical functions or steps of the control process may be performed in a different order than that disclosed, e.g., substantially simultaneously, or in a reverse order to that disclosed.

[0078] "Processing Circuitry" may include a CPU that executes one or more of the control processes disclosed herein. The process data and instructions may be stored in memory. The process data and instructions may be stored on a storage medium disk, such as a Hard Disk Drive (HDD) or a portable storage medium. The process data and instructions may be stored remotely.

[0079] The claimed advancements are not limited by the form of computer-readable medium on which the instructions for the inventive process are stored. For example, the instructions may be stored on a CD, DVD, FLASH memory, RAM, ROM, PROM, EPROM, EEPROM, hard disk, or any other non-volatile computer-readable medium of an information processing device (such as another server device or computer) with which the "Processing Circuitry" communicates. The instructions may also be stored on network-based storage, cloud-based storage, or other mobile-accessible storage, provided that the instructions are executable by the "Processing Circuitry."

[0080] The claimed advancements may be provided as a utility application, a background daemon, a component of an operating system, or any combination thereof, and may execute in conjunction with a CPU and an operating system, including Microsoft Windows, UNIX, Solaris, LINUX, Apple MAC-OS, Apple iOS, and other systems known to those skilled in the art.

[0081] The functions and features disclosed herein may be performed by various distributed components of the information processing system 1. For example, one or more processors may perform these system functions, and each processor may be distributed among multiple components that can communicate within a network. The distributed components may include various human interfaces, communication devices (display monitors, smartphones, tablets, personal digital assistants (PDAs), etc.), and one or more client and server devices that share processing. Input to the information processing system 1 may be received directly by a user or remotely in real time or through a batch process. Furthermore, some of the implementations may be performed in modules or hardware that are not identical to those disclosed herein, in which case other implementations are within the scope of the claims.

[0082] The terminal processing device 27 executes various commands included in the control program to function as a generation unit 271, a transmission unit 272, a reception unit 273, a display processing unit 274, and an audio output unit 275. The functions of the generation unit 271, the transmission unit 272, the reception unit 273, the display processing unit 274, and the audio output unit 275 will be described later.

[0083] (Various tables) 5 is a diagram showing an example of the data structures of the user table T1, object table T2, and group table T3 stored in the terminal storage device 22. At least one of the user table T1, object table T2, and group table T3 may be stored in the server storage device 32 of the server device 3. In this case, the terminal device 2 may receive the latest table stored in the server storage device 32 at a predetermined timing and store it in the terminal storage device 22.

[0084] (User table T1) FIG. 5(a) is an example of a data structure of a user table T1 that manages users who use the service provided by the server device 3. The users managed in the user table T1 are broadcast users, but users other than broadcast users may also be managed. In the user table T1, for each user, a user ID, the user's name, the user's character object, the user's owned objects, and the objects used, etc. are stored in association with each other. In the user table T1, the user's owned coins (information indicating the total amount of "coins", a virtual monetary value, held by the user) and / or user points, etc. may be stored in association with the user's user ID. The user ID is an example of identification data for uniquely identifying each user. The name is an example of data indicating the name of each user.

[0085] For example, the character object is a model ID for identifying model data for generating animation of the character object. The model data is stored in the terminal storage device 22 in association with the model ID. The model data may be three-dimensional model data for generating three-dimensional animation, or may be two-dimensional model data for generating two-dimensional model data. The model data includes, for example, rig data (so-called "skeleton data") indicating the skeleton of the face and parts other than the face of the character object, and surface data indicating the shape and texture of the surface of the character object. The model data may include a plurality of model data different from each other. Each of the plurality of model data may have different rig data from each other, or may have the same rig data. Each of the plurality of model data may have different surface data from each other, or may have the same surface data. Note that a plurality of object IDs associated with each of a plurality of partial objects constituting the character object may be associated with the model ID. In this case, three-dimensional model data for generating three-dimensional animation may be stored in the terminal storage device 22 and the server storage device 32 for each object ID of the plurality of partial objects. In this case, in the display process of the character object, the object IDs of the partial objects associated with the model ID may be extracted, the three-dimensional model data (of each partial object) corresponding to each object ID may be specified, and the character object may be displayed based on the specified three-dimensional model data (of each partial object). Also, image information corresponding to each three-dimensional model data may be stored.

[0086] The owned object is an object ID indicating an object owned by each user. The owned object is an attachment object that can be associated with a specific part of a character object, a wallpaper object that is placed behind the user's character object in a delivery image, etc. Details of the attachment object and the wallpaper object will be described later. The owned object is not limited to the above example, and may be any object that can be displayed on the delivery screen. The owned object may be an object that the user has come to own depending on the result of an event in which the user has participated, or depending on the result of a selection game (e.g., a lottery game) executed by the user. The owned object may also be an object that the user has come to own by spending "coins" of virtual monetary value in a purchase function in the information processing system 1.

[0087] The used object is an object ID indicating an object used in a delivery image being displayed on the user's terminal device 2. For example, when a user's change instruction is input during a video delivery service, an attachment object held as a user's owned object is attached to the user's character object included in the delivery image being displayed. In this case, the object ID of the attachment object attached to the user's character object is stored in the user table T1 as a used object in association with the user ID. Also, for example, when a user's change instruction is input during a video delivery service, a wallpaper object held as a user's owned object is placed behind the user's character object in the delivery image being displayed. In this case, the object ID of the wallpaper object placed in the delivery image is stored in the user table T1 as a used object in association with the user ID.

[0088] (Object table T2) 5(b) is an example of the data structure of the object table T2 that manages objects selected as additional objects. The additional object is, for example, a gift object given to the distribution user by the viewing user. In the object table T2, the object ID, name, image information, rarity, placement location, etc. of each additional object are stored in association with each other.

[0089] The object ID is an example of identification data for uniquely identifying each additional object. The image information is one or more still images corresponding to each additional object. The image information may be one or more types of moving images corresponding to each additional object. The rarity is information indicating the rarity of each additional object. The placement location is information indicating the location where the image information of each additional object is displayed. In addition, when the display position and display range of a character object are determined, information indicating a relative position with respect to the character object may be stored as the placement location. Instead of the image information, model data of the additional object may be stored. The model data of the additional object includes rig data and surface data indicating a three-dimensional shape, and an image of the additional object viewed from a predetermined viewpoint is drawn based on the model data.

[0090] The additional objects are classified into a plurality of types (categories). For example, the additional objects include effect objects used as effect gifts, normal objects used as normal gifts, attachment objects used as attachment gifts, and message objects used as message gifts. The additional objects may include wallpaper objects placed behind the character objects in the delivery images. Information indicating the type (category) of the additional object may be stored in the object table T2 in association with the object ID of the additional object. Below, image information and placement locations will be described according to the type of additional object.

[0091] The effect object is an object that affects the overall impression of the delivery image. The effect object is, for example, an object that resembles confetti. When the effect object is an object that resembles confetti, an image showing a plurality of pieces of paper is stored as the image information.

[0092] Information indicating "space" as the placement location of the effect object is stored. For example, an object resembling confetti is displayed over the entire distribution image. The effect object may be displayed over the entire distribution screen including the distribution image of the broadcasting user in response to an instruction from the viewing user. The effect object may be displayed so as to overlap with a character object included in the distribution image. Unlike an attachment object described later, the effect object is displayed without being associated with a specific part of the character. In this way, the terminal device 2 of the information processing system 1 can change the overall impression of the distribution image of the broadcasting user compared to before the effect object was displayed by displaying the effect object in the distribution image of the broadcasting user in response to an instruction from the viewing user.

[0093] The normal object is, for example, an object that imitates a stuffed toy, a bouquet, an accessory, or an item suitable for a gift or present. Information indicating a "space" is stored as the placement location of the normal object. For example, information regarding a predetermined movement route is associated with the normal object, and the normal object is displayed moving along the predetermined movement route in the delivery image. The information indicating the "space" stored as the placement location of the "normal object" may be classified into multiple types. The multiple types of "space" are, for example, a "center space" (a region in the center of the image when the delivery image is divided into three parts horizontally), a "left space" (a region on the left side of the image when the delivery image is divided into three parts horizontally), and a "right space" (a region on the right side of the image when the delivery image is divided into three parts horizontally). In addition, the multiple types of "space" may be a "center space" (a region in the center of the image when the delivery image is divided into three parts vertically), an "upper space" (a region on the upper side of the image when the delivery image is divided into three parts vertically), and a "lower space" (a region on the lower side of the image when the delivery image is divided into three parts vertically). In this case, the movement route associated with the normal object is set within the "space" of the type associated with the placement location of the normal object. For example, when information indicating "right space" is stored in a normal object, information regarding a movement route that moves within the area on the right side of the image when the delivery image is divided into three parts horizontally is associated with the normal object. In addition, a normal object that moves according to a predetermined movement rule from a position set for each normal object in the delivery image or a position automatically and randomly determined may be displayed. The predetermined movement rule is, for example, a movement rule set so that the normal object moves as if it is falling freely. Note that the normal object may be displayed so as to overlap with the character object. In this case, the normal object is displayed without being associated with a specific part of the character object, unlike an attached object described later. In addition, the normal object may be displayed so as to move as if it bounces back when it comes into contact with the character object.

[0094] When a normal object is displayed overlapping with a character object included in a distributed image, for example, the normal object may be displayed so as to overlap with "parts other than the 'head', including the 'face', of the character object. In this case, the normal object is displayed so as not to overlap with the "head" of the character object. Also, the normal object may be displayed so as to overlap with "parts other than the 'upper body', including the 'face', of the character object. In this case, the normal object is displayed so as not to overlap with the "upper body" of the character object.

[0095] The wearing object is an object that is displayed in the delivered image in association with a specific part (wearing location) of the character object. For example, the specific part is the front left side, front right side, rear left side, rear right side, front center side, rear center side, left eye, right eye, left ear, right ear, and all of the hair on the head of the character object. The specific part may also be the thumb, ring finger, wrist, elbow, shoulder, upper arm, entire hand, entire arm, torso, upper body, lower body, etc.

[0096] The wearable object associated with a specific part of the character object is displayed in the delivered image so as to be in contact with the specific part of the character object. The wearable object associated with a specific part of the character object may be displayed in the delivered image so as to cover a part or the whole of the specific part of the character object. The specific part may be specified by three-dimensional position information indicating a position in a three-dimensional coordinate space, or may be associated with the position information in the three-dimensional coordinate space.

[0097] The image information of the wearing object is, for example, image information that imitates accessories (headbands, necklaces, earrings, etc.), clothing (T-shirts or dresses, etc.), costumes, and other items that can be worn by the character object.

[0098] As the placement location of the wearing object, information indicating the wearing part, which indicates which part of the character object the wearing object is associated with, is stored. For example, if the wearing object is a "headband", information indicating the "head" of the character object is stored as the placement location of the wearing object. Also, if the wearing object is a "T-shirt", information indicating the "torso" of the character object is stored as the placement location of the wearing object.

[0099] Information indicating a plurality of attachment positions in a three-dimensional coordinate space may be stored as the placement position of the attachment object. For example, if the attachment object is a "headband," information indicating two positions of the attachment object, the "left rear of the head" and the "right rear of the head" of the character object, may be stored as the placement position of the attachment object. In this way, an attachment object modeled after a "headband" is displayed as being attached to both the "left rear of the head" and the "right rear of the head" of the character object.

[0100] When multiple types of attachment objects that are attached to the same attachment location are attached to the character object, the multiple types of attachment objects are attached to the character object at different times. As a result, multiple types of attachment objects that are attached to the same attachment location are not attached to the character object at the same time. For example, if the "head" is stored as the attachment location of both an attachment object indicating a "headband" and an attachment object indicating a "hat", the attachment object indicating the "headband" and the attachment object indicating the "hat" are not displayed on the head of the character object at the same time.

[0101] In the object table T2, a display time according to the type of additional object may be stored in association with the object ID of the additional object. For example, the display time of the attached object may be stored as a time longer than the display time of the effect object and the display time of the normal object. For example, when "60 seconds" is stored as the display time of the attached object, "5 seconds" may be stored as the display time of the effect object, and "10 seconds" may be stored as the display time of the normal object.

[0102] (Group table T3) Fig. 5(c) is an example of the data structure of a group table T3 that manages information exchange groups to which a user belongs. The group table T3 is provided individually for each user, and the group table T3 shown in Fig. 5(c) is the group table T3 of the user who operates the terminal device 2 that stores the group table T3.

[0103] In the group table T3, for each information exchange group, the group ID of the information exchange group, the name of the information exchange group, the users belonging to the information exchange group, etc. are stored in association with each other. The group ID is an example of identification data for uniquely identifying each information exchange group. The name is an example of data indicating the name of each information exchange group. The users belonging to the information exchange group are the user IDs of each user belonging to the information exchange group.

[0104] When the group table T3 is stored in the server storage device 32, the user ID of the corresponding user (the user who operates the terminal device 2 storing the group table T3) is stored in association with each group table T3.

[0105] Returning to FIG. 4, the functions of the generation unit 271, the transmission unit 272, the reception unit 273, the display processing unit 274, and the audio output unit 275 will be described.

[0106] (Generation part 271) The generating unit 271 functions when the terminal device 2 is used as a broadcasting terminal device operated by a broadcasting user. The generating unit 271 acquires imaging data continuously output by the imaging device 25 from the imaging device 25. For example, the imaging device 25 of the terminal device 2 (broadcasting terminal device) held by the broadcasting user is provided on the terminal device 2 so that the face of the broadcasting user looking at the display screen of the display device 23 is the shooting direction. The imaging device 25 continuously captures the face of the broadcasting user in the shooting direction, acquires imaging data of the face of the broadcasting user, and outputs it to the generating unit 271 of the terminal processing device 27. The imaging device 25 may acquire imaging data by capturing an image of a part other than the face of the broadcasting user, for example, a head, an arm, a hand (including a finger), a chest, a torso, a leg, or a part other than these. The imaging device 25 may also be a 3D camera capable of detecting the depth of a person's face.

[0107] The generation unit 271 generates face motion data, which is a digital representation of the facial movement of the broadcast user, as time passes, based on the continuously acquired imaging data. The face motion data may be generated at predetermined sampling time intervals. In this way, the face motion data generated by the generation unit 271 can digitally represent the facial movement (changes in facial expression) of the broadcast user in chronological order.

[0108] The generation unit 271 may generate body motion data, which is a digital representation of the position and orientation of each part of the distribution user (head, arms, hands (including fingers), chest, torso, legs, or other parts), together with or separately from the facial motion data, based on continuously acquired imaging data.

[0109] The body motion data may be generated based on detection information from a known motion sensor worn by the broadcast user. In this case, the terminal communication I / F 21 of the terminal device 2 (broadcast terminal device) has a predetermined communication circuit for acquiring detection information from the motion sensor worn by the broadcast user via wireless communication. Then, the generation unit 271 generates the body motion data based on the detection information acquired by the terminal communication I / F 21. The body motion data may be generated at predetermined sampling time intervals. In this way, the body motion data generated by the generation unit 271 can digitally represent the body movements of the broadcast user in a chronological order.

[0110] The generation of body motion data based on detection information from a motion sensor worn by a broadcast user may be performed, for example, in a photography studio. In this case, the photography studio may be equipped with a base station, a tracking sensor, and a display. The base station may be, for example, a multi-axis laser emitter. The motion sensor worn by the broadcast user may be, for example, a Vive Tracker provided by HTC CORPORATION, and the base station may be, for example, a base station provided by HTC CORPORATION.

[0111] Also, the supporter computer may be installed in a separate room adjacent to the photography studio. The display of the photography studio may be configured to display information received from the supporter computer. The server device 3 may be installed in the same room as the room in which the supporter computer is installed. The room in which the supporter computer is installed and the photography studio may be separated by a glass window. In this case, the operator of the supporter computer can see the broadcast user. The supporter computer may be configured to be able to change the settings of various devices provided in the photography studio in response to the operation of the supporter. The supporter computer can, for example, change the setting of the scanning interval by the base station, the setting of the tracking sensor, and various settings of various other devices. The operator may input a message to the supporter computer, and the input message may be displayed on the display of the photography studio.

[0112] The generation unit 271 generates character video data including facial motion data and / or body motion data generated at any time, and outputs the generated character video data to the display processing unit 274. Hereinafter, facial motion data and body motion data may be collectively referred to as motion data. The generation unit 271 also outputs the generated character video data and the user ID of the broadcast user stored in the terminal storage device 22 to the transmission unit 272. When the generation unit 271 acquires voice data of the broadcast user output from the microphone 26, it outputs the generated character video data, the acquired voice data, and the user ID of the broadcast user stored in the terminal storage device 22 to the transmission unit 272.

[0113] (Transmitter 272) In response to an instruction to start a broadcast by the broadcast user, the transmitting unit 272 of the broadcast user's terminal device 2 (broadcast terminal device) transmits information regarding a broadcast start instruction for instructing the broadcast user to start a broadcast event, together with the broadcast user's user ID stored in the terminal storage device 22, to the server device 3 via the terminal communication I / F 21. Hereinafter, the user ID of the broadcast user may be referred to as the broadcast user ID.

[0114] Additionally, the transmission unit 272 of the broadcast user's terminal device 2 (broadcast terminal device) transmits the broadcast user's character video data output from the generation unit 271 to the server device 3 via the terminal communication I / F 21 together with the broadcast user ID stored in the terminal storage device 22 as information for displaying a broadcast image of the broadcast user in a broadcast event. When the broadcast user's voice data is output from the generation unit 271 together with the broadcast user's character video data, the transmission unit 272 transmits the broadcast user's character video data and voice data together with the broadcast user ID to the server device 3 via the terminal communication I / F 21.

[0115] In addition, in response to an instruction to end the broadcast by the broadcast user, the transmitting unit 272 of the terminal device 2 (broadcasting terminal device) transmits information regarding a broadcast end instruction to instruct the broadcast user to end the broadcast event, together with the broadcast user ID stored in the terminal memory device 22, to the server device 3 via the terminal communication I / F 21.

[0116] Furthermore, in response to an instruction by the viewing user to start viewing a distributed image, the transmitting unit 272 of the terminal device 2 (viewing terminal device) of the viewing user transmits information regarding the viewing start instruction to the server device 3 via the terminal communication I / F 21 together with information regarding the distributed image and the user ID of the viewing user stored in the terminal storage device 22. The information regarding the distributed image is information for identifying the distributed image for which the viewing user has been instructed to start viewing, and is, for example, identification information of the distributed image (distributed image ID) or the distributed user ID of the distributed user distributing the distributed image. Hereinafter, the user ID of the viewing user may be referred to as the viewing user ID.

[0117] Furthermore, the transmission unit 272 transmits information indicating various instructions input by the user by operating the input device 24, together with the user ID stored in the terminal storage device 22, to the server device 3 via the terminal communication I / F 21. Various instructions from the user include, for example, addition instructions, modification instructions, and generation instructions. The information indicating the addition instructions is transmitted to the server device 3, together with the object ID of the additional object to be included in the distribution image of the distribution user, the distribution user ID, and the viewing user ID stored in the terminal storage device 22 of the terminal device 2 (viewing terminal device) of the viewing user.

[0118] When a partial object is selected by the broadcasting user and an instruction is given regarding the creation of a character object composed of the selected partial objects, a model ID for identifying model data of the character object and object IDs of multiple partial objects constituting the character object may be transmitted to server device 3 via terminal communication I / F 21 together with a user ID and the like stored in terminal storage device 22 of terminal device 2 (broadcasting terminal device) of the broadcasting user. In this case, server device 3 may store the user ID, model ID, and object IDs of multiple partial objects constituting the character object in server storage device 32 in association with each other.

[0119] Furthermore, the terminal device 2 (viewing terminal device) of the viewing user that has received the model ID for identifying model data of the character object of the distribution user, the object IDs of the multiple partial objects constituting the user's character object, and the user ID of the user via the server device 3 may store the received user ID, model ID, and object IDs of the multiple partial objects constituting the character object in the terminal storage device 22 in association with each other. In this way, by simply transmitting the user ID (or model ID) of the distribution user to the terminal device 2 (viewing terminal device) of the viewing user via the server device 3, the terminal device 2 (viewing terminal device) of the viewing user can extract the object IDs of the multiple partial objects associated with the received user ID (or model ID). Then, the terminal device 2 (viewing terminal device) of the viewing user can display a distribution image including the character object of the distribution user based on the three-dimensional model data (of each partial object) corresponding to each extracted object ID.

[0120] In response to an instruction from the user to start a new information exchange, the transmitting unit 272 of the user's terminal device 2 transmits information regarding the new start instruction to the server device 3 via the terminal communication I / F 21. Together with the information regarding the new start instruction, the user ID of the other user designated by the user and the user ID stored in the terminal storage device 22 of the terminal device 2 are transmitted to the server device 3. The other users designated by the user are one or more users designated by the user as partners for information exchange. Hereinafter, a group to which the user and one or more other users belong may be referred to as an information exchange group.

[0121] In response to an instruction from the user to start information exchange (e.g., text chat) within the information exchange group, the transmitting unit 272 of the terminal device 2 of the user belonging to the information exchange group transmits information related to the information exchange start instruction to the server device 3 via the terminal communication I / F 21. Together with the information related to the information exchange start instruction, the group ID of the information exchange group to which the user belongs, stored in the terminal storage device 22 of the terminal device 2, is transmitted as destination information to the server device 3. Note that if the server device 3 does not store the user IDs of the users belonging to the information exchange group, destination information including the group ID indicating the information exchange group or the user IDs of other users belonging to the information exchange group may be transmitted to the server device 3 together with the information related to the information exchange start instruction.

[0122] When input information is input by the user while the information exchange screen is being displayed, the transmitting unit 272 of the terminal device 2 of a user belonging to the information exchange group transmits the input information to the server device 3 via the terminal communication I / F 21. Together with the input information, the user ID and destination information stored in the terminal storage device 22 of the terminal device 2 of the user belonging to the information exchange group are transmitted to the server device 3. The destination information is a group ID indicating the information exchange group, or a user ID of another user belonging to the information exchange group.

[0123] (Receiving unit 273) The receiving unit 273 of the terminal device 2 (viewing terminal device) of the viewing user receives the character video data and user ID of the broadcasting user transmitted from the server device 3 via the terminal communication I / F 21. The receiving unit 273 of the terminal device 2 (viewing terminal device) of the viewing user receives the character video data, voice data, and user ID of the broadcasting user transmitted from the server device 3 via the terminal communication I / F 21. The receiving unit 273 of the terminal device 2 (viewing terminal device) of the viewing user receives the voice data and user ID of the broadcasting user transmitted from the server device 3 via the terminal communication I / F 21. In addition, the receiving unit 273 receives information indicating various instructions of other users, etc. transmitted from the server device 3, via the terminal communication I / F 21. The receiving unit 273 passes the received information to the terminal processing device 27.

[0124] The broadcast user's character animation data and user ID transmitted from server device 3 were transmitted to server device 3 by the transmitting unit 272 of the broadcast user's terminal device 2 (broadcast terminal device). In addition, the broadcast user's character animation data, voice data and user ID transmitted from server device 3 were transmitted to server device 3 by the transmitting unit 272 of the broadcast user's terminal device 2 (broadcast terminal device). In addition, the broadcast user's voice data and user ID transmitted from server device 3 were transmitted to server device 3 by the transmitting unit 272 of the broadcast user's terminal device 2 (broadcast terminal device).

[0125] The receiving unit 273 of the terminal device 2 of a user belonging to the information exchange group receives information regarding the new start instruction sent from the server device 3 and the user ID of the user belonging to the information exchange group in response to information regarding the new start instruction sent from the terminal device 2 of another user belonging to the information exchange group.

[0126] The receiving unit 273 of the terminal device 2 of the user belonging to the information exchange group receives, via the terminal communication I / F 21, the input information and user IDs of the other users belonging to the information exchange group transmitted from the server device 3.

[0127] (Display processing unit 274) In response to an instruction to start a distribution event, the display processing unit 274 of the terminal device 2 (distribution terminal device) of the distribution user displays an image drawn based on the character video data including the motion data generated by the generation unit 271 as a distribution image of the distribution user on the display device 23. The motion data is transmitted to the terminal device 2 (viewing terminal device) of the viewing user as character video data. The display processing unit 274 of the terminal device 2 (viewing terminal device) of the viewing user also displays an image drawn based on the character video data (motion data) of the distribution user received by the reception unit 273 on the display device 23 as a distribution image of the distribution user.

[0128] The drawing process of the delivered image based on the motion data will be described below. In the following description, the drawing process of the delivered image based on the motion data in the delivery terminal device uses the user ID stored in the terminal storage device 22, and the drawing process of the delivered image based on the motion data in the viewing terminal device uses the user ID received from the server device 3.

[0129] First, the display processing unit 274 extracts the model ID of the character object associated with the user ID and the object ID of the used object from the user table T1. Next, the display processing unit 274 reads out the model data associated with the extracted model ID and the image information and arrangement location associated with the extracted object ID of the used object from the terminal storage device 22. Next, the display processing unit 274 generates an animation of the character object in which the used object is attached to the arrangement location based on the read model data, the image information of the used object, the arrangement location, and the motion data. Note that, when the object IDs of a plurality of partial objects constituting the character object are associated with the model ID of the character object, the display processing unit 274 extracts model data corresponding to the object IDs of the plurality of partial objects associated with the extracted model ID. Next, the display processing unit 274 generates an animation of the character object in which the used object is attached to the arrangement location based on the read model data of the plurality of partial objects, the image information of the used object, the arrangement location, and the motion data. Note that the used object may be included in a plurality of partial objects.

[0130] When the motion data includes face motion data, the display processing unit 274 generates animation of the character object so that the facial expression of the character object changes based on the face motion data. As a result, the display processing unit 274 can generate animation of the character object that moves in synchronization with the facial expression of the broadcast user based on the read model data and face motion data. When the motion data includes face motion data and body motion data, the display processing unit 274 generates animation of the character object that moves in synchronization with the facial expression and body movement of the broadcast user based on the read model data, face motion data, and body motion data. Next, the display processing unit 274 draws a delivery image including the generated animation (moving image) of the character object and a background image showing the background generated using the background data stored in the terminal storage device 22. Then, the display processing unit 274 outputs drawing data showing the drawn delivery image, and ends the drawing process of the delivery image based on the motion data. When three-dimensional model data is used in the drawing process of the delivery image, the delivery image drawn in the drawing process of the delivery image is a moving image drawn in a three-dimensional virtual space including a character object that moves in synchronization with the facial expression and body movement of the broadcast user. Hereinafter, a description will be given mainly of a delivery image including an animation (moving image) of a character object, but the delivery image is not limited to an image including an animation (moving image) of a character object. The delivery image may include a still image of a character object. Furthermore, the delivery image may be an image in which a new still image of a character object is displayed at predetermined intervals.

[0131] In addition, the display processing unit 274 may execute a change process to display on the display device 23 a character object to which a wearable object that is an object owned by the distribution user or a wearable object sent by the viewing user as a wearable gift is applied, in response to a change instruction from the distribution user. For example, if the wearable object instructed to be changed by the distribution user is a "headband", the display processing unit 274 extracts the object ID of the wearable object ("headband") from the owned objects associated with the user ID of the distribution user in the user table T1, and stores it as an object used by the distribution user. The display processing unit 274 extracts image information and a placement location associated with the object ID stored as an object used by the distribution user from the object table T2. Next, the display processing unit 274 generates a distribution image including a character object of the distribution user wearing a wearable object ("headband") on the head (placement location associated with the object ID of the "headband") based on the extracted image information and placement location. Note that the display processing unit 274 may generate a distribution image including a character object wearing a wearable object that moves in conjunction with the movement of the wearing part (placement location) of the character object. For example, in the delivered image, the attachment object "headband" is displayed so as to move in conjunction with the movement of the character object's head.

[0132] Furthermore, the display processing unit 274 of the terminal device 2 of the viewing user (viewing terminal device) displays a delivery image including the additional object in response to an instruction to add the additional object from the viewing user. Furthermore, the display processing unit 274 of the terminal device 2 of the broadcasting user (broadcasting terminal device) displays a delivery image including the additional object in response to information indicating an instruction to add the additional object received from the terminal device 2 of the viewing user.

[0133] When the object ID received from the terminal device 2 (viewing terminal device) of the viewing user together with the information indicating an instruction to add an additional object is the object ID of one of the effect objects, the display processing unit 274 of the terminal device 2 (viewing terminal device) of the distribution user refers to the object table T2, extracts a still image or a moving image (image information) of the effect object associated with the object ID received together with the information indicating the instruction to add from the terminal storage device 22, and generates a distribution image including the extracted still image or moving image. For example, when an instruction to add an effect object showing confetti or fireworks is given, the display processing unit 274 of the terminal device 2 (viewing terminal device) of the distribution user generates a distribution image including a moving image of an effect object (not shown) imitating confetti or fireworks. When the object ID received from the terminal device 2 (viewing terminal device) together with the information indicating an instruction to add an additional object from the viewing user is the object ID of one of the effect objects, the display processing unit 274 of the terminal device 2 (viewing terminal device) of the viewing user similarly refers to the object table T2, extracts a still image or a moving image (image information) of the effect object associated with the object ID received together with the information indicating the instruction to add from the terminal storage device 22, and generates a distribution image of the viewing user including the extracted still image or moving image.

[0134] When the object ID received together with the information indicating an instruction to add an additional object from the viewing user's terminal device 2 (viewing terminal device) is the object ID of a normal object, the display processing unit 274 of the terminal device 2 (viewing terminal device) of the broadcasting user refers to the object table T2 and extracts a still image or video (image information) of the normal object and the placement location associated with the object ID of the normal object. Next, the display processing unit 274 of the terminal device 2 (broadcast terminal device) of the broadcasting user generates a broadcast image of the viewing user including a still image or video of the normal object placed at the position indicated by the placement location. In addition, when the object ID received together with the information indicating an instruction to add an additional object from the viewing user is the object ID of a normal object, the display processing unit 274 of the terminal device 2 (viewing terminal device) of the viewing user similarly refers to the object table T2 and extracts a still image or video (image information) of the normal object and the placement location associated with the object ID of the normal object, and generates a broadcast image of the viewing user including a still image or video of the normal object placed at the position indicated by the placement location.

[0135] The display processing unit 274 may generate a delivery image including a moving image of a normal object moving in a display area of ​​the delivery image. For example, the display processing unit 274 may generate a delivery image including a moving image of a normal object that moves as if the normal object is falling from the top to the bottom of the delivery image. In this case, the normal object may be displayed in the display area of ​​the delivery image from the start of the fall until it falls to the bottom end of the output image, and may disappear from the delivery image after it falls to the bottom end of the delivery image. Note that the movement route of the normal object may be a route from left to right, right to left, or top left to bottom left of the delivery image, or a route along a straight orbit, a circular orbit, an elliptical orbit, a spiral orbit, or a route along a orbit other than these.

[0136] When the object ID received together with the information indicating an instruction to add an additional object from the viewing user is the object ID of a wearing object, the display processing unit 274 of the terminal device 2 (distribution terminal device) displays notification information regarding the wearing instruction in the distribution image. Next, the display processing unit 274 refers to the object table T2 and extracts image information and a placement location of the wearing object associated with the object ID received together with the information indicating the addition instruction. Next, the display processing unit 274 generates a distribution image including the character object of the broadcast user wearing the wearing object at the wearing part indicated by the placement location based on the extracted image information and placement location. Also, when the object ID received together with the information indicating an instruction to add an additional object from the viewing user is the object ID of a wearing object, the display processing unit 274 of the terminal device 2 (viewing terminal device) displays notification information regarding the wearing instruction in the distribution image. Next, the display processing unit 274 refers to the object table T2 and extracts image information and a placement location of the wearing object associated with the object ID received together with the information indicating the addition instruction. Next, based on the extracted image information and placement location, the display processing unit 274 generates a delivery image including the broadcast user's character object wearing the attachment object on the attachment part indicated by the placement location.

[0137] Furthermore, in response to an instruction from the user to start a new information exchange, the display processing unit 274 of the user's terminal device 2 displays an information exchange screen related to a new information exchange group to which the user and all other users designated by the user belong. Furthermore, when the receiving unit 273 receives information related to a new start instruction from other users and the user IDs of each user belonging to the new information exchange group, the display processing unit 274 of the user's terminal device 2 displays the information exchange screen related to the information exchange group.

[0138] Furthermore, the display processing unit 274 of the terminal device 2 of a user belonging to an information exchange group displays an information exchange screen related to the information exchange group in response to an instruction from the user to start information exchange within the information exchange group. Furthermore, when the receiving unit 273 receives information related to an instruction to start information exchange from other users belonging to the information exchange group and the user IDs of the users belonging to the information exchange group, the display processing unit 274 of the terminal device 2 of a user belonging to an information exchange group displays an information exchange screen related to the information exchange group.

[0139] Furthermore, when input information is input by the user while the information exchange screen is displayed, the display processing unit 274 of the terminal device 2 of the user belonging to the information exchange group displays the input information below the most recently input information in the user display area of ​​the information exchange screen. Furthermore, when input information by another user is received by the receiving unit 273 while the information exchange screen is displayed, the display processing unit 274 of the terminal device 2 of the user belonging to the information exchange group displays the received input information below the most recently input information in the other user display area of ​​the information exchange screen.

[0140] (Audio output unit 275) When the audio data of the broadcast user is received by the receiving unit 273, the audio output unit 275 of the viewing user's terminal device 2 (viewing terminal device) outputs the broadcast user's voice generated based on the audio data from a speaker (not shown). Furthermore, when the audio output unit 275 of the viewing terminal device receives the broadcast user's character video data and audio data, it outputs the broadcast user's voice based on the audio data from a speaker along with the display of the broadcast user's broadcast image by the display processing unit 274. As a result, the broadcast user's voice is output during the movement of the lips of the broadcast user's character object included in the broadcast image, so the viewing user can view a broadcast image including the broadcast user's character object that appears to be making a sound.

[0141] (Server device 3) FIG. 6 is a diagram showing an example of a schematic configuration of the server device 3. The server device 3, which provides various services such as a video distribution service and an information exchange service to each of the terminal devices 2 of a plurality of users, transmits information indicating character video data of the distribution user and various instructions transmitted from the terminal device 2 of the distribution user to the terminal device 2 of one or more viewing users. The server device 3 also transmits input information (text information, stamp image, etc.) received from the terminal device of one user to the terminal device 2 of one or more other users belonging to the same information exchange group as the one user. The server device 3 may also transmit information received from the terminal device 2 of one user to the terminal device 2 of a user indicated by a user ID included in the destination information received simultaneously with the information. To this end, the server device 3 includes a server communication I / F 31, a server storage device 32, and a server processing device 33.

[0142] The server communication I / F 31 is implemented as hardware, firmware, or communication software such as a TCP / IP driver or a PPP driver, or a combination of these. The server device 3 can transmit information to other devices and receive information from other devices via the server communication I / F 31.

[0143] The server storage device 32 is, for example, a semiconductor memory device such as a ROM or a RAM. The server storage device 32 may be, for example, a magnetic disk, an optical disk, or any other storage device capable of storing data. The server storage device 32 stores an operating system program, a driver program, an application program, data, and the like used in processing in the server processing device 33. The server storage device 32 may store, as data, a user table T1, an object table T2, and a group table T3. The server storage device 32 may store, as data, a broadcast user ID of a broadcast user currently broadcasting, and stores a user ID (viewing user ID) of a viewing user currently viewing a broadcast image of the broadcast user in association with the broadcast user ID of the broadcast user. The broadcast user ID is a type of information related to a broadcast image. The information related to a broadcast image may be identification information (broadcast image ID) of the broadcast image. For example, a broadcast image ID, a broadcast user ID, and a viewing user ID may be stored in association with each other.

[0144] The server processing device 33 is a processor that loads the operating system program, the driver program, and the control program stored in the server storage device 32 into a memory and executes instructions included in the loaded programs. The server processing device 33 is, for example, an electronic circuit such as a CPU, an MPU, a DSP, or a GPU, or a combination of various electronic circuits. The server processing device 33 may be realized by an integrated circuit such as an ASIC, a PLD, an FPGA, or an MCU. In FIG. 6, the server processing device 33 is illustrated as a single component, but the server processing device 33 may be a collection of multiple physically separate processors. For example, multiple processors that operate in parallel in a cooperative manner to execute instructions may be implemented. The server processing device 33 functions as a server receiving unit 331 and a server transmitting unit 332 by executing various instructions included in the control program.

[0145] (Server receiving unit 331) The server receiving unit 331 receives information regarding a broadcast start instruction sent from the broadcast user's terminal device 2 (broadcast terminal device) together with the broadcast user ID via the server communication I / F 31. The server receiving unit 331 receives information regarding a broadcast end instruction sent from the broadcast user's terminal device 2 (broadcast terminal device) together with the broadcast user ID via the server communication I / F 31. The server receiving unit 331 also receives information for displaying the broadcast user's broadcast image sent from the broadcast user's terminal device 2 (broadcast terminal device) together with the broadcast user ID via the server communication I / F 31.

[0146] Furthermore, the server receiving unit 331 receives information regarding a viewing start instruction transmitted from the viewing user's terminal device 2 (viewing terminal device) together with information regarding the distributed image and the viewing user ID via the server communication I / F 31. The server receiving unit 331 receives information indicating an additional instruction transmitted from the viewing user's terminal device 2 (viewing terminal device) together with the object ID, the distribution user ID, and the viewing user ID via the server communication I / F 31.

[0147] The server receiving unit 331 also receives information related to a new start instruction, the user ID of the user, and the user ID of the other user selected as the partner of the information exchange, from the terminal device 2 of the user who wishes to start a new information exchange (e.g., text chat), via the server communication I / F 31. In this case, the server receiving unit 331 sets the group ID of the new information exchange group. Then, the server receiving unit 331 stores the received user ID of the user and the user ID of the other user as user IDs belonging to the new information exchange group in the server storage device 32 in association with the group ID of the new information exchange group.

[0148] Furthermore, the server receiving unit 331 receives information related to an information exchange start instruction, the user ID of the user, and destination information from the terminal device 2 of a user belonging to the information exchange group via the server communication I / F 31. When the received destination information is a group ID indicating an information exchange group, the server receiving unit 331 refers to the server storage device 32 and extracts the user ID of the user associated with the group ID received as the destination information. Then, the server receiving unit 331 specifies the extracted user ID as the user ID of the user of the destination terminal device 2. When the received destination information is a user ID, the server receiving unit 331 specifies the user ID received as the destination information as the user ID of the user of the destination terminal device 2. Then, the server receiving unit 331 passes the received information related to the information exchange start instruction and the user ID of the user of the destination terminal device 2 to the server transmitting unit 332.

[0149] The server receiving unit 331 also receives input information input by a user, the user ID of the user, and destination information from the terminal device 2 of a user belonging to the information exchange group via the server communication I / F 31. Next, the server receiving unit 331 extracts the user ID of the user of the destination terminal device 2 based on the received destination information. The server receiving unit 331 passes the received input information, the user ID of the user who input the input information, and the user ID of the user of the destination terminal device 2 to the server transmitting unit 332.

[0150] (Server transmission unit 332) The server transmitting unit 332 transmits the broadcast user's character video data and user ID received by the server receiving unit 331 via the server communication I / F 31 to the viewing user's terminal device 2 (viewing terminal device) indicated by the viewing user ID associated with the broadcast user ID of the broadcast user stored in the server storage device 32. The server transmitting unit 332 transmits the broadcast user's character video data, audio data, and user ID received by the server receiving unit 331 via the server communication I / F 31 to the viewing user's terminal device 2 (viewing terminal device) indicated by the viewing user ID associated with the broadcast user ID of the broadcast user stored in the server storage device 32.

[0151] Additionally, the server transmitting unit 332 transmits information indicating various instructions from the broadcast user received by the server receiving unit 331, via the server communication I / F 31, to the terminal device 2 (viewing terminal device) of the viewing user indicated by the broadcast user ID associated with the broadcast user ID of that broadcast user stored in the server storage device 32. Additionally, the server transmitting unit 332 transmits information indicating various instructions from the viewing user received by the server receiving unit 331, via the server communication I / F 31, to the terminal device 2 of the broadcast user indicated by the broadcast user ID associated with the user ID of that viewing user, and to the terminal device 2 of a viewing user indicated by the user ID of another viewing user associated with that broadcast user ID.

[0152] Furthermore, in response to the setting of the group ID of the new information exchange group by the server receiving unit 331, the server transmitting unit 332 extracts the user IDs belonging to the new information exchange group, which are associated with the group ID of the new information exchange group, from the server storage device 32. Then, the server transmitting unit 332 transmits information regarding a display instruction for an information exchange screen for the new information exchange group, via the server communication I / F 31, to the terminal device 2 of the user identified by the extracted user ID.

[0153] In addition, when the server transmitting unit 332 receives information regarding an information exchange start instruction and destination information from the server receiving unit 331, it transmits information regarding an instruction to display an information exchange screen regarding the received information exchange start instruction to the terminal device 2 of the user identified by the user ID related to the destination information, via the server communication I / F 31.

[0154] In addition, when the server transmitting unit 332 receives input information, the user ID of the user who input the input information, and the user ID of the user of the destination terminal device 2 from the server receiving unit 331, it transmits the received input information via the server communication I / F 31 to the terminal device 2 of a user identified by a user ID of the user of the destination terminal device 2 other than the user ID of the user who input the input information.

[0155] (Examples of various screens) Examples of various screens displayed on the display device 23 of the user's terminal device 2 will be described below with reference to Fig. 7 to Fig. 15. In Fig. 7 to Fig. 15, multiple components denoted with the same reference numerals are components having equivalent functions.

[0156] Fig. 7(a) is a diagram showing an example of a distribution screen 600 displayed on the display device 23 of the terminal device 2 (distribution terminal device) of the distribution user. The distribution screen 600 includes a distribution image 602 including a character object 601, a response information display area 603, and a distribution end button 604. Fig. 7(b) is a diagram showing an example of a distribution screen 610 displayed on the display device 23 of the terminal device 2 (viewing terminal device) of the viewing user. The distribution screen 610 includes a distribution image 612 including a character object 611, a rating button 613, an add button 614, a comment button 615, a response information display area 616, and a viewing end button 617.

[0157] In response to the start of a distribution event, the terminal device 2 (distribution terminal device) of the distribution user executes a process of transmitting information related to a distribution start instruction by the transmission unit 272 to the server device 3. The distribution event is started when an instruction to start a distribution event by the distribution user is input from the input device 24 to the terminal processing device 27 in response to an operation by the distribution user on a predetermined screen (not shown) such as a home screen displayed on the display device 23. Next, the terminal device 2 (distribution terminal device) simultaneously (in parallel) executes a process of transmitting character video data, etc. by the transmission unit 272 and a process of displaying the distribution screen 600 by the display processing unit 274.

[0158] When the receiving unit 273 of the terminal device 2 (viewing terminal device) of the viewing user receives information indicating one or more broadcast users currently broadcasting, the display processing unit 274 of the terminal device 2 (viewing terminal device) displays a broadcast event selection screen (not shown) on the display device 23. The broadcast event selection screen includes option objects for selecting one or more broadcast users currently broadcasting. Next, the transmitting unit 272 of the terminal device 2 (viewing terminal device) acquires information indicating a broadcast user selected from one or more broadcast users included in the broadcast event selection screen in response to the viewing user's operation of the input device 24 from the input device 24. In this way, the transmitting unit 272 accepts a viewing start instruction for the broadcast image from the viewing user from the input device 24. Then, the transmitting unit 272 transmits information regarding the viewing start instruction including information indicating the selected broadcast user, information regarding the broadcast image, and the user ID of the viewing user stored in the terminal storage device 22 to the server device 3 via the terminal communication I / F 21.

[0159] The server receiving unit 331 of the server device 3 receives the information on the viewing start instruction, the information on the distributed image, and the user ID of the viewing user transmitted from the terminal device 2 (viewing terminal device) via the server communication I / F 31. Next, the server transmitting unit 332 of the server device 3 specifies the terminal device 2 (viewing terminal device) of the viewing user identified by the user ID received together with the information on the viewing start instruction. Next, the server transmitting unit 332 distributes information for displaying the distributed image identified by the information on the distributed image received together with the information on the viewing start instruction to the specified terminal device 2 (viewing terminal device) via the server communication I / F 31. Note that the information for displaying the distributed image is character video data of the distribution user, or character video data and audio data of the distribution user.

[0160] When the character video data is received by the receiving unit 273 of the terminal device 2 (viewing terminal device), the display processing unit 274 of the terminal device 2 (viewing terminal device) displays the distribution screen 610 based on the received character video data. When the character video data and audio data are received by the receiving unit 273 of the terminal device 2 (viewing terminal device), the display processing unit 274 of the terminal device 2 (viewing terminal device) displays the distribution screen 610 based on the received character video data, and the audio output unit 275 of the terminal device 2 (viewing terminal device) outputs the audio of the distribution user generated based on the audio data from a speaker (not shown). The server receiving unit 331 of the server device 3 registers (stores in the server storage device 32) the viewing user ID of the viewing user of the terminal device 2 (viewing terminal device) that transmitted the information related to the viewing start instruction as the viewing user ID of the viewing user currently viewing.

[0161] Each display element included in the distribution screen 600 will be described below.

[0162] The character object 601 included in the distribution screen 600 is an animation (moving image) of the broadcast user's character object generated by the display processing unit 274. The display processing unit 274 generates animation of the broadcast user's character object based on the generated motion data of the broadcast user and the character object associated with the broadcast user's user ID stored in the user table T1. When the motion data includes face motion data, the display processing unit 274 changes the facial expression of the character object 601 so as to synchronize with the facial movement of the broadcast user. When the motion data includes body motion data, the display processing unit 274 moves the corresponding part of the character object 601 so as to synchronize with the body movement of the broadcast user. The display processing unit 274 displays a distribution image 602 including the generated character object 601 on the display device 23.

[0163] The response information display area 603 included in the distribution screen 600 includes response information (comment information, rating information, and / or information related to additional objects) transmitted from each terminal device 2 (viewing terminal device) displaying the distribution screen 610.

[0164] In the example of the response information display area 603 shown in Fig. 7(a), it is shown that comment information indicating the comment "cool" has been transmitted from the terminal device 2 (viewing terminal device) of a viewing user named "User R". Next, it is shown that evaluation information has been transmitted from the terminal device 2 (viewing terminal device) of a viewing user named "User F". Then, it is shown that evaluation information has most recently been transmitted from the terminal device 2 (viewing terminal device) of a viewing user named "User A".

[0165] The distribution end button 604 included in the distribution screen 600 is an object image such as a button object for ending the display of the distribution screen 600 and ending the distribution of the distribution image 612. When the distribution end button 604 is selected by the distribution user operating the input device 24, the display processing unit 274 of the terminal device 2 (distribution terminal device) of the distribution user ends the display of the distribution screen 600. Then, the transmission unit 272 of the terminal device 2 (distribution terminal device) transmits information regarding the distribution end instruction to the server device 3 via the terminal communication I / F 21 together with the distribution user ID stored in the terminal storage device 22. When the server receiving unit 331 of the server device 3 receives information regarding the distribution end instruction transmitted from the terminal device 2 (distribution terminal device) of the distribution user, it deletes the distribution user ID received together with the information regarding the distribution end instruction from the server storage device 32. Furthermore, the server receiving unit 331 deletes the viewing user ID associated with the deleted distribution user ID from the server storage device 32. This ends the distribution of the distribution image of the distribution user who selected the distribution end button 604. The broadcast user ID of the broadcast user who has ended the broadcast may not be deleted from the server storage device 32. In this case, the broadcast user ID of the broadcast user who has ended the broadcast may be stored in the server storage device 32 as a broadcast end user ID. Furthermore, the viewing user ID associated with the broadcast user ID of the broadcast user who has ended the broadcast may not be deleted from the server storage device 32. In this case, the viewing user ID associated with the broadcast user ID of the broadcast user who has ended the broadcast may be stored in the server storage device 32 as a viewing end user ID. Furthermore, the server transmission unit 332 may transmit information regarding a broadcast end instruction to the terminal device 2 (viewing terminal device) of the viewing user identified by the viewing user ID associated with the broadcast user ID of the broadcast user who has ended the broadcast. In this case, the display of the broadcast screen 610 will also end in response to receiving information regarding the broadcast end instruction on the terminal device 2 (viewing terminal device) of the viewing user that is displaying the broadcast image of the broadcast user.

[0166] Each display element included in the distribution screen 610 will be described below.

[0167] Character object 611 included in distribution screen 610 is an animation (moving image) of the broadcast user's character object generated by display processing unit 274. Display processing unit 274 generates animation of the broadcast user's character object based on the character moving image data (motion data) of the broadcast user received by receiving unit 273 and the character object associated with the broadcast user's user ID stored in user table T1. An example of display processing of a distribution image 612 including character object 611 will be described below.

[0168] When the server device 3 receives information regarding an instruction to start viewing the distribution image of the distribution user, information regarding the distribution image (distribution user ID of the distribution image), and the user ID of the viewing user from the viewing user's terminal device 2 (viewing terminal device), it stores the received viewing user ID of the viewing user in association with the distribution user ID of the received distribution image.

[0169] When the broadcast user's character video data (motion data) received by the receiving unit 273 of the terminal device 2 (viewing terminal device) of the viewing user includes face motion data of the broadcast user, a broadcast image 612 including a character object 611 whose facial expression changes in synchronization with the broadcast user's facial expression is displayed. For example, when the broadcast user emits a sound, a broadcast image 612 including a character object 611 whose lips are moving is displayed. Also, when the broadcast user's motion data received by the receiving unit 273 includes the broadcast user's body motion data, a broadcast image 612 including a character object 611 whose corresponding body part moves in synchronization with the broadcast user's body movement is displayed. When the broadcast user's character video data and voice data are received, a broadcast image 612 based on the received character video data is displayed by the display processing unit 274, and the voice of the broadcast user generated based on the received voice data of the broadcast user is output as voice by the voice output unit 275. This allows user A to hear the voice of the broadcast user while watching the broadcast user's character object 611 that is moving as if it is emitting a sound. This completes the display process of the delivered image 612. The display process of the delivered image 612 is executed every time output information of a broadcast user is received at a predetermined time interval (for example, every two seconds).

[0170] The rating button 613 included in the distribution screen 610 is an object image for instructing the transmission of rating information regarding the character object 611 included in the distribution screen 610, or the transmission of rating information regarding the broadcast user related to the character object 611. When the rating button 613 is selected in response to the operation of the input device 24 by the viewing user, the transmission unit 272 of the terminal device 2 (viewing terminal device) transmits, to the server device 3 via the terminal communication I / F 21, information regarding an instruction to display the rating information, together with the rating information indicating that a "positive rating has been given to the character object 611 included in the distribution screen 610" or the rating information indicating that a "positive rating has been given to the broadcast user related to the character object 611 included in the distribution screen 610". The server receiving unit 331 of the server device 3 may store the received rating information in the server storage device 32, and the server transmitting unit 332 of the server device 3 may transmit the received rating information to the terminal device 2 (broadcast terminal device) of the broadcast user.

[0171] The rating button 613 may be an object image for instructing the transmission of rating information indicating that "a negative rating has been made to the character object 611 included in the distribution screen 600" or rating information indicating that "a negative rating has been made to the broadcast user related to the character object 611 included in the distribution screen 600." Furthermore, the distribution screen 610 may include multiple types of rating buttons 613 (for example, two types of rating buttons, a rating button for sending positive rating information and a rating button for sending negative rating information).

[0172] When the distribution screen 610 is displayed on a plurality of terminal devices 2 (viewing terminal devices), aggregate information that aggregates the evaluation information transmitted from each terminal device 2 (viewing terminal device) may be included in the distribution screen 600 and / or the distribution screen 610. For example, the server device 3 may count the number of evaluation information transmitted from each of the plurality of terminal devices 2 (viewing terminal devices) and generate display information for including the number of evaluation information in the distribution screen 600 and / or the distribution screen 610. The aggregate information is not limited to the number of evaluation information, and may be information on the time period in which the evaluation information was transmitted. The aggregation period may be a period from the start of the distribution event to the current time, a period from the start of the display of the distribution screen 600 to the current time, a period from the start of the distribution screen 610 to the current time, or a predetermined period (for example, a period from 9 p.m. to midnight). A plurality of aggregation periods may be set. When the server device 3 can transmit the evaluation information to the terminal device 2 (distribution terminal device) of the distribution user, the aggregation process of the evaluation information may be executed by the terminal device 2 (distribution terminal device). In this case, the process of including the aggregate information in the distribution screen 600 may be performed by the terminal device 2 (distribution terminal device), and the process of including the aggregate information in the distribution screen 610 may be performed by the terminal device 2 (viewing terminal device) displaying the distribution screen 610.

[0173] The add button 614 included in the distribution screen 610 is an object image for instructing the display of a selection screen 700 for selecting an additional object to be displayed on the distribution screen 600 and the distribution screen 610. When the add button 614 is selected in response to an operation of the input device 24 by the viewing user, the selection screen 700 is displayed on the terminal device 2 (viewing terminal device) of the viewing user. Details of the selection screen 700 will be described later.

[0174] The comment button 615 included in the distribution screen 610 is an object image for instructing the display of a comment input window (not shown) for inputting a comment (character string) to be displayed on the distribution screen 600 and the distribution screen 610. When the comment button 615 is selected in response to an operation of the input device 24 by the viewing user, the comment input window (not shown) is superimposed on the distribution screen 610.

[0175] The comment input window is displayed only on the display device 23 of the terminal device 2 of the viewing user who selected the comment button 615, and the character string being entered in the comment input window is not displayed on the display device 23 of the other terminal devices 2. For example, the display processing unit 274 of the terminal device 2 of the viewing user creates comment information indicating the comment (character string) entered in the comment input window in response to the viewing user's operation of the input device 24. The comment information is a type of response information.

[0176] The response information display area 616 included in the distribution screen 610 includes response information (comment information, rating information, and / or information regarding additional objects (described below)) created in response to operations by the viewing user, as well as response information transmitted from the terminal devices 2 of other viewing users viewing the distribution image 612 of the same distribution user.

[0177] The example of the response information display area 616 shown in Fig. 7(b) shows that comment information indicating a comment (character string) "cool" has been transmitted from the terminal device 2 of a viewing user named "User R". It also shows that evaluation information has been transmitted from the terminal device 2 of a viewing user named "User F". It also shows that evaluation information created by the terminal device 2 of user A who operates this terminal device 2 has been transmitted.

[0178] When response information is created by the viewing user of the terminal device 2 and a transmission instruction is input, the transmitting unit 272 of the terminal device 2 displaying the distribution screen 610 transmits information related to an instruction to display the response information together with the created response information to the server device 3 via the terminal communication I / F 21. Note that the viewing user ID of the viewing user stored in the terminal storage device 22 is transmitted together with the information related to the instruction to display the response information.

[0179] The server receiving unit 331 of the server device 3 receives information related to a display instruction for the response information transmitted from the terminal device 2 displaying the distribution screen 610, together with the response information and the viewing user ID. The server transmitting unit 332 of the server device 3 extracts the broadcast user ID stored in association with the received viewing user ID from the server storage device 32. The server transmitting unit 332 then transmits the received response information and the viewing user ID to the terminal device 2 of the broadcast user indicated by the extracted broadcast user ID and the terminal device 2 of the viewing user indicated by the extracted viewing user ID. The receiving unit 273 of each terminal device 2 displaying the distribution screen 610 receives the response information and the viewing user ID transmitted from the server device 3 via the terminal communication I / F 21. The display processing unit 274 of each terminal device 2 displaying the distribution screen 610 displays the distribution screen 610 in which the response information and the name of the viewing user who inputted the response information are arranged in chronological order in the response information display area 616, based on the received response information and the name associated with the received viewing user ID. The name associated with the received user ID is information extracted by referring to the user table T1 by the display processing unit 274. In this manner, the display processing unit 274 of each terminal device 2 displaying the distribution screen 610 displays the received response information and the name of the viewing user who input the response information in chronological order in the response information display area 616 every time response information is received.

[0180] An upper limit may be set for the number of pieces of response information that can be displayed in the response information display area 603 and / or the response information display area 616. In the example of the response information display area 603 shown in Fig. 7(a) and the response information display area 616 shown in Fig. 7(b), the upper limit for the number of pieces of response information that can be displayed is "3." When the number of pieces of response information displayed on each terminal device 2 exceeds the set upper limit, the display processing unit 274 of each terminal device 2 displays the delivery screen 600 including the three newly received pieces of response information in the response information display area 603 in order, or the delivery screen 610 including the three pieces of newly received response information in the response information display area 616.

[0181] The response information in the response information display area 616 may be displayed in a scrollable manner. In this case, in response to a swipe operation by the user (for example, an operation in which a finger touches the screen and moves from the bottom to the top of the screen), the response information that is not currently included in the response information display area 616 is scrolled and displayed in the response information display area 616. The response information in the response information display area 603 included in the delivery screen 600 may also be displayed in a scrollable manner, similar to the response information display area 616.

[0182] The viewing end button 617 is an object image such as a button object for ending the display of the distribution screen 610 and ending the viewing of the distribution image 612. When the viewing end button 617 is selected by the viewing user operating the input device 24, the display processing unit 274 of the terminal device 2 of the viewing user ends the display of the distribution screen 610. Then, the transmission unit 272 of the terminal device 2 transmits information regarding the viewing end instruction to the server device 3 via the terminal communication I / F 21 together with the viewing user ID and the transmission destination information stored in the terminal storage device 22. When the server receiving unit 331 of the server device 3 receives the information regarding the viewing end instruction and the viewing user ID transmitted from the user's terminal device 2, it stores the viewing user ID, which is the same as the received viewing user ID and is associated with the distribution user ID of the distribution user related to the ended distribution screen 610, as the viewing end user ID in the server storage device 32. Note that the viewing end user ID is stored in association with the distribution user ID of the distribution user related to the ended distribution screen 610. This ends the viewing of the distribution image of the viewing user who selected the viewing end button 617.

[0183] Fig. 8(a) is a diagram showing an example of a selection screen 700 displayed on the display device 23 of the terminal device 2 (viewing terminal device) of a viewing user. As shown in Fig. 8(a), the selection screen 700 is displayed so as to be superimposed on the distribution screen 610. The selection screen 700 may be displayed later while the display of the distribution screen 610 is temporarily suspended, and the distribution screen 610 may be redisplayed in response to the end of the display of the selection screen 700.

[0184] In the example shown in FIG. 8(a), images showing eight kinds of candidate objects for addition are included in the display area of ​​the selection screen 700. A name showing the candidate object for addition may be arranged near the image showing the candidate object for addition. The number of candidate objects for addition included in the display area of ​​the selection screen 700 is not limited to eight, and may be seven or less, or nine or more. When a part of the image showing a plurality of candidate objects for addition cannot be included in the display area of ​​the selection screen 700, the display processing unit 274 of the terminal device 2 may display the image showing the candidate object for addition in a scrollable manner. In this case, in response to a swipe operation by the viewing user (for example, an operation in which a finger is moved from right to left on the screen while touching the screen), an image showing a candidate object for addition that is not currently included in the display area of ​​the selection screen 700 is scrolled and displayed in the display area of ​​the selection screen 700.

[0185] Each of the additional candidate objects indicates an additional object. The image indicating the additional candidate object is, for example, an icon image, a thumbnail image, or the like indicating a gift object, which is a type of additional object. In the selection screen 700, the image of each additional candidate object may be displayed classified according to the type of additional object (free gifts, paid gifts, accessories, support goods, appeals, variety, cooperation gifts that require the cooperation of users belonging to the group, etc.). In addition, an additional object belonging to a paid gift is an additional object that can be purchased by a user by spending "coins" of virtual monetary value. In addition, an additional object belonging to a free gift is an additional object that can be obtained by spending points that can be obtained by watching a broadcast instead of spending monetary value. In addition, the additional object may be an object that can be transmitted to the broadcasting user by the viewing user playing a predetermined mission executed on the terminal device 2 and clearing the predetermined mission. In addition, the additional object may be an object that can be transmitted to the broadcasting user by the viewing user playing a known lottery game executed on the terminal device 2 and winning the lottery game. In addition, the additional object may be managed (stored) as an object owned by the viewing user.

[0186] When an addition candidate object is selected in response to an operation of the input device 24 by the viewing user, the input device 24 inputs an addition instruction together with an object ID corresponding to the selected addition candidate object to the terminal processing device 27. An example of an addition process in response to an addition instruction will be described below.

[0187] First, when the display processing unit 274 acquires the input addition instruction and object ID, it refers to the object table T2 and extracts a still image or video image (image information) of the added object associated with the acquired object ID from the terminal storage device 22. Next, the display processing unit 274 generates a new delivery image 612 that includes the delivery image 612 including the delivery user's character object 611 and includes the extracted still image or video image.

[0188] Next, when the transmission unit 272 acquires the input addition instruction and object ID, it transmits information indicating the addition instruction together with the acquired object ID, the viewing user ID of the viewing user stored in the terminal storage device 22, and information regarding the distributed image (distributor user ID of the distributed image) to the server device 3 via the terminal communication I / F 21. Next, the server receiving unit 331 of the server device 3 receives the information indicating the addition instruction, the object ID, the viewing user ID, and information regarding the distributed image (distributor user ID of the distributed image) via the server communication I / F 31. Note that if the information regarding the distributed image (distributor user ID of the distributed image) is not transmitted, the server transmission unit 332 of the server device 3 extracts the distribution user ID stored in association with the received viewing user ID from the server storage device 32. The server transmission unit 332 extracts viewing user IDs other than the received viewing user ID from the information regarding the distributed image (distributor user ID of the distributed image) or the viewing user ID associated with the extracted distribution user ID. Then, the server transmitting unit 332 transmits information indicating the received addition instruction, the object ID, and the received user ID (user ID of the viewing user who issued the addition instruction) to the terminal device 2 of the viewing user indicated by the extracted viewing user ID, via the server communication I / F 31. In addition, the server transmitting unit 332 transmits information regarding the distributed image (distributor user ID of the distributed image) or information indicating the received addition instruction, the object ID, and the received viewing user ID to the terminal device 2 of the broadcast user indicated by the extracted broadcast user ID, via the server communication I / F 31.

[0189] Then, the receiving unit 273 of the terminal device 2 of the viewing user (excluding the viewing user who issued the addition instruction) and the broadcasting user receives information indicating the addition instruction and the user ID and object ID of the viewing user who issued the addition instruction via the terminal communication I / F 21. Next, the display processing unit 274 of the terminal device 2 refers to the object table T2 and extracts a still image or a moving image (image information) of the additional object associated with the received object ID from the terminal storage device 22. Next, the display processing unit 274 generates a new delivery image 612 that includes the extracted still image or moving image in the delivery images 602, 612 including the character objects 601, 611 of the broadcasting user. Note that response information including the name of the viewing user who issued the addition instruction and the name of the additional object associated with the received object ID may be newly displayed in the new delivery images 602, 612. With the above, the addition process ends.

[0190] 8(b) is a diagram showing another example of a distribution screen 710 displayed on the display device 23 of the terminal device 2. Like the distribution screen 610, the distribution screen 710 includes a distribution image 612 including a character object 611, a rating button 613, an add button 614, a comment button 615, a response information display area 616, and an end viewing button 617.

[0191] Unlike the distribution screen 610, the distribution screen 710 includes an additional object 711 and an additional object 712. The additional object 711 and the additional object 712 are both still images or moving images showing the additional objects. Hereinafter, an example of the display process of the additional object 712 will be described, taking as an example a case where the viewing user who has issued an instruction to add the additional object 712 is user A.

[0192] When an addition candidate object (corresponding to the additional object 712) is selected in response to user A's operation of the input device 24, the input device 24 of user A's terminal device 2 inputs an addition instruction to the terminal processing device 27 together with the object ID corresponding to the selected addition candidate object. When the display processing unit 274 of user A's terminal device 2 acquires the input addition instruction and object ID, it refers to the object table T2 and extracts a still image or video image (image information) of the additional object associated with the acquired object ID from the terminal storage device 22. Then, the display processing unit 274 of user A's terminal device 2 generates a new delivery image 612 including the delivery user's character object 611 and the additional object 712 based on the extracted still image or video image.

[0193] The transmitting unit 272 of the terminal device 2 of the user A transmits information indicating the addition instruction related to the additional object 712, the acquired object ID, and the viewing user ID of user A stored in the terminal storage device 22 to the terminal devices 2 of the viewing users (excluding the viewing user who issued the addition instruction) and the broadcasting users via the server device 3. When the information indicating the addition instruction related to the additional object 712 and the viewing user ID and object ID of user A are received by the receiving unit 273 of the terminal devices 2 of the viewing users (excluding user A) and the broadcasting users, the display processing unit 274 of the terminal devices 2 of the viewing users (excluding user A) and the broadcasting users refers to the object table T2 and extracts a still image or a moving image (image information) of the additional object associated with the received object ID from the terminal storage device 22. Then, the display processing unit 274 of the terminal device 2 generates a new delivery image 612 based on the extracted still image or moving image in the delivery image 612 including the character object 611 of the broadcasting user. With the above, the display processing of the additional object 712 ends.

[0194] When a predetermined change condition is satisfied, the shape, size, arrangement, color, brightness, and / or design of the additional object added to the distribution screen may change during the display of the distribution screen. The predetermined change condition may be a condition related to information indicating an instruction to add an additional object. For example, the predetermined change condition may be that a condition related to the timing of an instruction to add by any one of all viewing users viewing the same distribution screen is satisfied, and / or that at least two viewing users of all viewing users viewing the same distribution screen have issued an instruction to add a plurality of specific types of additional objects different from each other. For example, the condition related to the timing of an instruction to add is satisfied when the difference in the timing of the instructions to add by a plurality of viewing users is within a predetermined period. In response to an instruction to add by a viewing user, the server device 3 grants the viewing user a predetermined number of points. When the predetermined change condition is satisfied, the viewing user who issued an instruction to add that satisfies the predetermined change condition may be granted more points than the predetermined number of points. The predetermined change condition may be that a condition related to the timing of an additional instruction by any one of a plurality of viewing users having a predetermined relationship is satisfied, and / or that a plurality of viewing users having a predetermined relationship have issued additional instructions for a plurality of specific types of additional objects that are different from one another. In this case, even if a condition related to the timing of an additional instruction by a viewing user who does not have a predetermined relationship is satisfied, the predetermined change condition is not satisfied.

[0195] A predetermined relationship among a plurality of viewing users means, for example, that one of the plurality of viewing users is a mutual follower of another of the plurality of viewing users. Note that when one user follows another user and the other user also follows the one user (the one user and the other user mutually follow each other), the one user and the other user become mutual followers. Also, a predetermined relationship among a plurality of viewing users may mean that one of the plurality of viewing users is in a mutual follow relationship with a predetermined viewing user who mutually follows another of the plurality of viewing users.

[0196] Furthermore, the predetermined relationship among the multiple viewing users may be that the multiple viewing users are in a friend relationship. Furthermore, the predetermined relationship among the multiple viewing users may be that a terminal device of one of the multiple viewing users stores specific information (such as a phone number, an email address, or a predetermined ID (identification)) of another of the multiple viewing users. Furthermore, the predetermined relationship among the multiple viewing users may be a relationship in which the number of times that the multiple viewing users have exchanged information using the same information exchange service is equal to or greater than a predetermined number. Furthermore, the predetermined relationship among the multiple viewing users may be a relationship in which the number of times that each of the multiple viewing users has participated in the same and / or different information exchange services is equal to or greater than a predetermined number.

[0197] Furthermore, the predetermined relationship among multiple viewing users may be a relationship regarding operations of other viewing users among the multiple viewing users when one of the multiple viewing users distributes a distribution image as a distribution user. Here, the relationship regarding operations of the other viewing users is a relationship in which the number of times the other viewing users have viewed the distribution image distributed by the one viewing user is equal to or greater than a predetermined number of times. Furthermore, the relationship regarding operations of the other viewing users may be a relationship in which the other viewing users have sent comment information about the distribution image distributed by the one viewing user equal to or greater than a predetermined number of times.

[0198] The predetermined relationship between the multiple viewing users may be a relationship regarding a predetermined event in which each of the multiple viewing users participates. Here, the relationship regarding the predetermined event is a relationship in which each of the multiple viewing users is currently participating in the same event. The relationship regarding the predetermined event may be a relationship in which each of the multiple viewing users has participated in the same event in the past. The relationship regarding the predetermined event may be a relationship in which another viewing user among the multiple viewing users has participated in an event in which one or more viewing users among the multiple viewing users participate in the past.

[0199] When a change button (not shown) of the delivery image 602 is selected by the broadcasting user operating the input device 24, a selection screen 800 is displayed. For example, the change button is included in the delivery screen 600. The change button may be included in a predetermined screen such as a home screen displayed on the display device 23. FIG. 9(a) is a diagram showing an example of the selection screen 800 displayed on the display device 23 of the broadcasting user's terminal device 2 (broadcasting terminal device). The selection screen 800 shown in FIG. 9(a) is displayed superimposed on the delivery screen 600. The selection screen 800 may be displayed after the display of the delivery screen 600 is ended.

[0200] The selection screen 800 includes a plurality of placement candidate objects 801 indicating a plurality of placement locations, and a candidate attachment object 802 which is a candidate for an attachment object to be newly attached to the character object 601. When one of the plurality of placement candidate objects 801 is selected by the delivery user operating the input device 24, the attachment candidate object 802 corresponding to the object ID of the object owned by the delivery user associated with the placement candidate corresponding to the selected placement candidate object 801 is displayed on the selection screen 800. When one of the plurality of attachment candidate objects 802 is selected by the delivery user operating the input device 24, the input device 24 inputs a change instruction to the terminal processing device 27, and the display processing unit 274 executes the above-mentioned change process. Then, the display processing unit 274 displays a delivery screen 810 (FIG. 9(b)) including a delivery image 812 including a character object 811 equipped with a new attachment object.

[0201] Fig. 9(b) is a diagram showing another example of a broadcast screen 810 displayed on the display device 23 of the broadcast user's terminal device 2 (broadcast terminal device). The broadcast screen 810 includes at least a broadcast image 812 including a broadcast user's character object 811 wearing a new attachment object. In the example shown in Fig. 9(b), a character object 811 is displayed in which the clothes worn by the broadcast user's character object 601 have been changed.

[0202] Information regarding the change instruction input by a broadcasting user who has started a video broadcasting service is transmitted via server device 3 to the terminal device 2 (viewing terminal device) of the viewing user who is viewing the broadcast image of that broadcasting user. As a result, a change process according to the change instruction input by the broadcasting user is also executed in the terminal device 2 (viewing terminal device) of the viewing user. As a result, a broadcast image including the broadcasting user's character object with the new attachment object attached is displayed on the terminal device 2 (viewing terminal device) of the viewing user.

[0203] 8 and 9, at least a portion of the distribution image including the broadcaster's character object is changed in response to an additional instruction from the viewing user, and at least a portion of the broadcaster's character object is changed in response to a change instruction from the broadcaster. This makes it possible to prevent the viewing user from getting bored during the distribution event and to increase the user's motivation to continue using the video distribution service.

[0204] In addition, in the conventional information processing system, after starting a distribution event to a viewing user via a character object, the distribution user had to interrupt the distribution event in order to change the distribution image including the character object. Then, after the distribution event was interrupted, it was necessary to activate a function for changing the appearance of the character object and / or the image of the background in the distribution image including the character object. Then, the distribution user had to carry out a cumbersome procedure of using the change function to change the character object and / or the distribution image, and then resuming the distribution event. In contrast, the information processing system 1 of the present embodiment allows the distribution user to change the character object without interrupting the distribution event and resetting the character object of the distribution user. As a result, in the information processing system 1 of the present embodiment, the user interface for changing the character object of the distribution user during the distribution event is not complicated, so that it is possible to improve usability compared to the conventional information processing system. In addition, in the information processing system 1 of the present embodiment, it is possible to change the user's character object without interrupting the distribution event, so that it is possible to reduce the number of communications and the amount of communication between the server device 3 and the terminal device 2. In this way, in the information processing system 1 of the present embodiment, it is possible to improve the communication between the server device 3 and the terminal device 2.

[0205] FIG. 10(a) is a diagram showing an example of a group setting screen 900 displayed on the display device 23 of a user's terminal device 2. The group setting screen 900 is displayed in response to a user's operation on a predetermined screen, such as a home screen, displayed on the display device 23. A user who can operate a terminal device 2 capable of displaying the group setting screen 900 is a broadcasting user or a viewing user. A user who can operate a terminal device 2 capable of displaying the group setting screen 900 is not limited to a broadcasting user or a viewing user, and may be any user who is capable of receiving services provided by the server device 3. A new information exchange group is set in response to a user's operation on the group setting screen 900.

[0206] As shown in FIG. 10(a), the group setting screen 900 includes a candidate display area 901, a selection object 902, and a start button 903. The candidate display area 901 displays information indicating one or more candidate users who can belong to the information exchange group. For example, the one or more candidate users who can belong to the information exchange group are users who are mutual followers of the user who operates the terminal device 2 that displays the group setting screen 900. Alternatively, the one or more candidate users who can belong to the information exchange group may be users who have a friend relationship with the user who operates the terminal device 2 that displays the group setting screen 900. The one or more candidate users who can belong to the information exchange group are not limited to the above, and may be any users who have a predetermined relationship with the user who operates the terminal device 2 that displays the group setting screen 900. In the example shown in FIG. 10(a), the display processing unit 274 of the user's terminal device 2 displays thumbnail images and names of the character objects of the multiple users who can belong to the information exchange group in the candidate display area 901 as information indicating the candidate users.

[0207] The selection object 902 is an operation object for selecting each candidate user displayed in the candidate display area 901. In the example shown in FIG. 10(a), the display processing unit 274 of the user's terminal device 2 displays a check box object corresponding to each candidate user as the selection object 902. For example, when the selection object 902 is selected by the user's operation of the input device 24, the selection object 902 is displayed including a "check" mark. In this case, the candidate user corresponding to the selection object 902 displayed including the "check" mark is selected. Also, when the selection object 902 displayed including the "check" mark is selected by the user's operation of the input device 24, the selection object 902 is displayed not including a "check" mark. In this case, the candidate user corresponding to the selection object 902 displayed not including a "check" mark is not selected. Each of the selection objects 902 is associated with the user ID of the corresponding candidate user.

[0208] The start button 903 is a button object or the like for the user to create a new information exchange group and start exchanging information with other users who belong to the newly created information exchange group. When the start button 903 is selected by the user operating the input device 24, a new information exchange group is created to which the user operating the terminal device 2 and the candidate user selected by the selection object 902 belong as belonging users. For example, the display processing unit 274 of the user's terminal device 2 identifies the user ID of the user stored in the terminal storage device 22 and the user ID of the candidate user associated with the selection object 902 selected by the selection object 902. Then, the display processing unit 274 associates the user ID of the identified user and the user ID of the candidate user with the newly created group ID as belonging users and stores them in the group table T3. At this time, the display processing unit 274 may associate the name of the automatically created information exchange group with the newly created group ID, or may associate the name of the information exchange group input by the user with the newly created group ID. When a new information exchange group is created in response to selection of the start button 903, the display processing unit 274 displays an information exchange screen 1000 for users belonging to the new information exchange group.

[0209] FIG. 10(b) is a diagram showing an example of an information exchange start screen 910 displayed on the display device 23 of the user's terminal device 2. The information exchange start screen 910 is displayed in response to a user's operation on a predetermined screen such as a home screen displayed on the display device 23. A user who can operate a terminal device 2 capable of displaying the information exchange start screen 910 is a broadcasting user or a viewing user. A user who can operate a terminal device 2 capable of displaying the information exchange start screen 910 is not limited to a broadcasting user or a viewing user, and may be any user who is capable of receiving services provided by the server device 3. In response to a user's operation on the information exchange start screen 910, information exchange by the selected information exchange group is started.

[0210] 10(b), the information exchange start screen 910 includes a candidate group area 911. The candidate group area 911 displays information indicating an information exchange group to which a user who operates the terminal device 2 displaying the information exchange start screen 910 belongs. The information indicating the information exchange group can be selected by the user, and the information indicating the information exchange group is associated with the group ID of the corresponding information exchange group.

[0211] When the user operates the input device 24 to select information indicating any one of the information exchange groups included in the information exchange start screen 910, the group ID associated with the information indicating the selected information exchange group is specified. The display processing unit 274 displays an information exchange screen 1000 by each user identified by a user ID associated with the specified group ID (each user belonging to the information exchange group indicated by the specified group ID).

[0212] FIG. 11(a) is a diagram showing an example of an information exchange screen 1000 displayed on the display device 23 of the user's terminal device 2. On the information exchange screen 1000, information exchange is performed by a plurality of users belonging to an information exchange group newly set by the user selecting a start button 903 included in the group setting screen 900. Alternatively, on the information exchange screen 1000, information exchange is performed by a plurality of users belonging to an information exchange group selected by the user on the information exchange start screen 910. Hereinafter, among a plurality of users belonging to the information exchange group performing information exchange, users other than the user operating the terminal device 2 displaying the information exchange screen 1000 may be referred to as other users. The information exchange screen 1000 (and each information exchange screen described later) may be realized by a so-called chat screen or the like that realizes text chat or the like.

[0213] The information exchange screen 1000 includes other user input information 1001, other user information 1002, a text input object 1003, a candidate display button 1004, and user input information 1005. The other user input information 1001 is input information input by another user. The other user information 1002 is information indicating another user (such as an icon image of a character object) that is associated with each other user input information 1001 and displayed. The text input object 1003 is an operation object for a user to input text information as input information. The candidate display button 1004 is an operation object for displaying candidate information 1101 corresponding to a character-related image so that the user can select it. The user input information 1005 is input information input by a user.

[0214] Each time the input information of another user transmitted from the terminal device 2 of the other user is received via the server device 3, the display processing unit 274 displays other user input information 1001 indicating the received input information of the other user in the other user display area of ​​the information exchange screen 1000. For example, the other user display area is an area near the left side of the information exchange screen 1000, and the other user input information 1001 is displayed in the other user display area in chronological order from the oldest to the top to the bottom of the screen. Each time the user input information is input by the user's input operation on the text input object 1003, the display processing unit 274 displays user input information 1005 indicating the input user input information in the user display area of ​​the information exchange screen 1000. For example, the user display area is an area near the right side of the information exchange screen 1000, and the user input information 1005 is displayed in chronological order from the oldest to the top to the bottom of the screen in the user display area.

[0215] When the information exchange screen 1000 related to the newly set information exchange group is displayed, the transmitting unit 272 of the user's terminal device 2 transmits information related to the new information exchange group to the server device 3 via the terminal communication I / F 21. The information related to the new information exchange group includes the group ID of the new information exchange group, the name of the new information exchange group, and the user IDs of the users belonging to the new information exchange group.

[0216] The server receiving unit 331 of the server device 3 receives information related to the information exchange group via the server communication I / F 31. The server receiving unit 331 stores the received information related to the new information exchange group in the server storage device 32. For example, the server receiving unit 331 stores in the server storage device 32 the name of the new information exchange group and the user IDs of the users belonging to the new information exchange group in association with the group ID of the new information exchange group.

[0217] The server transmitting unit 332 of the server device 3 transmits information regarding the new information exchange group via the server communication I / F 31 to terminal devices 2 of other users who belong to the new information exchange group and are identified by the user IDs of users other than the user who transmitted the information regarding the new information exchange group.

[0218] When the receiving unit 273 of the other user's terminal device 2 receives information about the new information exchange group transmitted from the user's terminal device 2 via the server device 3, it stores the name of the new information exchange group and the user IDs of the users belonging to the new information exchange group in the group table T3 in association with the group ID of the new information exchange group. Note that when the display processing unit 274 of the other user's terminal device 2 receives information about the new information exchange group, it may automatically display the information exchange screen 1000 related to the new information exchange group.

[0219] On the information exchange screen 1000, when new text information is input as input information to the text input object 1003 by a user operation, the display processing unit 274 displays the input new text information in the user display area as user input information 1005. On the information exchange screen 1000, when image information, video information, or the like is input as input information by a user operation, the display processing unit 274 displays the input image information, video information, or the like in the user display area as user input information 1005. Information input as input information is not limited to text information, image information, and video information, and may be information in a format that can be used by a predetermined application program.

[0220] When input information input by another user's operation on the information exchange screen 1000 displayed by the terminal device 2 of the other user is received via the server device 3 together with the user ID of the other user, the received input information is displayed in the other user display area as other user input information. In this case, an image or the like related to the received user ID of the other user is displayed as other user information 1002.

[0221] The transmission unit 272 of the user's terminal device 2 transmits the input new text information and the user's user ID to the terminal device 2 of the other user via the server device 3. As a result, the input new text information is also displayed as other user input information 1001 on the information exchange screen 1000 displayed on the terminal device 2 of the other user who is a user belonging to a group other than the user who input the text information.

[0222] When the candidate display button 1004 is selected by the user's operation of the input device 24 on the information exchange screen 1000, the input device 24 sends a display instruction of candidate information for selecting a character-related image to the terminal processing device 27. Next, the display processing unit 274 determines whether or not a character-related image including a character object of the user has been generated in the past and stored in the terminal storage device 22. If a character-related image including a character object of the user has not been stored, the display processing unit 274 displays a generation instruction object 1006 for instructing the generation of a character-related image. Also, if a character-related image including a character object of the user has been stored, for example, the display processing unit 274 displays the information exchange screen 1100 including the four types of stored candidate information 1101.

[0223] 11(b) is a diagram showing an example of an information exchange screen 1010 displayed on the display device 23 of the user's terminal device 2. The information exchange screen 1010 is a screen that is displayed when a candidate display button 1004 included in the information exchange screen 1000 is selected by the user and it is determined that a character-related image including the user's character object is not stored in the terminal storage device 22.

[0224] Similar to the information exchange screen 1000, the information exchange screen 1010 includes other user input information 1001, other user information 1002, a text input object 1003, a candidate display button 1004, and user input information 1005. Furthermore, the information exchange screen 1010 includes a generation instruction object 1006. In the example shown in Fig. 11(b), the information exchange screen 1010 displays information prompting the generation of a character-related image together with the generation instruction object 1006.

[0225] When a generation instruction object 1006 is selected on the information exchange screen 1010 by a user operating the input device 24, the input device 24 sends an instruction to generate a character-related image to the terminal processing device 27. When the generation unit 271 acquires an instruction to generate a character-related image from the input device 24, it executes a generation process of the character-related image. An example of the generation process of the character-related image will be described below.

[0226] First, the generation unit 271 extracts the model ID of the character object associated with the user ID stored in the terminal storage device 22 and the object ID of the used object from the user table T1. Next, the generation unit 271 reads out the model data associated with the extracted model ID and the image information and arrangement location associated with the extracted object ID of the used object from the terminal storage device 22. Next, the generation unit 271 reads out the motion data for the character-related image and the object data for the character-related image from the terminal storage device 22. Next, the generation unit 271 generates a character-related image including a character object with a predetermined action and expression, and object data such as a background and characters, based on the read out model data, the image information and arrangement location of the used object, the motion data for the character-related image, and the object data for the character-related image. Then, the generation unit 271 associates the generated character-related image with an associated image ID for uniquely identifying the character-related image, and stores the generated character-related image in the terminal storage device 22. Furthermore, the generation unit 271 generates candidate information indicating the generated character-related image, associates the generated candidate information with a related image ID for identifying the character-related image, and stores the associated information in the terminal storage device 22. Furthermore, the generation unit 271 associates the object ID of the object used when generating the character-related image with the related image ID and stores the associated information in the terminal storage device 22, and the character-related image generation process ends. Note that some or all of the generated character-related images may be set so as not to be usable in the information exchange service. In this case, a usable group consisting of character-related images usable in the information exchange service may be set. Then, when a new character-related image usable in the information exchange service is generated, the new character-related image is set so as to be included in the usable group.

[0227] An example of a character-related image transmission process will be described below. First, the transmission unit 272 of the terminal device 2 that generated the character-related image transmits the character-related image, candidate information, and object ID of the used object associated with the related image ID stored in the terminal storage device 22 to the server device 3 via the terminal communication I / F 21. The user ID stored in the terminal storage device 22 is transmitted to the server device 3 together with the character-related image, candidate information, and object ID of the used object associated with the related image ID. When the server receiving unit 331 of the server device 3 receives the character-related image, candidate information, and object ID of the used object associated with the related image ID and the user ID, it stores the related image ID in association with the user ID, and stores the character-related image, candidate information, and object ID of the used object in association with the related image ID in the server storage device 32. Then, the server transmission unit 332 transmits the received related image ID associated with the user ID of the user, the character-related image associated with the related image ID, the candidate information, and the object ID of the used object to the terminal device 2 of the other user belonging to the information exchange group to which the user of the terminal device 2 that transmitted the character-related image, etc. belongs, via the server communication I / F 31. The terminal device 2 of the other user stores the transmitted related image ID associated with the user ID, the character-related image associated with the related image ID, the candidate information, and the object ID of the used object in the terminal storage device 22. With the above, the transmission process of the character-related image is completed. Note that the transmission unit 272 of the terminal device 2 that generated the character-related image may not transmit the candidate information associated with the related image ID and the object ID of the used object stored in the terminal storage device 22 to the server device 3. Also, the candidate information and the object ID of the used object associated with the user ID of one user belonging to the same information exchange group may not be transmitted to the terminal device 2 of the other user belonging to the information exchange group.Furthermore, the timing at which a character-related image associated with the user ID of a user belonging to the same information exchange group is transmitted to the terminal device 2 of another user belonging to the same information exchange group may be the timing at which the user selects the character-related image as input information. The character-related image transmission process may be performed every time the character-related image generation process and / or additional generation process is performed.

[0228] The display processing unit 274 reads out the candidate information generated by the generation unit 271 from the terminal storage device 22 and displays it. Fig. 12(a) is a diagram showing an example of an information exchange screen 1100 displayed on the display device 23 of the user's terminal device 2. The information exchange screen 1100 is a screen that is displayed when a generation instruction object 1006 included in the information exchange screen 1010 is selected by the user.

[0229] The information exchange screen 1100 includes other user input information 1001, other user information 1002, a text input object 1003, a candidate display button 1004, and user input information 1005, similarly to the information exchange screen 1000. Furthermore, the information exchange screen 1010 includes four types of generated candidate information 1101 and an additional generation button 1102. The type of candidate information 1101 included in the information exchange screen 1100 is not limited to four types, and may be three or less, or five or more types. When a part of the multiple types of candidate information 1101 cannot be included in the display area of ​​the information exchange screen 1100, the display processing unit 274 of the terminal device 2 may display the candidate information 1101 in a scrollable manner. In this case, the candidate information 1101 that is not currently included in the display area of ​​the information exchange screen 1100 is scrolled and displayed in the display area of ​​the information exchange screen 1100 in response to a swipe operation by the user (for example, an operation in which a finger is moved from right to left while touching the screen).

[0230] When one piece of candidate information 1101 is selected from the four types of candidate information 1101 in response to the user's operation of the input device 24 (FIG. 12(b)), the input device 24 sends the related image ID associated with the selected candidate information to the terminal processing device 27. Next, the display processing unit 274 extracts a character-related image identified by the related image ID received from the input device 24 from the terminal storage device 22, and displays the extracted character-related image in the user display area of ​​the information exchange screen 1200 as user input information 1201 (FIG. 13). The character-related image displayed as the user input information 1201 is a two-dimensional still image or a three-dimensional still image in which a three-dimensional virtual space including a character object is depicted. When the character-related image displayed as the user input information 1201 is generated based on a plurality of continuous motion data, the character-related image may be a two-dimensional moving image including a character object that moves for a predetermined time (e.g., three seconds). In this case, the character-related image may be a three-dimensional moving image in which a three-dimensional virtual space including a character object that moves for a predetermined time (e.g., three seconds) is depicted. As shown in FIG. 13, the display size of the character-related image displayed as the user input information 1201 on the display screen of the display device 23 is smaller than the display size of the delivered image 602 on the display screen of the display device 23. In this way, the drawing processing load of drawing the character-related image on the display screen of the display device 23 is larger than the drawing processing load of drawing the delivered image 602 on the display screen of the display device 23. In addition, since the character moving image data including the motion data for displaying the delivered image 602 is continuously transmitted, the transmission amount of the character moving image data is larger than the transmission amount of the related image ID for displaying the character-related image. In this way, by using the character-related data in various services, it is possible to display the character object of a user on the terminal device 2 of another user while reducing the processing load of the terminal device 2 and / or the server device 3 and the transmission load between the terminal device 2 and the server device 3, as compared with the case where the delivered image is used.

[0231] Along with displaying the user input information 1201 by the display processing unit 274, the transmission unit 272 transmits the related image ID received from the input device 24 and the user ID stored in the terminal storage device 22 to the server device 3 via the terminal communication I / F 21. The server transmission unit 332 of the server device 3 extracts, from the server storage device 32, the user IDs of other users associated with the same information exchange group as the user ID received by the server reception unit 331. In addition, the server transmission unit 332 transmits, via the server communication I / F 31, the related image ID and user ID received by the server reception unit 331 to the terminal device 2 of the other user identified by the extracted user ID.

[0232] Then, the receiving unit 273 of the terminal device 2 of the other user receives the related image ID and the user ID transmitted from the server device 3 via the terminal communication I / F 21. Next, the display processing unit 274 of the terminal device 2 of the other user extracts the character-related image identified by the related image ID received by the receiving unit 273 from the terminal storage device 22, and displays the extracted character-related image in the other user display area of ​​the distribution screen as other user input information. In this way, the same image as the character-related image used by the user is also displayed in the terminal device 2 of the other user associated with the information exchange group to which the user belongs. Note that, if the display processing unit 274 of the terminal device 2 of the other user does not store the character-related image identified by the received related image ID in the terminal storage device 22, the transmitting unit 272 of the terminal device 2 of the other user transmits an image transmission request to the server device 3. The image transmission request is transmitted together with the received related image ID and / or user ID, and the user ID of the other user. When the server receiving unit 331 receives an image transmission request, the server transmitting unit 332 of the server device 3 extracts from the server storage device 32 a character-related image associated with the related image ID received together with the image transmission request. Next, the server transmitting unit 332 transmits the extracted character-related image together with the received related image ID and user ID to the terminal device 2 of the other user. The receiving unit 273 of the terminal device 2 of the other user receives the related image ID, character-related image, and user ID transmitted from the server device 3 via the terminal communication I / F 21, and stores them in the terminal storage device 22 in association with each other. Then, the display processing unit 274 of the terminal device 2 of the other user displays the stored character-related image in the other user display area of ​​the distribution screen as other user input information.

[0233] Returning to FIG. 12(a), when the additional generation button 1102 is selected in response to the user's operation of the input device 24, the display processing unit 274 and the generation unit 271 of the user's terminal device 2 execute the additional generation process. The additional generation process is, for example, a process of generating a new character-related image including a character object indicating another user exchanging information with the user and / or a character object indicating another user who has exchanged information with the user, using an information exchange service. The other user exchanging information with the user is, for example, another user of the terminal device 2 displaying an information exchange screen for information exchange related to the information exchange screen currently displayed on the user's terminal device 2. Alternatively, the other user exchanging information with the user may be another user of the terminal device 2 executing a control program for displaying an information exchange screen for information exchange related to the information exchange screen currently displayed on the user's terminal device 2.

[0234] 14(a) is a diagram showing an example of an other object selection screen 1300 displayed on the display device 23 of the user's terminal device 2. The other object selection screen 1300 is a screen for selecting object data included in a new character-related image generated by the additional generation process.

[0235] The object data included in the character-related image includes, for example, object data indicating a background and object data indicating characters. The object data included in the character-related image may also include object data indicating a design. For example, the object data indicating a design is object data indicating a pattern such as a frame. The object data included in the character-related image may also include object data indicating an effect. For example, the object data indicating an effect is object data indicating decoration (glitter, etc.), such as object data in the shape of "stars" arranged around the character object, or object data in the shape of "droplets" indicating tears or sweat.

[0236] The other object selection screen 1300 is a screen that is displayed when the additional generation process is started in response to a user selecting the additional generation button 1102 included in the information exchange screen 1100. For example, the display processing unit 274 of the terminal device displays the other object selection screen 1300 in response to the user selecting the additional generation button 1102 by operating the input device 24.

[0237] The other object selection screen 1300 includes a candidate display area 1301 , a selection object 1302 , a user selection instruction button 1303 , and an end button 1304 .

[0238] In the candidate display area 1301, candidate information indicating object data for character-related images stored in the terminal storage device 22 is displayed. The candidate information is object data for character-related images. In the example shown in FIG. 14(a), object data indicating the character "OK" is displayed in the candidate display area 1301 as the candidate information. In the example shown in FIG. 14(a), object data indicating "Manpu" (a mark, symbol, or code used in Manga), which is a type of effect, is displayed in the candidate display area 1301 as the candidate information. Note that character information explaining the object data for character-related images may be displayed in the candidate display area 1301 as the candidate information. For example, when the object data is object data in the shape of a "star", the character "Star" may be displayed in the candidate display area 1301.

[0239] The types of candidate information displayed in the candidate display area 1301 are not limited to two, and may be one type or three or more types. When some of the multiple types of candidate information cannot be included in the display area of ​​the candidate display area 1301, the display processing unit 274 of the terminal device 2 may display the multiple types of candidate information in a scrollable manner. In this case, in response to a swipe operation by the user, the candidate information that is not currently included in the display area of ​​the candidate display area 1301 of the other object selection screen 1300 may be scrolled and displayed in the display area of ​​the candidate display area 1301.

[0240] The selection object 1302 is an operation object for selecting one piece of candidate information from the candidate information displayed in the candidate display area 1301. In the example shown in Fig. 14(a), the display processing unit 274 of the user's terminal device 2 displays a check box object corresponding to each piece of candidate information as the selection object 1302. Each of the selection objects 1302 is associated with an object ID of the object data for the corresponding character-related image.

[0241] For example, when a selection object 1302 is selected by a user operating the input device 24, the selected selection object 1302 is displayed with a "check" mark. In this case, the object data for the character-related image corresponding to the selection object 1302 displayed with a "check" mark is in a selected state. When a selection object 1302 displayed with a "check" mark is selected by a user operating the input device 24, the selected selection object 1302 is displayed without a "check" mark. In this case, the object data for the character-related image corresponding to the selection object 1302 displayed without a "check" mark is in a non-selected state. In this way, the user can select one or more object data for character-related images by selecting one or more selection objects 1302.

[0242] The user selection instruction button 1303 is a button object or the like for setting object data corresponding to the selected selection object 1302 as object data included in a new character-related image and instructing to display a user selection screen 1310. In response to the selection of the user selection instruction button 1303 by the user operating the input device 24, the input device 24 inputs a user selection instruction to the terminal processing device 27. When the user selection instruction is input, the display processing unit 274 of the terminal device 2 temporarily stores the object data corresponding to the selected selection object 1302 in the terminal storage device 22 as object data included in a new character-related image. The display processing unit 274 of the terminal device 2 ends the display of the other object selection screen 1300 and displays the user selection screen 1310 on the display device 23.

[0243] The end button 1304 is an operation object for ending the display of the other object selection screen 1300. When the end button 1304 is selected by the user operating the input device 24, the display processing unit 274 of the terminal device 2 ends the display of the other object selection screen 1300 and displays the information exchange screen 1100 on the display device 23.

[0244] In addition, the other object selection screen 1300 may not be displayed in response to the user selecting the additional generation button 1102 included in the information exchange screen 1100. For example, in response to the user selecting the additional generation button 1102, a user selection screen 1310 described later may be displayed (without displaying the other object selection screen 1300). In addition, in response to the user selecting the additional generation button 1102, an additional generation process described later may be executed (without displaying both the other object selection screen 1300 and the user selection screen 1310). When the other object selection screen 1300 is not displayed, a character-related image that does not use object data for a character-related image is generated in the additional generation process described later. Alternatively, when the other object selection screen 1300 is not displayed, a character-related image that uses predetermined object data for a character-related image may be generated. In addition, when the user selection screen 1310 is not displayed, in the additional generation process described later, another user that satisfies a selection condition may be automatically selected, and a character-related image that uses a character object of the other user that satisfies the selection condition may be generated. In addition, the details of the selection condition will be described later.

[0245] 14(b) is a diagram showing an example of a user selection screen 1310 displayed on the display device 23 of the user's terminal device 2. The user selection screen 1310 is a screen for selecting another user indicated by a character object included in a new character-related image generated by the additional generation process. The user selection screen 1310 includes a candidate display area 1311, a selection object 1312, an additional generation instruction button 1313, and an end button 1314.

[0246] The candidate display area 1311 displays information indicating other users of the terminal device 2 currently displaying the information exchange screen 1000 among other users who belong to the information exchange group to which the user belongs. The candidate display area 1311 may display information indicating all other users who belong to the information exchange group to which the user belongs. The information exchange group to which the user belongs is the information exchange group related to the information exchange screen 1100 selected by the user to start the additional generation process. In this way, when the additional generation button 1102 is selected by a user on the information exchange screen 1100 related to an information exchange group displayed on the terminal device 2 of a user who belongs to an information exchange group, the terminal device 2 of the user can generate a new character-related image including a character object indicating another user who belongs to the information exchange group. In the example shown in FIG. 14(b), the display processing unit 274 of the user's terminal device 2 displays thumbnail images and names of the character objects of the other users in the candidate display area 1311 as information indicating the other users.

[0247] The selection object 1312 is an operation object for selecting information indicating one or more other users from the information indicating the other users displayed in the candidate display area 1311. In the example shown in Fig. 14(b), the display processing unit 274 of the user's terminal device 2 displays check box objects corresponding to the information indicating each other user as the selection object 1312. Each of the selection objects 1312 is associated with the user ID of the corresponding other user.

[0248] For example, when a selection object 1312 is selected by a user operating the input device 24, the selected selection object 1312 is displayed with a "check" mark. In this case, the other user corresponding to the selection object 1312 displayed with a "check" mark is in a selected state. Also, when a selection object 1312 displayed with a "check" mark is selected by a user operating the input device 24, the selected selection object 1312 is displayed without a "check" mark. In this case, the other user corresponding to the selection object 1312 displayed without a "check" mark is in a non-selected state. In this way, the user can select one or more other users by selecting one or more selection objects 1312.

[0249] The additional generation instruction button 1313 is an example of an operation object for inputting an additional generation instruction to generate a new character-related image. For example, when the additional generation instruction button 1313 is selected in response to a user's operation of the input device 24, the input device 24 inputs an additional generation instruction to the generation unit 271, and the display processing unit 274 ends the display of the user selection screen 1310.

[0250] When the generation unit 271 acquires an additional generation instruction from the input device 24, it acquires from the terminal storage device 22 the object data temporarily stored in response to the user's selection of the user selection instruction button 1303 on the other object selection screen 1300. Next, the generation unit 271 acquires the user ID of the other user corresponding to the selected object 1312 selected by the user on the user selection screen 1310.

[0251] Next, the generating unit 271 executes an additional generating process for generating a new character-related image based on the acquired object data (for the character-related image) and the user ID of the other user. Hereinafter, an example of the additional generating process for generating a new character-related image will be described.

[0252] First, the generation unit 271 extracts the model ID of the user's character object and the object ID of the used object associated with the user ID stored in the terminal storage device 22 from the user table T1. Next, the generation unit 271 reads out the model data of the user's character object associated with the extracted model ID, and the image information and the arrangement location associated with the extracted object ID of the used object (of the user's character object) from the terminal storage device 22. Next, the generation unit 271 extracts the model ID of the character object of the other user and the object ID of the used object associated with the acquired user ID of the other user from the user table T1. Next, the generation unit 271 reads out the model data of the character object of the other user associated with the extracted model ID, and the image information and the arrangement location associated with the extracted object ID of the used object (of the other user's character object) from the terminal storage device 22.

[0253] Next, the generation unit 271 reads out image definition information from the terminal storage device 22. The image definition information includes motion data of a character object included in the new character-related image, a position of the character object, a position of object data for the character-related image, and / or a viewpoint position, etc. The terminal storage device 22 stores a plurality of image definition information for each number of character objects included in the new character-related image.

[0254] First, an example of image definition information in the case where the number of character objects included in the new character-related image is "2" (the user's character object and one character object of another user) will be described. This image definition information includes motion data of the user's character object and motion data of one character object of another user. The motion data may be time-series motion data for making the character object perform a predetermined motion during a predetermined motion period (e.g., 3 seconds). This image definition information also stores information indicating the position and front direction of the user's character object, the position and front direction of the other user's character object, and the position and line of sight of the viewpoint. The position of the character object and the position of the line of sight are represented by three-dimensional coordinates in a virtual three-dimensional space. The direction and line of sight of the character object are represented by a unit vector or the like in the virtual three-dimensional space. For example, image definition information is stored that is defined so that when looking in the line of sight from the position of the viewpoint, the front of the user's character object in a first pose (a pose based on stored motion data) is displayed on the left side, and the front of the other user's character object in a second pose (a pose based on stored motion data) is displayed on the right side. When the same image definition information is used, even if any character object of another user is selected as the character object included in the character-related image, the character object in the same second pose is included in the character-related image.

[0255] An example of image definition information in the case where the number of character objects included in the new character-related image is "3" (the user's character object and two other users' character objects) will be described. This image definition information includes motion data of the user's character object and motion data of the two other users' character objects. This image definition information also stores information indicating the position and front direction of the user's character object, the positions and front directions of the two other users' character objects, and the position and line of sight of the viewpoint. For example, when looking in the line of sight from the position of the viewpoint, the image definition information is stored so that the front of the first other user's character object in the third pose (a pose based on stored motion data) is displayed on the left, the front of the user's character object in the fourth pose (a pose based on stored motion data) is displayed in the middle, and the front of the second other user's character object in the fifth pose (a pose based on stored motion data) is displayed on the right. When the same image definition information is used, the character object in the same third pose is included in the character-related image regardless of which other user's character object is selected as the first character object located on the left. Similarly, even if any character object of another user is selected as the second character object located on the right side, the same character object in the fifth pose is included in the character-related image. The number of character objects included in the new character-related image is not limited to "2" or "3" and may be "4" or more.

[0256] In the above description of the embodiment, the "poses" are referred to as the "first", "second", "third", "fourth" and "fifth" poses, but these "first", "second", "third", "fourth" and "fifth" do not indicate an order, nor do they specify that the poses are different from each other. That is, the "first", "second", "third", "fourth" and "fifth" poses may be different from each other, or at least some of them may be the same pose. In addition, in the claims, the above-mentioned "first" pose may be referred to by other names (for example, any of the "second" to "fifth" poses), and similarly, the "second", "third", "fourth" and "fifth" poses may be referred to by other names. That is, the term "first" pose described in the claims is not interpreted as being limited to "the pose of the user's character object in a character-related image including two character objects", and should be understood by interpreting the claims reasonably. Similarly, the terms "second" pose, "third" pose, "fourth" pose, and "fifth" pose should not be interpreted as being limited to the above-described embodiments, but should be understood by reasonably interpreting the claims.

[0257] A plurality of types of image definition information for the user's character object and / or image definition information for the character object of another user may be stored in the terminal storage device 22. The image definition information for the user's character object and the image definition information for the character object of another user may be collectively referred to as image definition information for the character object. For example, different types of motion data may be stored for each parameter of the character object as motion data for the character-related image. The parameters include gender (female type, male type, etc.), size (height is equal to or greater than a predetermined value, less than a predetermined value, etc.), personality (calm, lively, etc.), age (30 years or older, less than 30 years old, etc.), etc. For example, as image definition information for the user's character object, motion data MF for the character-related image applied when the character object is a female type is stored, and motion data MM for the character-related image applied when the character object is a male type is stored. Also, for example, motion data MT for the character-related image applied when the character object is equal to or greater than a predetermined value is stored, and motion data MS for the character-related image applied when the character object is less than a predetermined value is stored. Also, for example, as image definition information for a user's character object, motion data MFS for character-related images that is applied when the character object is female and has a height less than a predetermined value may be stored, and motion data MMT for character-related images that is applied when the character object is male and has a height equal to or greater than a predetermined value may be stored.

[0258] Next, the generating unit 271 generates a new character-related image including a character object of the user with a predetermined action and expression, and object data such as a background and characters, based on each extracted model data, the image information and the arrangement location of each used object, the image definition information, and the acquired object data (for the character-related image). Next, the generating unit 271 associates the generated new character-related image with an associated image ID for uniquely identifying the character-related image, and stores the associated image ID in the terminal storage device 22. The generating unit 271 also generates candidate information indicating the generated new character-related image, associates the associated image ID for identifying the character-related image with the generated candidate information, and stores the associated image ID in the terminal storage device 22. Then, the generating unit 271 associates the object ID of the used object at the time of generating the new character-related image with the associated image ID, and stores the additional generation process of the new character-related image in the terminal storage device 22, and the additional generation process of the new character-related image is completed. Note that a part or all of the generated new character-related image may be set so as not to be usable in the information exchange service. In this case, an available group consisting of character-related images usable in the information exchange service may be set. Then, when a new character-related image that can be used in the information exchange service is generated, the new character-related image is set to be included in the usable group.

[0259] In response to the end of the new additional generation process, the transmission unit 272 of the terminal device 2 that executed the additional generation process executes a transmission process of the new character-related image. As a result, the new character-related image is transmitted to the terminal devices 2 of the other users who belong to the information exchange group to which the user of the terminal device 2 that executed the additional generation process belongs, via the server device 3. In addition, in response to the end of the new additional generation process, the display processing unit 274 of the terminal device 2 that executed the additional generation process executes a display process of the information exchange screen 1400.

[0260] The end button 1314 is an operation object for ending the display of the user selection screen 1310. When the end button 1314 is selected by the user operating the input device 24, the display processing unit 274 of the terminal device 2 ends the display of the user selection screen 1310.

[0261] In addition, the user selection screen 1310 may not be displayed in response to the user selecting instruction button 1303 included in the other object selection screen 1300 being selected by the user. For example, the additional generation process may be executed (without displaying the user selection screen 1310) in response to the user selecting the user selection instruction button 1303. When the user selection screen 1310 is not displayed, in the additional generation process, another user who satisfies the selection condition may be automatically selected, and a character-related image using the character object of the other user who satisfies the selection condition may be generated. Hereinafter, the information exchange group related to the information exchange screen 1100 selected by the user to start the additional generation process may be simply referred to as an "information exchange group", and users other than the user (the user who selected the information exchange screen 1100) who belong to this "information exchange group" may be simply referred to as "other users".

[0262] The selection condition is a condition related to the usage status of the information exchange service of other users. Hereinafter, an example of a process of determining other users who satisfy the selection condition related to the usage status of the information exchange service of other users will be described. First, the terminal device 2 of the other user transmits the input information and user ID of the other user to the terminal device 2 of the user via the server device 3 together with the transmission date and time. The terminal device 2 of the user stores the received input information and user ID of the other user together with the transmission date and time in the terminal storage device 22. The generation unit 271 arranges the input information most recently transmitted by each other user belonging to the information exchange group in the information exchange within the information exchange group in order of the transmission date and time from the current time, and identifies one or more pieces of input information whose transmission date and time are closest to the current time. Then, the generation unit 271 determines the other users who transmitted each of the one or more pieces of input information determined as other users who satisfy the selection condition. For example, if there are four other users (referred to as other user U1, other user U2, other user U3, and other user U4, respectively), and the transmission dates and times of the input information most recently transmitted by other user U1, other user U2, other user U3, and other user U4 are 10:55 on March 24, 2022, 23:03 on March 23, 2022, 11:21 on March 24, 2022, and 07:13 on March 24, 2022, the generation unit 271 determines that the two other users U3 and U1, whose transmission dates and times of the input information are closest to the current time, are other users who satisfy the selection condition. This makes it possible for the generation unit 271 to automatically generate a character-related image including a character object of the other user who most recently transmitted input information (text information, stamp image, etc.) in the information exchange related to the information exchange group. The input information transmitted by other users may include a known "reaction stamp." The number of other users satisfying the selection condition determined by the generation unit 271 is not limited to "2". The number of other users satisfying the selection condition may be "1", or may be "3" or more. The number of other users satisfying the selection condition may be a set number UA of other users, which will be described later. In the above embodiment, the reception date and time at which the user's terminal device 2 receives the input information and user ID of the other user may be used instead of the transmission date and time.

[0263] Hereinafter, another example of the process of determining other users who satisfy the selection condition regarding the usage status of the information exchange service of other users will be described. First, each time the terminal device 2 of the other user starts displaying an information exchange screen to start information exchange within the information exchange group, the terminal device 2 of the other user transmits the display start time of the information exchange screen to the terminal device 2 of the user via the server device 3. The terminal device 2 of the user stores the received display start time of the information exchange screen of the other user in the terminal storage device 22. The generation unit 271 determines other users who operate one or more terminal devices 2 whose display start time of the information exchange screen displayed most recently by the terminal device 2 of each other user belonging to the information exchange group to start information exchange within the information exchange group is closest to the current time. Then, the generation unit 271 determines the determined one or more other users as other users who satisfy the selection condition. For example, if there are four other users (referred to as other user U1, other user U2, other user U3, and other user U4, respectively), and the display dates and times when other user U1, other user U2, other user U3, and other user U4 most recently displayed the information exchange screen related to the information exchange group are 21:10, March 23, 2022, 23:00, March 23, 2022, 22:17, and March 23, 2022, 05:55, the generation unit 271 determines that the two other users U2 and U3, whose information exchange screen display start times are closest to the current time, are other users who satisfy the selection condition. This makes it possible for the generation unit 271 to automatically generate a character-related image including a character object of the other user who most recently started information exchange related to the information exchange group. The number of other users who satisfy the selection condition is not limited to "2". The number of other users that satisfy the selection condition may be “1,” or may be “3” or more. The number of other users that satisfy the selection condition may be a set number UA of other users, which will be described later.

[0264] Hereinafter, another example of the process of determining other users who satisfy the selection condition regarding the usage status of the information exchange service of other users will be described. First, each time the terminal device 2 of the other user logs in to the information exchange service to start information exchange within the information exchange group, the terminal device 2 of the other user transmits the login date and time of the information exchange service to the terminal device 2 of the user via the server device 3. The terminal device 2 of the user stores the received login date and time of the information exchange service of the other user in the terminal storage device 22. The generation unit 271 determines other users who operate one or more terminal devices 2 whose login date and time of the most recent login to the information exchange service by each other user belonging to the information exchange group to start information exchange within the information exchange group is closest to the current time. Then, the generation unit 271 determines the determined one or more other users as other users who satisfy the selection condition. For example, if there are four other users (referred to as other user U1, other user U2, other user U3, and other user U4, respectively), and the login dates and times when other user U1, other user U2, other user U3, and other user U4 most recently logged in to the information exchange service are 21:10, March 23, 2022, 23:00, March 23, 2022, 22:17, and March 23, 2022, 05:55, the generation unit 271 determines that the two other users U2 and U3, who are closest to the current time, are other users who satisfy the selection condition. This makes it possible for the generation unit 271 to automatically generate a character-related image including a character object of the other user who most recently started information exchange related to the information exchange group. The number of other users who satisfy the selection condition is not limited to "2". The number of other users who satisfy the selection condition may be "1" or may be "3" or more. Furthermore, the number of other users who satisfy the selection condition may be a set number UA of other users, which will be described later.

[0265] The other user set number UA indicating the number of other users included in the character-related image may be set in advance or automatically. For example, in the process of determining other users who satisfy the selection condition, the other user set number UA stored in the terminal storage device 22 may be used. In addition, in the process of determining other users who satisfy the selection condition, the other user set number UA randomly set by the terminal processing device 27 may be used. In addition, when the other user set number UA is stored in association with the object data for the character-related image, the other user set number UA associated with the object data (for the character-related image) acquired in response to the user's operation on the other object selection screen 1300 may be automatically used in the process of determining other users who satisfy the selection condition.

[0266] The selection condition may be a condition related to the relationship between the user and other users. An example of a process for determining other users who satisfy the selection condition related to the relationship between the user and other users will be described below. First, the generation unit 271 determines whether or not an information exchange group in which the belonging users are only the user and one other user is stored in the group table T3. When an information exchange group in which the belonging users are only the user and one other user is stored in the group table T3, the generation unit 271 extracts other users who belong to the information exchange group in which the belonging users are only the user and one other user. Then, when the extracted other users are belonging users of an information exchange group (an information exchange group related to the information exchange screen 1100 selected by the user to start the additional generation process), the generation unit 271 determines the extracted other users as other users who satisfy the selection condition. This enables the generation unit 271 to automatically generate a character-related image including a character object of another user who has had one-to-one information exchange with the user. In addition, other users who have had one-on-one information exchange with the user may also be other users who have had one-on-one information exchange with the user through a chat function or the like in an SNS service other than the information exchange service.

[0267] Hereinafter, another example of the determination process of the other user who satisfies the selection condition regarding the relationship between the user and the other user will be described. First, the transmission unit 272 of the user's terminal device 2 transmits information indicating a transmission request of a distribution end user ID indicating another user who has viewed a distribution image distributed by the user as a distribution user together with the user's user ID to the server device 3 via the terminal communication I / F 21. When the server reception unit 331 of the server device 3 receives the information indicating the transmission request and the user's user ID via the server communication I / F 31, it extracts the distribution end user ID associated with the received user ID of the user from the server storage device 32. Then, the server transmission unit 332 of the server device 3 transmits the extracted distribution end user ID to the user's terminal device 2 via the server communication I / F 31. The reception unit 273 of the user's terminal device 2 stores the received distribution end user ID in the terminal storage device 22. Then, when the other user indicated by the stored distribution end user ID is a user belonging to an information exchange group, the generation unit 271 determines that the other user indicated by the distribution end user ID is another user who satisfies the selection condition. This enables the generation unit 271 to automatically generate a character-related image including a character object of another user who has viewed a distribution image distributed by the user as a distribution user.

[0268] Hereinafter, another example of the determination process of other users who satisfy the selection condition regarding the relationship between the user and other users will be described. First, the transmission unit 272 of the user's terminal device 2 transmits information indicating a transmission request of the participation status of other users in a predetermined event in which the user has participated in the past or is currently participating, together with the user ID of the user, to the server device 3 via the terminal communication I / F 21. The server device 3 can provide a predetermined event to multiple users, and each user who wants to participate in the predetermined event can connect to the server device 3 and participate in the predetermined event by performing a predetermined operation on the terminal device 2. The server device 3 stores the user ID of each user who participated in the predetermined event in the server storage device 32 in association with information regarding the predetermined event (event ID, etc.). When the server receiving unit 331 of the server device 3 receives the information indicating the transmission request and the user ID of the user via the server communication I / F 31, it extracts the user ID of the other user associated with the event ID associated with the received user ID of the user from the server storage device 32. Note that the user ID of the other user associated with the extracted event ID may be the user ID of the other user whose participation period overlaps with that of the user in at least a part. The user ID of the other user associated with the extracted event ID may be a user ID of a user whose participation period overlaps with the user's at least partially and who belongs to the same group as the user set in the specified event. Then, the server transmission unit 332 of the server device 3 transmits the extracted user ID to the user's terminal device 2 via the server communication I / F 31. The receiving unit 273 of the user's terminal device 2 stores the received user ID in the terminal storage device 22. Then, when the other user indicated by the stored user ID is a user belonging to an information exchange group, the generating unit 271 determines that the other user indicated by the stored user ID is a user who satisfies the selection condition. This enables the generating unit 271 to automatically generate a character-related image including a character object of the other user who has previously participated in the same event as the user and / or the other user who is currently participating in the same event as the user.

[0269] Hereinafter, another example of the process of determining other users who satisfy the selection condition regarding the relationship between the user and other users will be described. First, the transmitting unit 272 of the user's terminal device 2 transmits information indicating a transmission request regarding other users who played a so-called online game that the user played in the past, together with the user's user ID, to the server device 3 via the terminal communication I / F 21.

[0270] The premise of this case is that the server device 3 has a function of providing one or more online games to multiple users, and each user who wants to play an online game can connect to the server device 3 and play the online game by performing a predetermined operation on the terminal device 2. The server device 3 stores the user ID of each user who played the online game in the server storage device 32, in association with information about the online game (such as the game ID).

[0271] When the server receiving unit 331 of the server device 3 receives the information indicating a transmission request and the user ID of the user via the server communication I / F 31, it extracts the user ID of the other user associated with the game ID associated with the received user ID of the user from the server storage device 32. The user ID of the other user associated with the extracted event ID may be the user ID of the other user whose participation period overlaps with that of the user in at least a part. Then, the server transmitting unit 332 of the server device 3 transmits the extracted user ID to the terminal device 2 of the user via the server communication I / F 31. The receiving unit 273 of the terminal device 2 of the user stores the received user ID in the terminal storage device 22. Then, when the other user indicated by the stored user ID is a user belonging to an information exchange group, the generating unit 271 determines that the other user indicated by the stored user ID is a user who satisfies the selection condition. This makes it possible for the generating unit 271 to automatically generate a character-related image including the character object of the other user who played the same game as the user in the past and / or the other user who is currently playing the same game as the user.

[0272] Other users who satisfy the selection conditions regarding the relationship between the user and other users are not limited to the examples described above, and for example, other users who satisfy the selection conditions regarding the relationship between the user and other users may include other users who are followed by the user, other users who follow the user, other users who are mutual followers of the user, other users who are in a mutual follow relationship with a specific user who is mutually following the user, other users who are friends with the user, other users whose specific information (telephone number, email address, a specific ID (identification), etc.) is stored in the user's terminal device 2, etc.

[0273] When the selection condition is a condition related to the relationship between the user and other users, among the other users who satisfy the selection condition, the character objects of the other users up to the other user set number UB may be used in the additional generation process. For example, when the other user set number UB is "3", even if there are ten other users who satisfy the selection condition related to the relationship between the user and other users, the character objects of three other users out of the ten other users are used in the additional generation process.

[0274] The other user set number UB may be preset or automatically set. For example, in the process of determining other users who satisfy the selection condition, the other user set number UB stored in the terminal storage device 22 may be used. In addition, in the process of determining other users who satisfy the selection condition, the other user set number UB randomly set by the terminal processing device 27 may be used. In addition, when the other user set number UB is stored in association with object data for character-related images, the other user set number UB associated with object data (for character-related images) acquired in response to a user's operation on the other object selection screen 1300 may be automatically used in the process of determining other users who satisfy the selection condition.

[0275] When the selection condition is a condition related to the relationship between the user and other users and the other user set number UB is used, if the number of other users who satisfy the selection condition is less than the other user set number UB, the additional generation process may not be executed. Alternatively, when the number of other users who satisfy the selection condition is less than the other user set number UB, other users may be selected from other users who do not satisfy the selection condition so that the number of character objects of other users used in the additional generation process reaches the other user set number UB. This selection of other users may be selected from other users by manual operation of the user, or may be selected randomly from other users. In addition, when the number of other users who satisfy the selection condition exceeds the other user set number UB, other users of the other user set number UB may be randomly selected from other users who satisfy the selection condition, or other users of the other user set number UB may be selected by manual operation of the user.

[0276] When the selection condition is a condition related to the relationship between the user and other users and the number of other users set UB is used, the selection condition (condition related to the relationship between the user and other users) may be used in a plurality of cases in the additional generation process. In this case, a priority order is set for each of the conditions related to the relationship between the user and other users.

[0277] Hereinafter, an example will be described in which the selection condition of the first priority is a condition of "having exchanged information one-to-one with the user" and the selection condition of the second priority is "belonging to the same group as the user set in the specified event". For example, the generation unit 271 extracts other users who "have exchanged information one-to-one with the user" from among the users belonging to the information exchange group. Next, the generation unit 271 determines the extracted other users who "have exchanged information one-to-one with the user" as other users who satisfy the selection condition of the first priority. Next, if the number of other users who satisfy the selection condition of the first priority is less than the number of other users set UB, other users who "belong to the same group as the user set in the specified event" are extracted from among the users who do not satisfy the selection condition of the first priority of the information exchange group. Next, the generation unit 271 determines the extracted other users who "belong to the same group as the user set in the specified event" as other users who satisfy the selection condition of the second priority. Next, the generation unit 271 sets the other users who satisfy the selection condition of the first priority and the other users who satisfy the selection condition of the second priority determined to be the other users who satisfy the selection condition. Then, the generation unit 271 calculates the total number of other users who satisfy the selection condition.

[0278] When the calculated total is the other user set number UB, the generation unit 271 uses the other users who satisfy the selection condition. When the calculated total exceeds the other user set number UB, the generation unit 271 sets the other users who are reselected from the other users who satisfy the selection condition of the second priority as the other users who satisfy the selection condition of the second priority so that the number of other users who satisfy the selection condition becomes the other user set number UB. The reselection may be a selection by a manual operation of the user, or may be an automatic random selection by the generation unit 271. When the calculated total is less than the other user set number UB, the generation unit 271 sets the other users who satisfy the selection condition by manual operation of the user or randomly from among the belonging users who do not satisfy the selection conditions of the first and second priorities of the information exchange group so that the number of other users who satisfy the selection condition becomes the other user set number UB.

[0279] When the selection condition is a condition related to the relationship between the user and other users, multiple types of other user setting numbers UB may be used. For example, four types of numerical values, "1", "2", "3", and "4", may be set as the other user setting number UB. The other user setting number UB may be two or three types of numerical values, or may be five or more types of numerical values. For example, when the number of other users satisfying the selection condition is "1", a character-related image in the case where there is one other user is generated in the additional generation process. Also, when the number of other users satisfying the selection condition is "2", a character-related image in the case where there are two other users is generated in the additional generation process.

[0280] 15 is a diagram showing an example of an information exchange screen 1400 displayed on the display device 23 of the user's terminal device 2. The information exchange screen 1400 is displayed when a new character-related image is generated by the additional generation process.

[0281] The information exchange screen 1400 includes other user input information 1401, other user information 1402, a text input object 1003, a candidate display button 1004, user input information 1404, and an additional generation button 1102, similar to the information exchange screen 1100. The candidate information 1403 of the information exchange screen 1400 includes the four types of candidate information 1101 of the information exchange screen 1100 and candidate information corresponding to one type of new character-related image. When candidate information corresponding to a new character-related image is included in the candidate information 1403, one or more types of candidate information among the four types of candidate information 1101 previously included may be deleted from the candidate information 1403. For example, by deleting two or more types of candidate information, the number of options for candidate information is reduced, making it possible to facilitate the user's selection of information.

[0282] When one piece of candidate information is selected from six types of candidate information 1403 in response to a user's operation of the input device 24, the input device 24 sends the related image ID associated with the selected candidate information to the terminal processing device 27. Next, the display processing unit 274 extracts, from the terminal storage device 22, a character-related image identified by the related image ID received from the input device 24, and displays the extracted character-related image in the user display area of ​​the information exchange screen 1400 as user input information 1404 (FIG. 15).

[0283] Along with displaying the user input information 1404 by the display processing unit 274, the transmission unit 272 transmits the related image ID received from the input device 24 and the user ID stored in the terminal storage device 22 to the server device 3 via the terminal communication I / F 21. The server transmission unit 332 of the server device 3 extracts the user IDs of other users associated with the same information exchange group as the user ID received by the server reception unit 331 from the server storage device 32. In addition, the server transmission unit 332 transmits the related image ID and user ID received by the server reception unit 331 via the server communication I / F 31 to the terminal device 2 of the other user identified by the extracted user ID.

[0284] Then, the receiving unit 273 of the terminal device 2 of the other user receives the related image ID and the user ID transmitted from the server device 3 via the terminal communication I / F 21. Next, the display processing unit 274 of the terminal device 2 of the other user extracts a character-related image identified by the related image ID received by the receiving unit 273 from the terminal storage device 22, and displays the extracted character-related image in the other user display area of ​​the distribution screen as other user input information. In this way, the same image as the character-related image used by the user is also displayed in the terminal device 2 of the other user associated with the information exchange group to which the user belongs.

[0285] In addition, when the display processing unit 274 of the terminal device 2 of the other user does not store the character-related image identified by the received related image ID in the terminal storage device 22, the transmitting unit 272 of the terminal device 2 of the other user transmits an image transmission request to the server device 3. The image transmission request is transmitted together with the received related image ID and / or user ID, and the user ID of the other user. When the server receiving unit 331 receives the image transmission request, the server transmitting unit 332 of the server device 3 extracts the character-related image associated with the related image ID received together with the image transmission request from the server storage device 32. Next, the server transmitting unit 332 transmits the extracted character-related image together with the received related image ID and user ID to the terminal device 2 of the other user. The receiving unit 273 of the terminal device 2 of the other user receives the related image ID, the character-related image, and the user ID transmitted from the server device 3 via the terminal communication I / F 21, and stores them in the terminal storage device 22 in association with each other. Then, the display processing unit 274 of the terminal device 2 of the other user displays the stored character-related image in the other user display area of ​​the distribution screen as other user input information.

[0286] The new character-related image may also be usable in information exchanges related to information exchange groups other than the information exchange group related to the information exchange screen 1100 selected by the user to start the additional generation process. For example, candidate information corresponding to the new character-related image is also displayed on the information exchange screen related to the other information exchange group, and in response to the selection of candidate information by the user, a character-related image corresponding to the selected candidate information is displayed on the terminal device 2 of another user belonging to the other information exchange group.

[0287] In addition, the new character-related image may be controlled so as not to be used in information exchange related to an information exchange group that does not include other users represented by character objects included in the new character-related image. For example, the display processing unit 274 of the user's terminal device 2 does not display candidate information of a character-related image that includes only the character object of the user and character objects of one or more other users not included in the information exchange group related to the currently displayed information exchange screen. This makes it possible to restrict the use of the character-related image in an information exchange group that does not include any other users represented by character objects included in the character-related image, and to prevent unintended spread of the character object.

[0288] (Operation sequence of information processing system 1) 16 is a diagram showing an example of an operation sequence of the information processing system 1. This operation sequence is executed mainly by the terminal processing device 27 and the server processing device 33 in cooperation with each element of the two terminal devices 2 and the server device 3, based on a control program previously stored in the terminal storage device 22 and the server storage device 32, respectively. Hereinafter, of the two terminal devices 2, the distribution terminal device 2a is defined as the terminal device 2 operated by the distribution user, and the viewing terminal device 2b is defined as the terminal device 2 operated by the viewing user.

[0289] First, the transmitting unit 272 of the distribution terminal device 2a transmits the character video data generated by the generating unit 271 and the broadcast user ID of the broadcast user to the server device 3 via the terminal communication I / F 21 (step S101). The transmitting unit 272 may transmit the voice data of the broadcast user together with the character video data to the server device 3. From the start to the end of the video distribution service, the process of step S101 is executed continuously at predetermined time intervals (for example, at two-second intervals). Then, in response to the execution of the process of step S101, the processes of steps S102 to S104 are executed intermittently.

[0290] Each time the server receiving unit 331 receives character video data, the server transmitting unit 332 of the server device 3 distributes the received character video data to the viewing terminal device 2b via the server communication I / F 31 (step S102). The display processing unit 274 of the distribution terminal device 2a generates drawing data for displaying a distribution image based on the character video data generated by the generating unit 271, and displays a distribution screen on the display device 23 based on the generated drawing data (step S103). The display processing unit 274 of the viewing terminal device 2b generates drawing data for displaying a distribution image based on the character video data received by the receiving unit 273, and displays a distribution screen on the display device 23 based on the generated drawing data (step S104).

[0291] Next, the transmitting unit 272 of the viewing terminal device 2b transmits information on a display instruction for displaying the response information on the distribution screen and / or information indicating an additional instruction for displaying the additional object on the distribution screen, which is input in response to the operation of the input device 24 by the viewing user, to the server device 3 via the terminal communication I / F 21 (step S105). Every time the server receiving unit 331 receives information on various instructions (display instruction, additional instruction, etc.), the server transmitting unit 332 of the server device 3 distributes the received information on various instructions to the distribution terminal device 2a via the server communication I / F 31 (step S106). The display processing unit 274 of the viewing terminal device 2b generates drawing data for displaying the response information and / or the additional object based on the various instructions input in response to the operation of the input device 24 by the viewing user, and displays the distribution screen on the display device 23 based on the generated drawing data (step S107). The display processing unit 274 of the distribution terminal device 2a generates drawing data for displaying the response information and / or the additional object based on the information regarding the various instructions received by the receiving unit 273, and displays the distribution screen on the display device 23 based on the generated drawing data (step S108).

[0292] Next, the display processing unit 274 of the distribution terminal device 2a executes a candidate information display process (step S109). The details of the candidate information display process will be described later. Note that the candidate information display process is not limited to a process that can be executed only by the distribution terminal device 2a. For example, the display processing unit 274 of the viewing terminal device 2b, which has a character object representing a user available, may execute the candidate information display process.

[0293] Next, the generation unit 271 and the display processing unit 274 of the distribution terminal device 2a execute the additional generation process (step S110). The details of the additional generation process will be described later. The additional generation process is not limited to a process that can be executed only by the distribution terminal device 2a. For example, the generation unit 271 and the display processing unit 274 of the viewing terminal device 2b, which has a character object representing a user available, may execute the candidate information display process.

[0294] Next, the transmission unit 272 of the distribution terminal device 2a transmits the input information inputted in response to the operation of the input device 24 by the distribution user to the server device 3 via the terminal communication I / F 21 (step S111). Next, the server transmission unit 332 of the server device 3 distributes the received input information to the viewing terminal device 2b via the server communication I / F 31 every time the server reception unit 331 receives the input information (step S112). The display processing unit 274 of the distribution terminal device 2a generates drawing data for displaying the input information inputted in response to the operation of the input device 24 by the distribution user, and displays an information exchange screen on the display device 23 based on the generated drawing data (step S113). The display processing unit 274 of the viewing terminal device 2b generates drawing data for displaying the input information received by the reception unit 273, and displays an information exchange screen on the display device 23 based on the generated drawing data (step S114).

[0295] 17 is a diagram showing an example of an operational flow of the candidate information display process, which is executed in step S110 of FIG.

[0296] First, the display processing unit 274 determines whether or not a display instruction for candidate information has been acquired (step S201). If a display instruction for candidate information has not been acquired (step S201-No), the process returns to step S201 (waits until a display instruction for candidate information is acquired).

[0297] When an instruction to display candidate information is acquired (step S201-Yes), the display processing unit 274 determines whether or not a character-related image has been generated in the past and stored in the terminal storage device 22 (step S202). When a character-related image has not been stored (step S202-No), the generation unit 271 generates a character-related image (step S303) and advances the process to step S204.

[0298] If a character-related image has been generated in the past and stored in the terminal storage device 22 (step S202-Yes), the generation unit 271 generates candidate information corresponding to the character-related image, and the display processing unit 274 displays the generated candidate information (step S204), and terminates the candidate information display process.

[0299] 18 is a diagram showing an example of an operational flow of the additional generation process. The additional generation process shown in FIG. 18 is executed in step S110 of FIG.

[0300] First, when the additional generation process is started in response to the user's selection of the additional generation button 1102 included in the information exchange screen 1100, the display processing unit 274 displays the other object selection screen 1300 on the display device 23 (step S301). Next, the display processing unit 274 determines whether or not a user selection instruction has been input in response to the user's selection of the user selection instruction button 1303 included in the other object selection screen 1300 (step S302).

[0301] If it is determined that the user selection instruction has not been input (step S302-No), the display processing unit 274 ends the additional generation process. If it is determined that the user selection instruction has been input (step S302-Yes), the display processing unit 274 displays the user selection screen 1310 on the display device 23 (step S303).

[0302] Next, the display processing unit 274 determines whether or not an additional generation instruction has been input in response to the user's selection of the additional generation instruction button 1313 included in the user selection screen 1310 (step S304). If it is determined that an additional generation instruction has not been input (step S304-No), the display processing unit 274 ends the additional generation process. If it is determined that an additional generation instruction has been input (step S304-Yes), the generation unit 271 generates a new character-related image (step S305). Then, the display processing unit 274 displays the generated new character-related image (step S306), and the additional generation process ends.

[0303] As described above in detail, in the information processing system 1 of the present embodiment, the types of character-related images that can be used can be increased based on information about character objects that indicate users other than the user who instructed the generation or addition of the character-related image. In this way, the information processing system 1 of the present embodiment can improve the user interface that determines the information to be provided to other users without making the user perform complicated operations. In addition, in the information processing system 1 of the present embodiment, the types of character-related images can be changed based on information about character objects that indicate users other than the user who instructed the generation or addition of the character-related image. In this way, the information processing system 1 of the present embodiment can change the information provided to other users by the user without making the user perform complicated operations, thereby preventing the user from getting bored with the selection.

[0304] (Variation 1) The present invention is not limited to this embodiment. For example, on the information exchange screen, stamp images may be classified by type and displayed. For example, the stamp images include stamp images for conveying "OK", stamp images for conveying "NO", stamp images with "smiling" expressions, etc. In addition, the stamp images may include stamp images generated as character-related images, stamp images generated as new character-related images, etc.

[0305] FIG. 19 is a diagram showing an example of information exchange screens 1500 and 1510 including stamp images generated as character-related images. The information exchange screens 1500 and 1510 include type selection buttons 1501 for selecting the type of stamp image. When candidate information corresponding to any of the type selection buttons 1501 is selected in response to the user's operation of the input device 24, a stamp image of a type corresponding to the selected type selection button 1501 is displayed in a display area below the type selection button 1501. In FIG. 19, a button is exemplified as a widget for selecting the type of stamp image, but the widget for selecting the type of stamp image may be a tab. In this way, in the information exchange screens 1500 and 1510, a plurality of stamp images are displayed by switching between types.

[0306] 19(a) is a diagram showing an example of an information exchange screen 1500 when a type selection button 1501 corresponding to a stamp image generated as a character-related image is selected. On the information exchange screen 1500, candidate information 1101 corresponding to the stamp image generated as a character-related image is displayed. When the type selection button 1501 corresponding to the stamp image for conveying "OK" is selected, the information exchange screen 1500 displays candidate information corresponding to the stamp image for conveying "OK".

[0307] Fig. 19(b) is a diagram showing an example of an information exchange screen 1510 when a type selection button 1501 corresponding to a stamp image generated as a new character-related image is selected. On the information exchange screen 1510, the type selection button 1501 corresponding to the stamp image generated as a new character-related image is selected, and in response to this selection, candidate information 1511 corresponding to the stamp image generated as a new character-related image is displayed. In the example shown in Fig. 21(b), the type selection button 1501 corresponding to the stamp image generated as a new character-related image is a button object labeled "Add".

[0308] In this way, the information exchange screen is configured to display stamp images classified by type, thereby improving the user interface. Such an information exchange screen makes it possible to improve the visibility of a plurality of stamp images even if the size of the display screen of the terminal device 2 is limited. Furthermore, such information exchange screens 1500 and 1510 allow the user to intuitively and / or easily find the stamp image he or she wishes to use. Furthermore, such information exchange screens 1500 and 1510 allow the user to grasp at a glance whether or not there is a stamp image that has been generated as a new character-related image.

[0309] (Variation 2) At least a part of the functions realized by the terminal processing device 27 of the terminal device 2 may be realized by a processor other than the terminal processing device 27 of the terminal device 2. At least a part of the functions realized by the terminal processing device 27 may be realized, for example, by the server processing device 33 of the server device 3. Specifically, a part of the functions of the generation unit 271 and the display processing unit 274 may be executed in the server device 3. For example, the terminal device 2 transmits continuously acquired imaging data and audio data to the server device 3 together with the user ID of the user who operates the terminal device 2. Then, the server device 3 generates character video data using the functions of the generation unit 271 and the display processing unit 274, and generates display information for displaying a communication screen including the delivery images of all users on the display device 23. Then, the server device 3 transmits the display information together with audio information (audio data) to each terminal device 2 of all users. Each terminal device 2 outputs each piece of information received from the server device 3 (so-called "server rendering").

[0310] (Variation 3) The terminal device 2 has a function of displaying a communication screen or the like by executing various commands included in a control program stored in the terminal device 2, but in order to realize an information exchange service and / or a video distribution service, the terminal device 2 may call up a browser function incorporated in a web application, receive a web page (a document written in a markup language such as HTML (Hyper Text Markup Language)) from a server device using this browser function, and execute the control program incorporated in this web page (so-called "browser rendering"). For example, by using HTML 5 as a markup language, the terminal device 2 can easily execute new information processing. By adopting such a web application to realize an information exchange service and / or a video distribution service in the terminal device 2, a program creator can provide new information processing to the client side (terminal device side) simply by incorporating a new program in a web page transmitted by the server device, and it becomes possible to greatly reduce the man-hours required for creating a new program. In addition, the client side can receive the new service simply by receiving a web page without downloading a new control program, which makes it possible to reduce the load on the communication network, communication costs, and / or communication time compared to downloading a control program, and to simplify the user interface.

[0311] (Variation 4) The generating unit 271 of the terminal device 2 of the user A may generate output information including face motion data without using imaging data. For example, the terminal storage device 22 of the terminal device 2 stores face motion data corresponding to voice data, and the generating unit 271 may extract face motion data corresponding to the acquired voice data of the user A from the terminal storage device 22 and generate output information including the extracted face motion data, the voice data, and the user ID of the user A. The generating unit 271 of the terminal device 2 of the user A may generate output information including the voice data of the user A output from the microphone 26 and the user ID stored in the terminal storage device 22 without including the face motion data. In this case, the display processing unit 274 of the terminal device 2 of the user B may extract face motion data corresponding to the voice data of the user A included in the output information of the user A received via the server device 3 and generate animation of the character object of the user A. The generation process of the face motion data corresponding to the voice data of the user may be a process using a known lip sync algorithm.

[0312] For example, in the case where the terminal device 2 of the user A uses an HMD as the display device 23, communication with the user B can be performed via a character object without acquiring image data of the user A.

[0313] (Variation 5) The delivery images 602, 612, 812 displayed on each terminal device 2 are not limited to moving images or still images including animation of a three-dimensional or two-dimensional character object generated based on the actions of the delivery user. The delivery images 602, 612, 812 may include animation of a character object generated in response to the operation of the delivery user, and may also include moving images or still images of the delivery user himself. Note that, during the video delivery service, each terminal device 2 does not have to display animation of these character objects or moving images or still images of the delivery user himself. In this case, each terminal device 2 may output only the voice of the delivery user from a speaker.

[0314] In this way, the "broadcast user" disclosed in this embodiment is not limited to a user who at least uses the function of the terminal device to transmit character video data to the server device, but may also be a user who transmits information and / or audio data for displaying various broadcast images. Information for displaying various broadcast images is, for example, character video data, video data of the broadcast user himself, etc.

[0315] (Variation 6) Furthermore, the "broadcast user" disclosed in this embodiment is not limited to a user who hosts or hosts a single video distribution service. The "broadcast user" may be a user who hosts or hosts a collaborative distribution service in which multiple people can participate, or a video chat service or voice chat service in which multiple people can participate and / or watch. Alternatively, the "broadcast user" may be a user who hosts or hosts an event (such as a party) service in a virtual space in which multiple people can participate and / or watch. In this way, the "broadcast user" may be a user who mainly performs these services, and may be rephrased as a host user, organizer user, or organizer user.

[0316] The "viewing user" disclosed in this embodiment is not limited to a user who uses at least the function of a terminal device to receive character video data from a distribution terminal device and display a distribution image of a distribution user. The "viewing user" may be a user who receives information and / or audio data for displaying various distribution images.

[0317] The "viewing user" may be not only a user who receives information and / or audio data for displaying various distribution images, but also a user who can react. For example, the "viewing user" may be a user who watches a video distribution service or a collaboration distribution service, or a user who participates in a video chat service, a voice chat service, or an event service and / or a user who watches these services. Therefore, the "viewing user" in the present disclosure may be rephrased as a guest user, a participating user, a listener, a viewing user, or a supporting user.

[0318] (Variation 7) Furthermore, the information processing system 1 according to the embodiment of the present disclosure may provide a digital world in which multiple people can participate simultaneously and freely engage in social activities such as social interaction, work, and / or play through character objects (e.g., avatars) at a level close to that of the real world, such as the next Internet space (metaverse), where social activities can take place beyond the gap between reality and virtuality. In such a space, users' avatars can freely roam the world and communicate with each other.

[0319] The information processing system 1 may distribute a distribution image including one avatar (character object) among multiple avatars in the space as a character object of a distribution user. That is, one-to-many video distribution may be performed in a many-to-many metaverse space. In such a space, it is possible that there is no particular distinction between a distribution user and a viewing user.

[0320] The relationship between the broadcasting user's character object and the viewing user's character object, which exist in the metaverse, may be used in the process of determining other users who satisfy a selection condition regarding the relationship between the user and other users. For example, the server device 3 stores, in the server storage device 32 at predetermined time intervals (e.g., every second), position information indicating the position in the metaverse of the broadcasting user's character object and position information indicating the position in the metaverse of the viewing user's character object.

[0321] When the user space of the broadcasting user is set in the metaverse, the server device 3 stores a user ID indicating the broadcasting user as an entering user ID in the server storage device each time the character object of the viewing user enters the user space of the broadcasting user. The entering user ID is stored in association with a user ID (broadcasting user ID) indicating the broadcasting user. For example, the user space is a space set in the metaverse as a private room of the character object of the broadcasting user. The user space may also be a space set in the metaverse as a studio where the character object of the broadcasting user broadcasts videos. The user space may also be a space set in the metaverse as a gathering place for a group to which the viewing user belongs at a specified event.

[0322] In this case, first, the transmitting unit 272 of the user's terminal device 2 transmits information indicating a transmission request for the entering user ID of another user who has entered the user space of the user, together with the user ID of the user, to the server device 3 via the terminal communication I / F 21. When the server receiving unit 331 of the server device 3 receives the information indicating the transmission request and the user ID of the user via the server communication I / F 31, it extracts the entering user ID associated with the received user ID of the user from the server storage device 32. Then, the server transmitting unit 332 of the server device 3 transmits the extracted entering user ID to the user's terminal device 2 via the server communication I / F 31.

[0323] The receiving unit 273 of the user's terminal device 2 stores the received entering user ID in the terminal storage device 22. Then, when the other user indicated by the stored entering user ID is a user belonging to an information exchange group, the generating unit 271 determines that the other user indicated by the entering user ID is a other user who satisfies the selection condition. This enables the generating unit 271 to automatically generate a character-related image including a character object of the other user who has entered the user space of the user set in the metaverse.

[0324] Although the present embodiment and its modified examples have been described in detail above, the present invention is not limited to the specific examples. Furthermore, various changes, substitutions, and modifications can be made to the present invention without departing from the scope of the present invention. [Explanation of symbols]

[0325] 1. Information Processing Systems 2 Terminal Equipment 21 Terminal communication I / F 22 Terminal storage device 23 Display device 24 Input Devices 25 Imaging device 26. Mike 27 Terminal Processing Device 271 Generation part 272 Transmitter 273 Receiving section 274 Display Processing Unit 275 Audio output section 3. Server equipment 31 Server communication I / F 32 Server Storage Device 33 Server Processing Unit 331 Server Reception Unit 332 Server transmission unit 4 base station 5. Mobile communication networks 6. Gateway 7. Internet

Claims

1. An information processing system including one or more computer processors, wherein the one or more computer processors set a user group including a first user and one or more second users, generate a related image including a first object indicating the first user and a second object indicating at least one second user among the one or more second users in response to an instruction by the first user, and transmit information regarding display of the related image to terminal devices of the second users included in the user group. An information processing system characterized by performing the above.

2. The information processing system according to claim 1, wherein, in setting the user group, the one or more computer processors set a plurality of the user groups.

3. The information processing system according to claim 1 or 2, wherein the related image includes the second object indicating one or more of the second users selected by the first user.

4. The one or more computer processors select one or more of the second users, and the related image includes the second object indicating the selected one or more of the second users. The information processing system according to claim 1.

5. In the selection of the one or more second users, the one or more computer processors select the one or more second users who transmitted the information in the order of the time of transmission being closer to the current time among the information transmitted most recently by each second user in a chat among the users included in the user group. The information processing system according to claim 4.

6. In the selection of the one or more second users, the one or more computer processors execute the selection of the one or more second users whose time when a chat screen among users included in the user group is displayed on the terminal device is close to the current time, for the information processing system according to claim 4.

7. In the selection of the one or more second users, the one or more computer processors execute the selection of the one or more second users having a predetermined relationship with the first user, for the information processing system according to claim 4.

8. The second user having the predetermined relationship with the first user is a user who has exchanged information one-on-one with the first user, for the information processing system according to claim 7.

9. The second user having the predetermined relationship with the first user is a user followed by the first user or a user having a mutual follow relationship with the first user, for the information processing system according to claim 7.

10. The second user having the predetermined relationship with the first user is a user who participated in the same event as the first user or a user who participated in the same game as the first user, for the information processing system according to claim 7.

11. The one or more computer processors in response to the number of second users participating in the information exchange among users included in the user group reaching a predetermined number, select the predetermined number of second users, generate the related image including the second object indicating the selected predetermined number of second users, and execute the above for the information processing system according to claim 1 or 2.

12. The related image includes a third object related to the design of a character-related image that is different from the first object and the second object, and includes the first object and the second object in poses corresponding to the third object. The information processing system according to claim 1 or 2.

13. The one or more computer processors Transmit information regarding the display of a video including the first object to a terminal device of a user different from the first user. The information processing system according to claim 1, which executes the above.

14. The one or more computer processors Generate a related image including the first object without including the second object and a related image including the first object and the second object, Transmit information regarding the display of a related image selected by the first user from a group including the related image including the first object without including the second object and the related image including the first object and the second object to a terminal device of the second user included in the user group. The information processing system according to claim 1 or 7, which executes the above.

15. The second user having the predetermined relationship with the first user is a user of a terminal device that has received information regarding the display of a video including a first object indicating the first user. The information processing system according to claim 14, which depends on claim 7.

16. A control method for an information processing system including one or more computer processors, The one or more computer processors Set a user group including a first user and one or more second users, In response to an instruction from the first user, generate a related image including a first object indicating the first user and a second object indicating at least one second user among the one or more second users. Transmitting information regarding display of the related image to a terminal device of the second user included in the user group. A control method including this.

17. A server device that communicates with a first terminal device of a first user and second terminal devices of one or more second users, setting a user group including the first user and one or more second users, generating a related image including a first object indicating the first user and a second object indicating at least one second user among the one or more second users in response to an instruction by the first user, transmitting information regarding display of the related image to terminal devices of the second users included in the user group, A server device characterized by this.

Images