Control program for terminal device, terminal device, method for controlling terminal device, control program for server device, server device, and method for controlling server device

Abstract

This makes it possible to increase users' motivation to continue communicating with other users. [Solution] The terminal device control program causes the terminal device to display at least a first image including a first object representing the first user and a second image including a second object representing the second user on the first user's terminal device, transmit information regarding the display of the first image to the second user's terminal device, transmit the first voice to the second user's terminal device if the first voice of the first user is acquired, display the first image including a first object that changes according to the actions or first voice of the first user, output the second voice if the second voice of the second user is received, display the second image including a second object that changes according to the actions or second voice of the second user, and display the first image or the second image that has been modified in at least part in response to instructions from the first user or the second user.

Description

Technical Field

[0001] The embodiments disclosed in this specification relate to a control program for a terminal device, a terminal device, a control method for a terminal device, a control program for a server device, a server device, and a control method for a server device.

Background Art

[0002] Conventionally, there has been known an information system in which a plurality of users communicate with each other by transmitting various information such as an image indicating each user himself / herself and / or the voice of each user to the terminal devices of other users using terminal devices such as a PC (Personal Computer). For example, Patent Document 1 discloses a technology of an information system in which each user performs a voice chat with a character object of another user using a character object such as an avatar that acts as a user's alter ego in a virtual space.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the conventional information system, when a user communicates with another user, there is no diversity in the output from the user's terminal device, so there has been a problem that the user gets bored with communicating with another user.

[0005] The disclosed control program for a terminal device, terminal device, control method for a terminal device, control program for a server device, server device, and control method for a server device make it possible to improve the willingness of a user who intends to continue communicating with another user.

Means for Solving the Problems

[0006] The terminal device control program is a control program for a first user's terminal device, and causes the first user's terminal device to display at least a first image including a first object representing the first user and a second image including a second object representing a second user different from the first user; transmit information regarding the display of the first image to the second user's terminal device; transmit the first voice to the second user's terminal device if the first voice of the first user is acquired; display the first image including a first object that changes according to the actions or first voice of the first user; output the second voice if the second voice of the second user is received; display the second image including a second object that changes according to the actions or second voice of the second user; and display the first image or the second image that has been modified in at least part in response to instructions from the first user or the second user.

[0007] Furthermore, in the control program of the terminal device, when displaying a first image in which at least a part has been modified, it is preferable to cause the terminal device of the first user to display a first image that includes a first object in which a part has been modified, based on a first candidate object selected from one or more first candidate objects associated with the first user in response to instructions from the first user.

[0008] Furthermore, in the control program for the terminal device, the first object is a three-dimensional object placed in the virtual space, and in displaying the first image including the first object, it is preferable to cause the terminal device of the first user to display an image showing the virtual space as viewed from a predetermined viewpoint within the virtual space as the first image, and to cause the terminal device of the first user to display an image showing the virtual space as viewed from a predetermined viewpoint that moves automatically within the virtual space as the first image.

[0009] Furthermore, in the control program of the terminal device, it is preferable that a predetermined viewpoint is controlled to move closer to the modified part of the first object.

[0010] Furthermore, in the control program of the terminal device, it is preferable that a predetermined viewpoint is controlled to move around the first object while aligning the direction of line of sight from the predetermined viewpoint with the direction facing the first object.

[0011] Furthermore, in the control program for the terminal device, when displaying a first image that has been modified in part, it is preferable to cause the terminal device of the first user to display a first image that includes a third object corresponding to a second candidate object selected from a selection group including one or more second candidate objects in response to instructions from the second user.

[0012] Furthermore, in the control program of the terminal device, it is preferable that a specific second candidate object is included in the selection group when predetermined conditions are met.

[0013] Furthermore, it is preferable that the control program for the terminal device includes conditions relating to the display time of the first image and / or the display time of the second image.

[0014] Furthermore, in the terminal device control program, it is preferable that the first user's terminal device is instructed to display a second image including a third object corresponding to the selected second candidate object, based on the second candidate object selected by the first user from a selection group including one or more second candidate objects, and to transmit information regarding the display of the third object to the second user's terminal device, and that the predetermined conditions include conditions regarding the number of times the first user and / or the second user have selected the second candidate object.

[0015] Furthermore, in the control program of the terminal device, it is preferable to cause the terminal device of the first user to display a plurality of second images representing each of the plurality of second users, and to cause the terminal device of the first user to display the first image and a plurality of second images representing each of the plurality of second users, including a third object corresponding to the second candidate object selected in response to the instructions of the second user.

[0016] Furthermore, it is preferable that the control program of the terminal device allows the first user to instruct the selection of a third object, and that the terminal device of the first user displays a first image including a partially modified first object in response to the first user's instruction to select a third object.

[0017] Furthermore, in the control program for the terminal device, when displaying the first image, which has been modified in part, it is preferable to cause the terminal device of the first user to display an image showing the progress of the game being played by the first user and / or the second user, replacing at least a portion of the first image.

[0018] Furthermore, in the terminal device control program, it is preferable to instruct the first user's terminal device to display an image showing the progress of the game being played by the first user and / or the second user, replacing at least a portion of the second image, if the second candidate object selected in response to instructions from the second user corresponds to a game.

[0019] The server device control program is a server device control program capable of communicating with the terminal device of a first user and the terminal device of a second user different from the first user, and receives information from the first user's terminal device regarding the display of a first image including a first object representing the first user, receives information from the second user's terminal device regarding the display of a second image including a second object representing the second user, transmits at least information for displaying the first image to at least the second user's terminal device, transmits at least information for displaying the second image to at least the first user's terminal device, and when it receives the first voice of the first user, transmits the first voice to the second user's terminal device, and the actions of the first user or The system causes the terminal device to perform the following actions: transmit information to at least the second user's terminal device for displaying a first image containing a first object that changes in response to a first voice; transmit the second voice to the first user's terminal device when it receives a second voice from the second user; transmit information to at least the first user's terminal device for displaying a second image containing a second object that changes in response to the second user's actions or second voice; transmit information to at least the second user's terminal device for displaying a first image that has been modified in at least part in response to instructions from the first user; and transmit information to at least the first user's terminal device for displaying a second image that has been modified in at least part in response to instructions from the second user.

[0020] The terminal device is a terminal device for a first user and includes a processing unit that displays at least a first image including a first object representing the first user and a second image including a second object representing a second user different from the first user, transmits information regarding the display of the first image to the terminal device of the second user, transmits the first voice to the terminal device of the second user when it receives the first voice of the first user, displays the first image including a first object that changes according to the actions or first voice of the first user, outputs the second voice when it receives the second voice of the second user, displays the second image including a second object that changes according to the actions or second voice of the second user, and displays the first image or the second image that has been modified in at least part in response to instructions from the first user or the second user.

[0021] The server device is a server device capable of communicating with the terminal device of the first user and the terminal device of a second user who is different from the first user, and receives information from the first user's terminal device regarding the display of a first image including a first object representing the first user, receives information from the second user's terminal device regarding the display of a second image including a second object representing the second user, transmits at least information for displaying the first image to at least the second user's terminal device, transmits at least information for displaying the second image to at least the first user's terminal device, and when it receives the first voice of the first user, transmits the first voice to the second user's terminal device, and responds to the actions of the first user or the first voice. The system includes a processing unit that transmits information for displaying a first image containing a first object that changes accordingly to at least the terminal device of a second user; when it receives a second voice from the second user, it transmits the second voice to the terminal device of the first user; transmits information for displaying a second image containing a second object that changes according to the actions or voice of the second user to at least the terminal device of the first user; transmits information for displaying a first image that has been modified in at least part according to instructions from the first user to at least the terminal device of the second user; and transmits information for displaying a second image that has been modified in at least part according to instructions from the second user to at least the terminal device of the first user.

[0022] A method for controlling a terminal device is a method for controlling a terminal device of a first user, the first user's terminal device displays at least a first image including a first object representing the first user and a second image including a second object representing a second user different from the first user; transmits information regarding the display of the first image to the second user's terminal device; if the first user's first voice is acquired, transmits the first voice to the second user's terminal device; displays the first image including a first object that changes according to the first user's actions or first voice; if the second user's second voice is received, outputs the second voice; displays the second image including a second object that changes according to the second user's actions or second voice; and displays the first image or the second image that has been modified in at least part in response to instructions from the first user or the second user.

[0023] A control method for a server device is a control method for a server device capable of communicating with each of a terminal device of a first user and a terminal device of a second user different from the first user. Information regarding the display of a first image including a first object indicating the first user is received from the terminal device of the first user, and information regarding the display of a second image including a second object indicating the second user is received from the terminal device of the second user. Information for at least displaying the first image is transmitted to at least the terminal device of the second user, and information for at least displaying the second image is transmitted to at least the terminal device of the first user. When the first voice of the first user is received, the first voice is transmitted to the terminal device of the second user. Information for displaying a first image including a first object that changes according to the operation or first voice of the first user is transmitted to at least the terminal device of the second user. When the second voice of the second user is received, the second voice is transmitted to the terminal device of the first user. Information for displaying a second image including a second object that changes according to the operation or second voice of the second user is transmitted to at least the terminal device of the first user. Information for displaying a first image that is at least partially changed according to an instruction from the first user is transmitted to at least the terminal device of the second user, and information for displaying a second image that is at least partially changed according to an instruction from the second user is transmitted to at least the terminal device of the first user.

Effect of the Invention

[0024] With a terminal device control program, a terminal device, a terminal device control method, a server device control program, a server device, and a server device control method, it becomes possible to improve the willingness of a user who wants to continue communication with other users.

Brief Description of the Drawings

[0025] [Figure 1] It is a schematic diagram for explaining an example of the outline of an information system. [Figure 2] It is a diagram showing an example of the schematic configuration of an information system. [Figure 3] It is a diagram showing an example of the schematic configuration of a terminal device. [Figure 4] This figure shows an example of the data structure of various tables. [Figure 5] This figure shows an example of a general configuration of a server device. [Figure 6] This figure shows an example of a screen displayed on the terminal device's display device. [Figure 7] This figure shows an example of a screen displayed on the terminal device's display device. [Figure 8] This figure shows an example of a screen displayed on the terminal device's display device. [Figure 9] This figure shows an example of a screen displayed on the terminal device's display device. [Figure 10] This figure shows an example of a screen displayed on the terminal device's display device. [Figure 11] This figure shows an example of a screen displayed on the terminal device's display device. [Figure 12] This figure shows an example of a screen displayed on the terminal device's display device. [Figure 13] This figure shows an example of a screen displayed on the terminal device's display device. [Figure 14] This figure shows an example of a screen displayed on the terminal device's display device. [Figure 15] This figure shows an example of a screen displayed on the terminal device's display device. [Figure 16] This figure shows an example of the operation sequence of an information system. [Figure 17] This figure shows an example of a screen displayed on the terminal device's display device. [Figure 18] This is a schematic diagram used to explain the virtual space. [Modes for carrying out the invention]

[0026] Various embodiments of the present invention will be described below with reference to the drawings. However, it should be noted that the technical scope of the present invention is not limited to these embodiments, but extends to the invention described in the claims and its equivalents.

[0027] (Overview of the information system) Figure 1 is a schematic diagram illustrating an example of an information system. The information system comprises terminal devices operated by multiple users and a server device. For example, a terminal device is an information processing device such as a multi-function mobile phone (a so-called "smartphone") owned by the user. For example, a server device is a computer that provides communication services between terminal devices via a communication network.

[0028] The terminal device stores control programs such as application programs, loads the control programs into memory in response to a user's start command, and initiates the communication service by executing the instructions contained in the loaded control programs. Once the communication service is initiated, the terminal device implements multiple functions by executing the instructions contained in the control programs.

[0029] For example, the user's terminal device implements a generation function that generates output information including character video data, which includes motion data based on various input data entered by the user. For example, the input data consists of multiple image data acquired at predetermined sampling time intervals by an imaging device provided in the user's terminal device, in this case, each image data shows the user. The character video data is an example of information related to the display of a user output image, which includes a character object representing the user. During the communication service, the output information is generated at predetermined time intervals.

[0030] Furthermore, for example, the user's terminal device displays a user output image based on character video data and transmits the generated output information to other users' terminal devices via the server device, thereby implementing an output function that transmits said output information to the server device at predetermined intervals. Also, for example, the user's terminal device implements a function that displays another user's output image, including a character object representing the other user, based on the output information of other users transmitted from the server device at predetermined intervals.

[0031] When the communication service is started, as shown in Figure 1(a), the user's terminal device will display, for example, the user output image (1) and the other user's output image (2). If no other users are participating in the communication service when it is started, only the user output image (1) will be displayed on the user's terminal device. If the user is not participating in the communication service when it is started, only the other user's output image (2) will be displayed on the other users' terminal devices.

[0032] The user output image (1) includes the user's character object, which operates according to the user's motion data contained in the character video data. The other user output image (2) includes the other user's character object, which operates according to the other user's motion data contained in the received output information from the other user. In this way, the user's terminal device displays the user output image (1), which includes the user's character object that changes according to the user's actions, and displays the other user output image (2), which includes the other user's character object that changes according to the other user's actions.

[0033] Since both the user's terminal device and other users' terminal devices are equipped with microphones, the user's terminal device acquires voices emitted by the user, and other users' terminal devices acquire voices emitted by other users. The user's terminal device then includes the acquired voice data of the user in its output information and transmits this output information, including the user's voice data, to the other users' terminal devices via the server device. The user's terminal device also receives the output information, including the other users' voice data, transmitted to the server device by the other users' terminal devices, from the server device.

[0034] For example, when a user produces sound, the user's motion data included in the character video data includes face motion data showing the user's lip movements. In this case, the user's terminal device will display a user output image (1) that includes the user's character object, whose lips move in roughly sync with the sound produced by the user. Output information from other users received by the user's terminal device may include the other user's voice data along with the other user's motion data. In this case, the user's terminal device will output the other user's voice, and display an other user output image (2) that includes the other user's character object, whose lips move in roughly sync with the other user's voice.

[0035] In addition to the functions described above, the user's terminal device may implement a function to modify at least a portion of the user output image (1) in response to user instructions and display the modified user output image (1). Furthermore, the user's terminal device may implement a function to modify at least a portion of another user output image (2) in response to user instructions and display the modified other user output image (2).

[0036] For example, the user's terminal device changes at least a portion of the user's character object in response to a change instruction from the user, thereby changing at least a portion of the user output image (1). As shown in Figure 1(b), for example, the user's terminal device changes a portion of the user's character object (the item "hat" attached to the head) in response to a change instruction from the user (the item "hat" is changed to the item "ribbon"). For example, the user's change instruction is input by a selection operation in which the user selects a predetermined operation object (such as a button object). Alternatively, the user's change instruction may be input by a selection operation in which the user selects one of several candidate objects, each corresponding to a different item. In this case, the items of the user's character object included in the user output image (1) are changed to items corresponding to the candidate object selected by the user. Or, a user output image (1) including the user's character object with the item corresponding to the candidate object selected by the user added may be displayed.

[0037] Furthermore, for example, the user's terminal device may, in response to additional instructions from the user, include additional objects in the other user's output image (2), thereby modifying at least a portion of the other user's output image (2). For example, as shown in Figure 1(c), the user's terminal device may, in response to additional instructions from the user, include additional objects in the other user's output image (2) so as to be positioned around the other user's character object or superimposed on at least a portion of the other user's character. Additional instructions from the user may be input by a selection operation in which the user selects a predetermined operation object (such as a button object). Alternatively, additional instructions from the user may be input by a selection operation in which the user selects one of several candidate objects, each corresponding to a different additional object. In this case, the other user's output image (2) containing the additional object corresponding to the candidate object selected by the user is displayed.

[0038] Furthermore, the user's terminal device may implement a function to modify at least a portion of the user output image (1) in response to instructions from other users and display the modified user output image (1). For example, in this function, the user's terminal device may include additional objects in the user output image (1) so as to be placed around the user's character object or superimposed on at least a portion of the user's character, in response to additional instructions from other users. Furthermore, the user's terminal device may implement a function to modify at least a portion of another user's output image (2) in response to instructions from other users and display the modified other user's output image (2). For example, in this function, the user's terminal device may change a portion of another user's character object in response to a change instruction from another user and display the other user's output image (2) including the changed other user's character object.

[0039] Furthermore, the user's terminal device may change its output in response to instructions from the user or other users. For example, the user's terminal device may change the sound quality of the user's voice or another user's voice for a predetermined period in response to instructions from the user or other users. In addition, the user's terminal device may operate its vibration unit for a predetermined period in response to instructions from the user or other users.

[0040] As explained above with reference to Figure 1, the terminal device control program, terminal device, terminal device control method, server device control program, server device, and server device control method display a user output image or another user output image that has been modified in at least part, in response to instructions from the user or another user. In this way, the terminal device control program, terminal device, terminal device control method, server device control program, server device, and server device control method make it possible to improve the user's motivation to continue using the communication service by allowing the output from the terminal device to be modified by the user or another user during the communication service.

[0041] In the example shown in Figure 1, the user and other users are users who belong to a predetermined communication group among multiple users who can use the communication service. The predetermined communication group is a group created by either the user or the other users. In the example shown in Figure 1, the character objects of two users are displayed, but the system may be configured to display the character objects of three or more users belonging to the predetermined communication group, allowing three or more users to communicate.

[0042] Furthermore, in the example shown in Figure 1, the user and other users may be users who have a predetermined relationship. For example, other users who have a predetermined relationship with the user are mutual followers of the user. Note that if the user follows another user and the other user also follows the user (the user and the other user are mutually following each other), the user and the other user become mutual followers. Also, other users who have a predetermined relationship with the user may be other users who are mutually following the predetermined user who is a mutual follower of the user. Furthermore, other users who have a predetermined relationship with the user may be other users who are friends with the user, or other users whose specific information (telephone number, email address, predetermined ID (identification), etc.) is stored on the user's terminal device. In addition, if the user's terminal device has a function to distribute user output images, including the user's character object, to one or more other users' terminal devices via a server device, other users who have a predetermined relationship with the user are other users who viewed the user output images distributed by the user. In this case, when a user and another user who have a predetermined relationship engage in communication as shown in Figure 1, a predetermined communication group composed of the user and the other user may be temporarily created by the user's or the other user's terminal device or server device.

[0043] The description of Figure 1 above is merely for the purpose of deepening understanding of the present invention. The present invention is specifically implemented in the embodiments described below and may be implemented by various modifications without substantially exceeding the principles of the present invention. All such modifications are included within the scope of the present invention and the disclosure herein.

[0044] (Information System 1) Figure 2 shows an example of the schematic configuration of information system 1. Information system 1 comprises terminal devices 2 operated by multiple users and a server device 3. Terminal devices 2 and server device 3 are interconnected via a communication network such as a base station 4, a mobile communication network 5, a gateway 6, and the Internet 7. Communication between terminal devices 2 and server device 3 is conducted based on a communication protocol such as the Hypertext Transfer Protocol (HTTP). Alternatively, a connection may be initially established between terminal devices 2 and server device 3 via HTTP communication, and thereafter, communication may be conducted based on WebSocket, which provides bidirectional communication at a lower cost (lower communication load and processing load) than HTTP communication. Note that the communication method between terminal devices 2 and server device 3 is not limited to the above method; any communication technology can be used between terminal devices 2 and server device 3 as long as it can realize this embodiment.

[0045] Terminal device 2 is an information processing device such as a smartphone. Terminal device 2 may also be a mobile phone, notebook PC, tablet device, tablet PC, head-mounted display (HMD), e-book reader, or wearable computer. Terminal device 2 may also be a portable game console or game console. Terminal device 2 may be any information processing device that is capable of displaying and outputting the user's and other users' character objects, as well as outputting the user's and other users' voices.

[0046] In the example shown in Figure 2, one server device 3 is depicted as a component of the information system 1, but the server device 3 may be a collection of multiple physically separate server devices 3. In this case, each of the multiple server devices 3 may have the same function, or they may have the functions of a single server device 3 distributed amongst themselves.

[0047] (Terminal device 2) Figure 3 shows an example of the schematic configuration of terminal device 2. Terminal device 2 connects to server device 3 via a communication network such as a base station 4, a mobile communication network 5, a gateway 6, and the Internet 7, and communicates with server device 3. Terminal device 2 generates character video data including motion data according to various data (image data, etc.) input by the user, and transmits output information including the generated character video data and / or the user's voice data to server device 3. In addition, terminal device 2 receives output information from other users transmitted from server device 3, displays the other user's character object and / or outputs the other user's voice based on the received output information. To realize these functions, terminal device 2 is equipped with a terminal communication I / F 21, a terminal storage device 22, a display device 23, an input device 24, an imaging device 25, a microphone 26, and a terminal processing device 27.

[0048] The terminal communication interface 21 is implemented as hardware, firmware, or communication software such as a TCP / IP (Transmission Control Protocol / Internet Protocol) driver or a PPP (Point-to-Point Protocol) driver, or a combination thereof. Terminal device 2 can send data to and receive data from other devices, such as server device 3, via the terminal communication interface 21.

[0049] The terminal storage device 22 is, for example, a semiconductor memory device such as ROM (Read Only Memory) or RAM (Random Access Memory). The terminal storage device 22 stores operating system programs, driver programs, control programs, and data used in processing by the terminal processing device 27. The driver programs stored in the terminal storage device 22 include output device driver programs that control the display device 23 and input device driver programs that control the input device 24. The control programs stored in the terminal storage device 22 are, for example, application programs that realize various functions related to communication services. The control programs may also be transmitted from the server device 3 or other devices.

[0050] The terminal storage device 22 stores, as data, identification information (e.g., user ID (identification)) for uniquely identifying the user operating the terminal device 2. The terminal storage device 22 also stores, as data, background data and model data. Furthermore, the terminal storage device 22 stores, as data, a user table T1, an object table T2, and a group table T3. The terminal storage device 22 may also temporarily store data related to predetermined processing.

[0051] Background data is asset data used to construct a virtual space in a user-generated image where the user's character objects reside. Background data includes data for rendering the background of the virtual space, data for rendering various objects included in the user-generated image, and data for rendering other background objects displayed in the user-generated image. Background data may also include object position information indicating the locations of various background objects in the virtual space.

[0052] The display device 23 is a liquid crystal display. Alternatively, the display device 23 may be an organic EL (Electro-Luminescence) display or the like. The display device 23 displays moving images and / or still images corresponding to moving image data supplied from the terminal processing device 27 on its display screen. Furthermore, the display device 23 does not have to be a component of the terminal device 2; in this case, the display device 23 may be a display for an HMD capable of communicating with the server device 3, a projection mapping device capable of communicating with the terminal device 2 via wired or wireless connection, a retinal projection type projection device, or the like.

[0053] The input device 24 is a pointing device such as a touch panel. If the input device 24 is a touch panel, it can detect various touch operations by the user, such as taps, double taps, and drags. The touch panel may also be equipped with a capacitive proximity sensor and configured to detect non-contact operations by the user. The input device 24 may also be an input key or the like. The user can use the input device 24 to input characters, numbers, symbols, or positions on the display screen of the display device 23. When the input device 24 is operated by the user, it generates a signal corresponding to that operation. The input device 24 then supplies the generated signal to the terminal processing device 27 as a user instruction.

[0054] The imaging device 25 is a camera having an imaging optical system, an image sensor, and an image processing unit. The imaging optical system is, for example, an optical lens, which forms an image of the light beam from the subject onto the imaging surface of the image sensor. The image sensor is a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor), etc., and outputs an image of the subject image formed on the imaging surface. The image processing unit creates moving image data in a predetermined file format from the images continuously generated by the image sensor at predetermined intervals and outputs it as imaging data. Alternatively, the image processing unit creates still image data in a predetermined file format from the images generated by the image sensor and outputs it as imaging data.

[0055] Microphone 26 is a sound collection device configured to acquire voices emitted by the user and convert them into audio data. Microphone 26 is configured to acquire the user's voice input, converts the user's voice input acquired by Microphone 26 into audio data, and outputs the audio data to the terminal processing device 27.

[0056] The terminal processing unit 27 is a processor that loads operating system programs, driver programs, and control programs stored in the terminal storage device 22 into memory and executes instructions contained in the loaded programs. The terminal processing unit 27 is, for example, an electronic circuit such as a CPU (Central Processing Unit), MPU (Micro Processing Unit), DSP (Digital Signal Processor), GPU (Graphics Processing Unit), or a combination of various electronic circuits. The terminal processing unit 27 may also be implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit), PLD (Programmable Logic Device), FPGA (Field Programmable Gate Array), or MCU (Micro Controller Unit). In Figure 3, the terminal processing unit 27 is shown as a single component, but the terminal processing unit 27 may be a collection of multiple physically separate processors.

[0057] The terminal processing unit 27 functions as a generation unit 271, a transmission unit 272, a reception unit 273, a display processing unit 274, and an audio output unit 275 by executing various instructions included in the control program. The functions of the generation unit 271, the transmission unit 272, the reception unit 273, the display processing unit 274, and the audio output unit 275 will be described later.

[0058] (Various Tables) Figure 4 shows an example of the data structure of user table T1, object table T2, and group table T3 stored in terminal storage device 22. Note that at least one of user table T1, object table T2, and group table T3 may be stored in server storage device 32 of server device 3. In this case, terminal device 2 may receive the latest table stored in server storage device 32 at a predetermined timing and store it in terminal storage device 22.

[0059] (User Table T1) Figure 4(a) shows an example of the data structure of user table T1, which manages users participating in a communication service. In user table T1, for each user, the user ID, user name, user character object, user owned objects, and used objects are stored in relation to each other. In user table T1, the user's owned coins (information indicating the total amount of virtual currency "coins" owned by the user) and / or user points may be stored in relation to the user's user ID. The user ID is an example of identification data to uniquely identify each user. The name is an example of data indicating the name of each user.

[0060] For example, a character object is a model ID used to identify model data for generating animations of the character object. The model data is stored in the terminal storage device 22 in association with the model ID. The model data may be 3D model data for generating 3D animations, or 2D model data for generating 2D model data. The model data may include, for example, rig data (so-called "skeleton data") that shows the skeleton of the character object's face and parts other than the face, and surface data that shows the shape and texture of the character object's surface. The model data may include multiple different model data sets. Each of the multiple model data sets may have different rig data, or may have the same rig data. Each of the multiple model data sets may have different surface data, or may have the same surface data.

[0061] A possessed object is an object ID that indicates the object each user possesses. Possessed objects include attachable objects that can be associated with specific parts of a character object, and wallpaper objects that are placed behind the user's character object in the user output image. Details of attachable objects and wallpaper objects will be described later. Possessed objects are objects selected by a lottery game executed by the user's terminal device 2 or server device 3 through user operation. Possessed objects may also be objects that the user comes to possess by spending virtual currency "coins" in the purchase function of information system 1.

[0062] The object used is the object ID that indicates the object being used in the user output image displayed on the user's terminal device 2. For example, if a user modification instruction is entered during a communication service, the user's character object included in the displayed user output image will be fitted with an attachment object owned by the user. In this case, the object ID of the attachment object fitted to the user's character object is stored in user table T1 as the object used, associated with the user ID. Also, for example, if a user modification instruction is entered during a communication service, a wallpaper object owned by the user will be placed behind the user's character object in the displayed user output image. In this case, the object ID of the wallpaper object placed in the user output image is stored in user table T1 as the object used, associated with the user ID.

[0063] (Object Table T2) Figure 4(b) shows an example of the data structure of object table T2, which manages objects selected as additional objects. Additional objects are, for example, gift objects that a user sends to another user. In object table T2, for each additional object, the object ID, name, image information, rarity, and location are stored in relation to each other.

[0064] The object ID is an example of identification data used to uniquely identify each additional object. The image information consists of one or more still images corresponding to each additional object. The image information may also consist of one or more moving images corresponding to each additional object. The rarity is information indicating the rarity of each additional object. The placement location is information indicating where the image information of each additional object is displayed. If the display position and display range of a character object are defined, information indicating the relative position to that character object may be stored as the placement location.

[0065] Additional objects are classified into multiple types (categories). For example, additional objects include effect objects used as effect gifts, normal objects used as normal gifts, equipable objects used as equipable gifts, and message objects used as message gifts. Additional objects may also include wallpaper objects placed behind character objects in user-output images. Information indicating the type (category) of an additional object may be stored in object table T2 in association with the object ID of the additional object. The image information and placement location will be described below according to the type of additional object.

[0066] An effect object is an object that influences the overall impression of a user-generated image and / or other user-generated images. An effect object might be, for example, an object that mimics confetti. If the effect object is a confetti-like object, then multiple images representing different pieces of paper are stored as image information.

[0067] Information indicating the "space" where the effect object will be placed is stored. For example, an object resembling confetti will be displayed across the entire user output image and / or other user output images. The effect object may also be displayed across the entire screen (the communication screen described later) including the user output image and other user output images, according to the user's instructions. Alternatively, the effect object may be displayed only within the output image selected by the user from among the user output image and other user output images. Furthermore, the effect object may be displayed in a way that overlaps with character objects included in the user output image and / or other user output images. Unlike the attachment objects described later, the effect object is displayed without being associated with a specific part of the character. In this way, the terminal device 2 of the information system 1 can change the overall impression of the user output image and / or other user output images compared to before the effect object was displayed by displaying the effect object according to the user's instructions.

[0068] A typical object is, for example, an object that imitates a stuffed animal, a bouquet of flowers, an accessory, or an item suitable for a gift or present. Information indicating the "space" where a typical object is placed is stored. For example, a typical object is associated with information about a predetermined movement route, and the typical object is displayed moving along the predetermined movement route within the user-output image or another user-output image. The information indicating the "space" where a "typical object" is placed may be classified into multiple types. Multiple types of "space" include, for example, "central space" (the central area of ​​the image when the user-output image or another user-output image is divided into three horizontally), "left space" (the left-side area of ​​the image when the user-output image or another user-output image is divided into three horizontally), and "right space" (the right-side area of ​​the image when the user-output image or another user-output image is divided into three horizontally). Furthermore, the multiple types of "space" may include "central space" (the central area of ​​the image when the user-output image or another user-output image is divided vertically into three parts), "upper space" (the upper area of ​​the image when the user-output image or another user-output image is divided vertically into three parts), and "lower space" (the lower area of ​​the image when the user-output image or another user-output image is divided vertically into three parts). In this case, the movement route associated with a normal object is set within the type of "space" associated with the placement location of that normal object. For example, if information indicating "right space" is stored in a normal object, information regarding a movement route that moves within the right-hand area of ​​the image when the user-output image or another user-output image is divided horizontally into three parts will be associated with that normal object. In addition, normal objects may be displayed that move according to predetermined movement rules from a position set for each normal object in the user-output image or another user-output image, or from a position automatically determined randomly. The predetermined movement rules are, for example, movement rules set so that the normal object moves as if it were in free fall. Note that normal objects may be displayed overlapping with character objects. In this case, unlike the equipable objects described later, the regular object is displayed without being associated with a specific part of the character object.Additionally, regular objects may be displayed to bounce off character objects when they come into contact with them.

[0069] When a regular object is displayed overlapping with a character object, for example, the regular object may be displayed overlapping with the "parts of the character object other than the 'head,' including the 'face.'" In this case, the regular object will not be displayed overlapping with the "head" of the character object. Also, the regular object may be displayed overlapping with the "parts of the character object other than the 'upper body,' including the 'face.'" In this case, the regular object will not be displayed overlapping with the "upper body" of the character object.

[0070] An attached object is an object that is associated with a specific part (attachment location) of a character object and displayed in a user-generated image or another user-generated image. For example, specific locations on a character object's head include the front left, front right, back left, back right, center front, center back, left eye, right eye, left ear, right ear, and the entire hair. Alternatively, specific locations may include the thumb, ring finger, wrist, elbow, shoulder, upper arm, entire hand, or entire arm.

[0071] An attachment object associated with a specific part of a character object is displayed in the user output image or another user output image so as to be in contact with that specific part of the character object. Alternatively, an attachment object associated with a specific part of a character object may be displayed in the user output image or another user output image so as to cover part or all of that specific part of the character object. The specific part may be identified by 3D position information indicating its location in 3D coordinate space, or it may be associated with position information in 3D coordinate space.

[0072] The image information for attachable objects includes, for example, images of accessories (headbands, necklaces, or earrings, etc.), clothing (T-shirts, dresses, etc.), costumes, and other items that can be attached to character objects.

[0073] The system stores information indicating the attachment location of an attachment object, showing which part of the character object the attachment object is associated with. For example, if the attachment object is a "headband," the system stores information indicating the "head" of the character object as the attachment location. Similarly, if the attachment object is a "T-shirt," the system stores information indicating the "torso" of the character object as the attachment location.

[0074] The system may store information indicating multiple attachment points in 3D coordinate space as the placement location for an attachment object. For example, if the attachment object is a "headband," the system may store information indicating two locations on the character object: the "rear left side of the head" and the "rear right side of the head." This ensures that the attachment object, which resembles a "headband," is displayed as being attached to both the "rear left side of the head" and the "rear right side of the head" of the character object.

[0075] When multiple types of attachments that share the same attachment location are attached to a character object, these attachments are attached to the character object at different times. As a result, multiple types of attachments that share the same attachment location cannot be attached to the character object simultaneously. For example, if the attachment location for both an attachment representing a "headband" and an attachment representing a "hat" is stored as "head," then the attachments representing the "headband" and the "hat" will not be displayed on the character object's head at the same time.

[0076] In object table T2, the display time corresponding to the type of added object may be stored in association with the object ID of that added object. For example, the display time of an attached object may be longer than the display time of an effect object and the display time of a normal object. For example, if "60 seconds" is stored as the display time of an attached object, "5 seconds" may be stored as the display time of an effect object and "10 seconds" as the display time of a normal object.

[0077] (Group Table T3) Figure 4(c) shows an example of the data structure of a group table T3 that manages the groups to which a user belongs (groups of users who are friends, groups with other users who are mutual followers, groups arbitrarily created by the user or other users, etc.). A group table T3 is provided individually for each user, and the group table T3 shown in Figure 4(c) is the group table T3 of the user operating the terminal device 2 that stores the group table T3.

[0078] In group table T3, each group stores its group ID, group name, and the users belonging to that group, all of which are related to each other. The group ID is an example of identification data used to uniquely identify each group. The name is an example of data indicating the name of each group. The users belonging to the group are the user IDs of each user belonging to the group.

[0079] Returning to Figure 3, the functions of the generation unit 271, transmission unit 272, reception unit 273, display processing unit 274, and audio output unit 275 will be explained.

[0080] (Generation part 271) The generation unit 271 acquires imaging data continuously output by the imaging device 25 from the imaging device 25. For example, the imaging device 25 of the terminal device 2, which is held by the user, is installed on the terminal device 2 so that the user's face, which is looking at the display screen of the display device 23, is in the direction of the imaging. The imaging device 25 continuously images the user's face in the direction of the imaging, acquires imaging data of the user's face, and outputs it to the generation unit 271 of the terminal processing device 27. The imaging device 25 may also image parts other than the user's face, such as the head, arms, hands (including fingers), chest, torso, legs, or other parts, and acquire imaging data. The imaging device 25 may also be a 3D camera capable of detecting the depth of a person's face.

[0081] The generation unit 271 generates face motion data, which is a digital representation of the user's facial movements, as time progresses, based on continuously acquired imaging data. The face motion data may be generated at predetermined sampling time intervals. In this way, the face motion data generated by the generation unit 271 can digitally represent the user's facial movements (changes in facial expressions) in a time series.

[0082] The generation unit 271 may generate body motion data, which is a digital representation of the position and orientation of each part of the user (head, arms, hands (including fingers), chest, torso, legs, or other parts), together with or separately from the face motion data.

[0083] Body motion data may be generated based on detection information from a known motion sensor attached to the user. In this case, the terminal communication I / F 21 of the terminal device 2 has a predetermined communication circuit for acquiring detection information from the motion sensor attached to the user via wireless communication. The generation unit 271 then generates body motion data based on the detection information acquired by the terminal communication I / F 21. The body motion data may be generated at predetermined sampling time intervals. In this way, the body motion data generated by the generation unit 271 can digitally represent the user's body movements in a time series.

[0084] The generation of body motion data based on detection information from motion sensors attached to the user may be performed, for example, in a shooting studio. In this case, the shooting studio may be equipped with a base station, a tracking sensor, and a display. The base station is, for example, a multi-axis laser emitter. The motion sensor attached to the user may be, for example, a Vive Tracker provided by HTC CORPORATION, and the base station may also be, for example, a base station provided by HTC CORPORATION.

[0085] Furthermore, a supporter computer may be installed in a separate room adjacent to the shooting studio. The display in the shooting studio may be configured to display information received from the supporter computer. Server device 3 may be installed in the same room as the supporter computer. The room where the supporter computer is installed and the shooting studio may be separated by a glass window. In this case, the operator of the supporter computer can see the user. The supporter computer may be configured to change the settings of various devices provided in the shooting studio in response to the supporter's operation. For example, the supporter computer can change the scanning interval by the base station, the tracking sensor settings, and various settings of other devices. The operator may input a message into the supporter computer, and the input message may be displayed on the display in the shooting studio.

[0086] The generation unit 271 generates character video data including face motion data and / or body motion data generated as needed, and outputs the generated character video data to the display processing unit 274. Hereinafter, face motion data and body motion data may be collectively referred to as motion data. The generation unit 271 also generates output information including the generated character video data and the user ID stored in the terminal storage device 22, and outputs the generated output information to the transmission unit 272. If the generation unit 271 acquires user voice data output from the microphone 26, it generates output information including the generated character video data, the acquired voice data, and the user ID stored in the terminal storage device 22, and outputs the generated output information to the transmission unit 272.

[0087] The generation unit 271 generates game image data showing the game screen of the game being played by the user as the game progresses, generates output information including the generated game image data and the user ID stored in the terminal storage device 22, and outputs the generated output information to the transmission unit 272. The output information may also include generated character video data and / or acquired audio data along with the generated game image data.

[0088] (Transmitter 272) The transmitting unit 272 transmits the output information output from the generating unit 271, along with destination information, to the server device 3 via the terminal communication interface 21. The destination information includes the user IDs of one or more other users participating in the communication service with the user. Alternatively, the user IDs included in the output information may be used as destination information. For example, the server device 3 stores the user IDs of one or more other users participating in the communication service with the user. Upon acquiring the output information, it identifies the user IDs of the one or more other users participating in the communication service with the user that are included in the output information, and transmits the received output information to the terminal device 2 of the other user indicated by the identified user ID.

[0089] Furthermore, the transmission unit 272 transmits information indicating various instructions input by the user through the operation of the input device 24 to the server device 3 via the terminal communication interface 21, along with the user ID and destination information stored in the terminal storage device 22. These user instructions include, for example, additional instructions. Additional instructions are transmitted to the server device 3 via the terminal communication interface 21, along with the object ID and user ID of the additional object to be included in the user's output image.

[0090] (Receiving unit 273) The receiving unit 273 receives information such as output information and various instructions from other users transmitted from the server device 3 via the terminal communication interface 21. The receiving unit 273 outputs the received information to the terminal processing device 27. The output information from other users transmitted from the server device 3 is generated by the generation unit 271 of the other user's terminal device 2 and transmitted to the server device 3 by the transmission unit 272 of the other user's terminal device 2. The output information from other users includes the other user's character video data, the other user's voice data, and the other user's user ID, etc. The output information from other users may not include the other user's voice data, and the output information from other users may not include the other user's character video data.

[0091] (Display processing unit 274) The display processing unit 274 displays the image drawn based on the motion data (character video data) generated by the generation unit 271 on the display device 23 as the user output image. The display processing unit 274 also displays the image drawn based on the motion data (character video data) included in the output information of other users received by the receiving unit 273 on the display device 23 as the other user output image. The image drawing process based on motion data will be described below. In the image drawing process based on the motion data of the user (the user operating the terminal device 2), the user ID stored in the terminal storage device 22 is used, and in the image drawing process based on the motion data included in the output information of other users received by the receiving unit 273, the user ID included in the output information is used.

[0092] First, the display processing unit 274 extracts the model ID of the character object associated with the user ID and the object ID of the object being used from the user table T1. Next, the display processing unit 274 reads the model data associated with the extracted model ID and the image information and placement location associated with the extracted object ID of the object being used from the terminal storage device 22. Next, the display processing unit 274 generates an animation of the character object with the object being used attached to the placement location based on the read model data, the image information and placement location of the object being used, and the motion data. If the motion data includes face motion data, the display processing unit 274 generates an animation of the character object so that the character object's facial expression changes based on the face motion data. In this way, the display processing unit 274 can generate an animation of the character object that moves in sync with the user's facial expression movements based on the read model data and face motion data. If the motion data includes face motion data and body motion data, the display processing unit 274 generates an animation of the character object that moves in sync with the user's facial expression and body movements based on the read model data, face motion data, and body motion data. Next, the display processing unit 274 draws an image that includes the animation (moving image) of the generated character object and a background image that shows the background generated using background data stored in the terminal storage device 22. The display processing unit 274 may also use a wallpaper object that has been pre-associated with the communication group and stored in the terminal storage device 22 as the background image. In this case, the same wallpaper object may be applied as the background image for all members of the communication group. Then, the display processing unit 274 outputs drawing data indicating the drawn image and terminates the image drawing process based on the motion data. The image drawn based on the character object associated with the user ID stored in the terminal storage device 22 is the user's user output image.Furthermore, images drawn based on character objects associated with user IDs contained in the output information of other users received are considered to be the output images of those other users.

[0093] Furthermore, the display processing unit 274 may, in response to a user's change instruction, display on the display device 23 a character object newly equipped with an attachment object owned by the user as an object owned by the user. For example, if the attachment object that the user has instructed to be changed is a "headband", the display processing unit 274 extracts the object ID of the attachment object ("headband") from the owned objects associated with the user's user ID in the user table T1 and stores it as a used object. The display processing unit 274 then extracts the image information and placement location associated with the object ID stored as a used object from the object table T2. Next, the display processing unit 274 generates a user output image that includes a character object with the attachment object ("headband") attached to its head (the placement location associated with the object ID of the "headband") based on the extracted image information and placement location. The display processing unit 274 may also generate a user output image that includes a character object equipped with an attachment object that moves in conjunction with the movement of the attachment part (placement location) of the character object.

[0094] Furthermore, the display processing unit 274 displays the user output image including the additional object in response to an additional instruction from another user to add an additional object. The display processing unit 274 displays the other user output image including the additional object in response to an additional instruction from the user to display the additional object in the other user output image.

[0095] If the object ID included in the instruction to add an additional object from another user is the object ID of any effect object, the display processing unit 274 refers to the object table T2 and extracts a still image or video (image information) of the effect object associated with the object ID included in the instruction from the terminal storage device 22, and generates a user output image including the extracted still image or video. For example, if an instruction to add an effect object showing confetti or fireworks is made, the display processing unit 274 generates a user output image including a video of an effect object (not shown) that imitates confetti or fireworks. Similarly, if the object ID included in the instruction to add an additional object from a user is the object ID of any effect object, the display processing unit 274 refers to the object table T2 and extracts a still image or video (image information) of the effect object associated with the object ID included in the instruction from the terminal storage device 22, and generates a user output image including the extracted still image or video.

[0096] If the object ID included in the instruction to add an additional object from another user is the object ID of a normal object, the display processing unit 274 refers to object table T2 and extracts the still image or moving image (image information) and placement location of the normal object associated with the object ID of the normal object. Next, the display processing unit 274 generates a user output image that includes the still image or moving image of the normal object placed at the location indicated by the placement location. Similarly, if the object ID included in the instruction to add an additional object from a user is the object ID of a normal object, the display processing unit 274 refers to object table T2 and extracts the still image or moving image (image information) and placement location of the normal object associated with the object ID of the normal object, and generates a user output image that includes the still image or moving image of the normal object placed at the location indicated by the placement location.

[0097] The display processing unit 274 may generate user output images and other user output images that include moving images of normal objects moving within the display area of ​​the user output image and other user output images. Hereinafter, user output images and other user output images may be collectively referred to as output images. For example, the display processing unit 274 may generate an output image that includes moving images of normal objects falling from the top to the bottom of the output image. In this case, the normal object may be displayed within the display area of ​​the output image from the start of the fall until it falls to the bottom edge of the output image, and may disappear from the output image after falling to the bottom edge of the output image. The movement route of the normal object may be from left to right, from right to left, from top left to bottom left, or in any other direction of the output image, and may also be a route along a straight line, circular orbit, elliptical orbit, spiral orbit, or any other trajectory.

[0098] The display processing unit 274 displays notification information regarding the attachment instruction within the user output image if the object ID included in the instruction to add an additional object from another user is the object ID of the attachment object. In response to the attachment instruction from the user, the display processing unit 274 refers to the object table T2 and extracts the image information and placement location of the attachment object associated with the object ID included in the instruction. Next, based on the extracted image information and placement location, the display processing unit 274 generates a user output image that includes the character object attached to the attachment area indicated by the placement location.

[0099] Furthermore, the display processing unit 274 starts the game to be played by the user in response to the user's instructions and displays the game screen of that game on the display device 23. Once the game has started, the display processing unit 274 displays the game screen of the game progressing according to the user's game operation instructions on the display device 23, and when the game termination conditions are met, it terminates the display of the game screen. The game termination conditions are predetermined according to the rules of the game, and may include, for example, the user's game medium (character, etc.) winning or losing against an opponent, the user's game medium reaching a predetermined destination, or a game object operated by the user reaching a predetermined state (for example, multiple pieces (game objects) being lined up on the squares of the game field).

[0100] Furthermore, if the output information of another user transmitted from the server device 3 includes game image data of the game that the other user is playing, the display processing unit 274 displays the game image of the game that the other user is playing on the display device 23. Note that when the game image of the game that the other user is playing is displayed, the other user's output image is not displayed, however, the game image of the game that the other user is playing and the other user's output image may be displayed simultaneously.

[0101] (Audio output unit 275) The audio output unit 275, when another user's output information includes another user's audio data, outputs the other user's voice generated based on the audio data from a speaker (not shown) along with the display processing unit 274 displaying the other user's output image. As a result, the other user's voice is output while the lips of the other user's character object included in the other user's output image are moving, allowing the user to see the other user's output image, which includes the other user's character object appearing to be emitting sound.

[0102] (Server device 3) Figure 5 shows an example of the schematic configuration of the server device 3. The server device 3, which provides communication services to each terminal device 2 of multiple users, transmits output information and information indicating various instructions sent from one user's terminal device 2 to terminal devices 2 of other users who are different from that user. To this end, the server device 3 is equipped with a server communication I / F 31, a server storage device 32, and a server processing device 33. For example, the terminal device 2 to which information is sent from one user's terminal device 2 is the terminal device 2 of the user indicated by the user ID included in the received destination information.

[0103] The server communication interface 31 is implemented as hardware, firmware, or communication software such as a TCP / IP driver or a PPP driver, or a combination thereof. Server device 3 can send information to and receive information from other devices via the server communication interface 31.

[0104] The server storage device 32 is, for example, a semiconductor memory device such as ROM or RAM. The server storage device 32 may also be, for example, a magnetic disk, an optical disk, or any other storage device capable of storing data. The server storage device 32 stores the operating system program, driver program, application program, and data used in the processing of the server processing device 33. The server storage device 32 may also store user table T1, object table T2, and group table T3 as data.

[0105] The server processing unit 33 is a processor that loads the operating system program, driver program, and control program stored in the server storage device 32 into memory and executes the instructions contained in the loaded program. The server processing unit 33 is, for example, an electronic circuit such as a CPU, MPU, DSP, GPU, or a combination of various electronic circuits. The server processing unit 33 may also be implemented by an integrated circuit such as an ASIC, PLD, FPGA, or MCU. In Figure 5, the server processing unit 33 is shown as a single component, but the server processing unit 33 may be a collection of multiple physically separate processors. The server processing unit 33 functions as a server receiving unit 331 and a server transmitting unit 332 by executing various instructions contained in the control program.

[0106] (Server receiving unit 331) The server receiving unit 331 receives output information and various instruction information transmitted from the terminal device 2 of any of the multiple users via the server communication interface 31. The server receiving unit 331 may also receive destination information along with the output information and instruction information transmitted from the terminal device 2.

[0107] (Server transmission unit 332) The server transmission unit 332 transmits the output information and various instruction information received by the server reception unit 331 to the terminal device 2 of another user indicated by the user ID included in the simultaneously received destination information via the server communication interface 31. The server transmission unit 332 may also transmit the various instruction information received by the server reception unit 331 to the terminal device 2 of one or more other users specified by the user who transmitted the information, via the server communication interface 31.

[0108] (Examples of various screens) The following describes examples of various screens displayed on the display device 23 of the user's terminal device 2, with reference to Figures 6 to 15. In Figures 6 to 15, multiple components with the same reference numerals are components that have equivalent functions.

[0109] Figure 6(a) shows an example of a group creation screen 600 displayed on the display device 23 of the user's terminal device 2. The group creation screen 600 includes an other user display area 601, a selection object 602, and a create button 603. For example, the group creation screen 600 is displayed when a group creation object, etc., on the home screen (not shown), which is displayed in response to the execution of a control program stored in the terminal storage device 22, is selected by the user's input device 24.

[0110] The other user display area 601 displays other user information for each other user who has a predetermined relationship with the user. In the example shown in Figure 6(a), the display processing unit 274 of the user's terminal device 2 displays the thumbnail image of the other user's character object and the other user's name as other user information.

[0111] The selection object 602 is an operation object for selecting each other user indicated by each other user information. In the example shown in Figure 6(a), the display processing unit 274 of the user's terminal device 2 displays checkbox objects corresponding to each other user information as selection objects 602. For example, when a selection object 602 is selected by the user's input device 24, the selection object 602 will be displayed with a "check" mark. In this case, the other user in the other user information corresponding to the selection object 602 that is displayed with a "check" mark is selected. Also, when a selection object 602 that is displayed with a "check" mark is selected by the user's input device 24, the selection object 602 will be displayed without a "check" mark. In this case, the other user in the other user information corresponding to the selection object 602 that is displayed without a "check" mark is not selected. Each selection object 602 is associated with the user ID of the other user related to the corresponding other user information.

[0112] The Create button 603 is a button object, etc., for a user to create a new communication group. When the Create button 603 is selected by the user operating the input device 24, a new group is created with the user operating the terminal device 2 and other users selected in the selection object 602 as its members. For example, the display processing unit 274 of the user's terminal device 2 identifies the user ID of the user stored in the terminal storage device 22 and the user ID of the other user associated with the selection object 602 related to the other user selected in the selection object 602. The display processing unit 274 then associates the identified user ID and the user ID of the other user as members of the newly created group ID and stores it in the group table T3. At this time, the display processing unit 274 may automatically associate the name of the created group with the newly created group ID, or it may associate the name of the group entered by the user.

[0113] When a new group is created in response to the selection of the create button 603, the display processing unit 274 displays an information exchange screen 700 for the users belonging to the new group. Figure 7(a) shows an example of the information exchange screen 700 displayed on the display device 23 of the user's terminal device 2. The information exchange screen 700 includes a display area 701 that displays information such as text or images entered by each user belonging to the new group, information (such as a thumbnail image of a character object) 702 that indicates the user who entered the information displayed in the display area 701, an input object 703 for the user to enter text or images to be displayed in the new display area 701, and a start object 704 for starting the communication service.

[0114] When the information exchange screen 700 is displayed, the transmission unit 272 of the user's terminal device 2 transmits a display instruction for the information exchange screen, along with information about the new group and destination information, to the server device 3 via the terminal communication interface 21. The information about the new group includes the group ID, name, and user IDs of each member of the new group. The destination information includes the user IDs of all other members of the new group, excluding the user operating the terminal device 2. The server receiving unit 331 of the server device 3 receives the display instruction for the information exchange screen, the information about the new group, and the destination information via the server communication interface 31. The server receiving unit 331 may also store the received destination information in the server storage device 32. The server transmission unit 332 of the server device 3 transmits the display instruction for the information exchange screen and the information about the new group to the terminal device 2 of the other user indicated by the user ID included in the destination information, via the server communication interface 31. When another user's terminal device 2 receives a display instruction for the information exchange screen sent from the user's terminal device 2 via the server device 3, it can display an information exchange screen 700 that allows users belonging to the new group to exchange information.

[0115] Let's return to the explanation in Figure 7(a). For example, when new information is entered into the input object 703 by an operation of any affiliated user, a display area 701 for displaying the new information is added to the information exchange screen 700 displayed on the terminal device 2 of the user who entered the information. In addition, the transmission unit 272 of the terminal device 2 of the user who entered the information transmits the newly entered information to the terminal devices 2 of other affiliated users via the server device 3. As a result, a display area 701 for displaying the new information is also added to the information exchange screen 700 displayed on the terminal devices 2 of other affiliated users who are not the user who entered the information.

[0116] The start object 704 is a button object or the like for initiating a communication service that each user belonging to a new group can participate in. When the user selects the start object 704 by operating the input device 24, the communication service initiation process is executed, which designates the group that can exchange information via the information exchange screen 700 as the communication group. An example of the initiation process in response to the user's selection of the start object 704 is described below.

[0117] First, the display processing unit 274 of the user's terminal device 2 displays the communication screen 810 (Figure 8(b)) that can be displayed by each user's terminal device 2 belonging to the communication group. The display processing unit 274 also stores the group ID of the communication group in the terminal storage device 22 as the group ID of the group that started the communication service. Furthermore, the display processing unit 274 stores the user's user ID in the terminal storage device 22 as the user ID of the user participating in the communication service. If all other users belonging to the communication group have not yet joined the communication service, only the user's user output information is displayed on the communication screen 810.

[0118] Next, the transmission unit 272 of the user's terminal device 2 transmits a communication service start instruction to the server device 3 via the terminal communication interface 21, along with the user's user ID, communication group information, and / or destination information stored in the terminal storage device 22. The communication group information includes the group ID, name, and user IDs of each member of the communication group. The destination information includes the user IDs of all other users belonging to the communication group, excluding the user operating the terminal device 2. Note that if destination information is stored in the server storage device 32, the destination information does not need to be transmitted. The server receiving unit 331 of the server device 3 receives the start instruction, user ID, communication group information, and destination information via the server communication interface 31. The server transmission unit 332 of the server device 3 transmits the start instruction, user ID, and communication group information to the terminal device 2 of the other user indicated by the user ID included in the destination information via the server communication interface 31.

[0119] The receiving unit 237 of the other user's terminal device 2 receives information about the start instruction, user ID, and communication group transmitted from the server device 3 to the user's terminal device 2. The receiving unit 237 stores the group ID from the communication group information in the terminal storage device 22 as the group ID of the group that started the communication service. Furthermore, the display processing unit 274 stores the transmitted user ID in the terminal storage device 22 as the user ID of the user participating in the communication service. The display processing unit 274 of the other user's terminal device 2 displays a notification screen (not shown) based on the received information about the communication group. The display processing unit 274 displays a group selection screen 710 (Figure 7(b)) for joining the communication service on the display device 23 according to the user's selection of a predetermined object on the notification screen. This completes the start process. The display processing unit 274 of the other user's terminal device 2 may also include a join button 805, described later, on the notification screen. This allows other users to immediately join the communication service without displaying the group selection screen 710.

[0120] The start process may be executed when the user selects the create button 603 on the group creation screen 600 by operating the input device 24. That is, in response to the user's selection of the create button 603, the start process for the communication service, which designates the newly created group on the group creation screen 600 as a communication group, may be executed. In this case, the information exchange screen 700 is not displayed.

[0121] Furthermore, the start process is not limited to the start process of a communication service that uses a newly created group on the group creation screen 600 as the communication group. For example, the start process of a communication service that uses an already created group as the communication group may be executed. For example, Figure 6(b) shows an example of a group selection screen 610 for selecting a communication group created by any user. For example, the group selection screen 610 is displayed when a group selection object, etc., on the home screen (not shown) displayed in response to the execution of a control program stored in the terminal storage device 22 is selected by the user's input device 24.

[0122] The group selection screen 610 displayed on the user's terminal device 2 includes a group display area 611 for selecting one of several communication groups. The group display area 611 displays group information for each communication group that has already been created and to which the user belongs. In the example shown in Figure 6(b), the display processing unit 274 of the user's terminal device 2 displays the thumbnail images of each user belonging to the communication group and the name of the group as group information. In the example shown in Figure 6(b), the thumbnail images of each other user belonging to the group are displayed, but the thumbnail images of all users belonging to the group may also be displayed. If the communication group contains only one other user besides the user, the name of that one other user may be displayed instead of the group name, as shown in Figure 6(b).

[0123] When a user selects a group from the group display area 611 by operating the input device 24, an information exchange screen 700 is displayed for users belonging to the communication group corresponding to the selected group information. The information exchange screen 700 contains information such as text or images entered by each user belonging to the communication group. In this case as well, the terminal device 2 of other users belonging to the communication group receives instructions to display the information exchange screen and information about the communication group via the server device 3, and the information exchange screen 700 is also displayed on the terminal device 2 of the other users.

[0124] Then, when a user selects a start object 704 on the information exchange screen 700, the communication group that will exchange information on the information exchange screen 700 executes the process to start the communication service. When the start process is executed, the display processing unit 274 of the user's terminal device 2 displays a communication screen 810 (Figure 8(b)) that can be displayed by each user's terminal device 2 belonging to the communication group. In addition, the display processing unit 274 of another user's terminal device 2 displays a notification screen (not shown), and in response to the other user's operation on the notification screen, displays a group selection screen 710 (Figure 7(b)) for joining the communication service on the display device 23. In this case as well, the display processing unit 274 of another user's terminal device 2 may also include a join button 805, which will be described later, on the notification screen.

[0125] In this case as well, the start process may be executed when a group is selected on the group selection screen 610 by the user operating the input device 24. That is, in response to the user's selection of a group, the start process for the communication service by the communication group corresponding to the selected group may be executed. In this case, the information exchange screen 700 will not be displayed.

[0126] Next, we will explain how users who have not yet joined a communication service that has already been started can join. Figure 7(b) shows an example of a group selection screen 710 displayed on the display device 23 of the user's terminal device 2. For example, the group selection screen 710 is displayed in response to the selection of a predetermined object in the displayed notification screen. Alternatively, the group selection screen 710 may be displayed when a group selection object or the like on the home screen, which is displayed in response to the execution of a control program stored in the terminal storage device 22, is selected by the user's input device 24.

[0127] In the group display area 711 of the group selection screen 710, group information indicating a communication group that has started a communication service is displayed in a way that makes it distinguishable from other group information. In the example shown in Figure 7(b), the group information displayed at the top indicates a communication group that has started a communication service. For example, information indicating the number of users participating in the communication service is displayed superimposed on or near each thumbnail image of a user belonging to the communication group. In addition, a mark image 712 indicating a communication service is displayed near the name of the communication group.

[0128] When a user operates the input device 24 and selects group information from the group information contained in the group display area 711 that indicates a communication group that has started a communication service, the information exchange screen 800 for users belonging to the communication group corresponding to the selected group information is displayed.

[0129] Figure 8(a) shows an example of an information exchange screen 800 displayed on the display device 23 of the user's terminal device 2. The information exchange screen 800 includes display areas 801a to 801b that display information such as text or images entered by each user belonging to a communication group that has started a communication service, information (such as thumbnail images of character objects) 802a to 802b that indicates the user who entered the information displayed in each of the display areas 801a to 801b, an input object 803 for the user to enter text or images to be displayed in a new display area 801, a service display area 804 that indicates a communication service that has already started, and a join button 805 for joining a communication service that has already started.

[0130] The service display area 804 shown in Figure 8(a) displays thumbnail images of the character objects of the three users participating in the communication service. The display area 801b shown in Figure 8(a) displays information indicating that the communication service has been started. The participation process is executed when the participant button 805 is selected by the user. An example of the participation process is described below.

[0131] When the user selects the participate button 805 by operating the input device 24, the display processing unit 274 of the user's terminal device 2 first stores the user's user ID stored in the terminal storage device 22 as the user ID of the user participating in the communication service related to the service display area 804. Next, the display processing unit 274 displays the communication screen 810 (Figure 8(b)) which includes user output images of all users participating in the communication service.

[0132] Next, the transmitting unit 272 of the user's terminal device 2 transmits a communication service participation instruction to the server device 3 via the terminal communication interface 21, along with the user's user ID, communication group information, and / or destination information stored in the terminal storage device 22. The communication group information includes the group ID, name, and user IDs of each member user of the communication group. The destination information includes the user IDs of all other users belonging to the communication group, excluding the user operating the terminal device 2. The server receiving unit 331 of the server device 3 receives the participation instruction, user ID, communication group information, and / or destination information via the server communication interface 31. The server transmitting unit 332 of the server device 3 transmits the participation instruction, user ID, and communication group information to the terminal device 2 of the other user indicated by the user ID included in the destination information via the server communication interface 31.

[0133] The receiving unit 237 of the other user's terminal device 2 receives information from the server device 3 regarding the participation instruction, user ID, and communication group transmitted from the user's terminal device 2. The receiving unit 237 stores the received user ID in the terminal storage device 22 as the user ID of the user participating in the communication service. If the communication screen 810 is displayed on the terminal device, the display processing unit 274 of the other user's terminal device 2 displays the communication screen 810, including the user output images of all users participating in the communication service. With this, the participation process is completed.

[0134] The selection process may be executed when group information is selected on the group selection screen 710 by the user operating the input device 24. In other words, depending on the user's selection of group information, the process of joining a communication service that the communication group corresponding to the group information can join may be executed. In this case, the information exchange screen 800 is not displayed.

[0135] The embodiments for initiating or joining a communication service are not limited to the examples described above. For example, other initiation processes different from the initiation process in response to the user's selection of the initiation object 704 may be executed. Other initiation processes are executed when a predetermined display object for specifying the initiation of a communication service is selected by the user on a predetermined screen, such as the home screen. The predetermined display object may be displayed when a thumbnail image of the user's character object is selected, or it may be included as a menu item on a predetermined screen. Other initiation processes may be executed automatically when predetermined initiation conditions are met. For example, predetermined initiation conditions may include the current time being a predetermined time, a predetermined amount of time having elapsed since the end of the previous communication service, or the number of mutual followers of the users exceeding a predetermined number. An example of other initiation processes is described below.

[0136] First, the transmission unit 272 of the user's terminal device 2 sends a communication service start instruction to the server device 3 via the terminal communication interface 21, along with the user's user ID and / or destination information stored in the terminal storage device 22. The transmission unit 272 also stores the user's user ID in the terminal storage device 22 as the user ID of the user participating in the communication service. The destination information includes the user IDs of one or more other users who have a predetermined relationship with the user. For example, the destination information may include the user IDs of one or more other users who are mutual followers of the user. Alternatively, the destination information may include, in place of or in addition to, the user IDs of other users who are mutual followers of the user, but who are mutual followers of the predetermined user who is a mutual follower of the user.

[0137] The server receiving unit 331 of server device 3 receives a start command, user ID, and / or destination information via the server communication interface 31. The server transmitting unit 332 of server device 3 transmits the start command, user ID, and destination information via the server communication interface 31 to the terminal device 2 of another user, indicated by the user ID included in the destination information. The receiving unit 237 of the other user's terminal device 2 receives the start command, user ID, and destination information transmitted from the server device 3 to the user's terminal device 2. The receiving unit 237 stores the transmitted user ID in the terminal storage device 22 as the user ID of the user participating in the communication service. The display processing unit 274 of the other user's terminal device 2 displays a notification screen (not shown) on the display device 23 based on the received user ID. The notification screen includes information prompting the user indicated by the received user ID to participate in the communication service, as well as a participate button, etc.

[0138] When another user selects the "Join" button on the notification screen by operating the input device 24, the display processing unit 274 of the other user's terminal device 2 stores the other user's user ID stored in the terminal storage device 22 as the user ID of the user participating in the communication service related to the service display area 804. Next, the display processing unit 274 displays the communication screen 810 (Figure 8(b)) which includes user output images of all users participating in the communication service.

[0139] Next, the transmitting unit 272 of the other user's terminal device 2 transmits a communication service participation instruction to the server device 3 via the terminal communication interface 21, along with the other user's user ID and / or destination information stored in the terminal storage device 22. The destination information is obtained by removing the user ID of the other user of the terminal device 2 that is transmitting the destination information from the user ID included in the destination information previously received by the receiving unit 237, and simultaneously adding the user ID received by the receiving unit 237. The server receiving unit 331 of the server device 3 receives the participation instruction, user ID, and / or destination information via the server communication interface 31. The server transmitting unit 332 of the server device 3 transmits the participation instruction, user ID, and destination information to the other user's terminal device 2 indicated by the user ID included in the destination information via the server communication interface 31.

[0140] The receiving unit 237 of the terminal device 2 of the user to whom the participation instruction is sent receives the participation instruction, user ID, and destination information sent from the server device 3. The receiving unit 237 stores the received user ID in the terminal storage device 22 as the user ID of the user participating in the communication service. If the communication screen 810 is displayed on the terminal device 2, the display processing unit 274 of the terminal device 2 displays the communication screen 810, including the user output images of all users participating in the communication service. With this, the series of participation processes is completed.

[0141] The embodiments for initiating or participating in a communication service with other users who have a predetermined relationship with the user are not limited to the examples described above. For example, when a user selects a predetermined display object for specifying the initiation of a communication service, a selection screen containing information indicating other users who are mutual followers of one or more users may be displayed. In this case, when the user selects information indicating any of the other users, other initiation processes based on an initiation instruction to start communication with the selected other user may be executed. Furthermore, in this case, an information exchange screen with the selected other user may be displayed, and by selecting the start button on the information exchange screen, other initiation processes based on an initiation instruction to start communication with the other user may be executed.

[0142] In conventional information systems, users could not initiate and / or participate in communication with other users who had a predetermined relationship with them, such as mutual followers, using a simple interface. In the information system 1 disclosed in this embodiment, the execution of the above-mentioned other initiation processes improves the user interface related to initiating and / or participating in communication, and reduces the communication load between the server device 3 and the terminal device 2.

[0143] Figure 8(b) shows an example of a communication screen 810 displayed on the display device 23 of the terminal device 2. The communication screen 810 is a communication screen for a communication group that includes the user operating the terminal device 2 as a member user. Hereinafter, the user operating the terminal device 2 will be referred to as User A, and users other than User A who are participating in the communication service corresponding to the communication screen 810 will be referred to as User B1, User B2, User B3, ... Note that Users B1, User B2, User B3, ... may be collectively referred to as User B.

[0144] The communication screen 810 includes at least a user output image 812a that includes user A's character object 811a. In the example shown in Figure 8(b), there are three users other than user A who are participating in the communication service (users B1, B2, and B3). That is, the communication screen 810 includes user output images 812b1, 812b2, and 812b3, which include the respective character objects 811b1, 811b2, and 811b3 of users B1, B2, and B3, respectively. Hereinafter, character objects 811b1, 811b2, and 811b3 may be collectively referred to as character object 811b, and user output images 812b1, 812b2, and 812b3 may be collectively referred to as user output image 812b. Note that user B is another user to user A, and the user output image 812b of user B displayed on user A's terminal device 2 is an output image of another user.

[0145] If there are no users other than User A participating in the communication service, only User Output Image 812a is included in the communication screen 810. Each time User B joins the communication service, User B's User Output Image 812b is newly included in the communication screen 810. There is no upper limit to the number of users who can participate in the communication service. For example, if nine Users B are participating in the communication service in addition to User A, then User Output Image 812a of User A will be included in the communication screen 810 along with the respective User Output Images 812b1, 812b2, 812b3, ... 812b9 of each of the nine Users B.

[0146] The character object 811a is an animation (moving image) of user A's character object, generated by the display processing unit 274 based on the motion data of user A generated by the generation unit 271 and the character object and usage object associated with user A's user ID in user table T1. The motion data of user A generated by the generation unit 271 is the same as the motion data included in the output information transmitted to user B's terminal device 2. User A's output information includes user A's voice data along with user A's motion data. If the motion data of user A generated by the generation unit 271 includes user A's face motion data, a user output image 812a is displayed that includes a character object 811a whose facial expression changes in sync with the movement of user A's facial expression. For example, when user A is speaking, a user output image 812a is displayed that includes a character object 811a whose lips are moving. Furthermore, if the motion data of user A generated by the generation unit 271 includes body motion data of user A, a user output image 812a is displayed that includes a character object 811a whose corresponding body parts move in sync with the movements of user A's body.

[0147] The transmitting unit 272 of User A's terminal device 2 transmits User A's output information, along with destination information (the user IDs of each User B (User B1 to User B3) included in the communication group), to the server device 3 via the terminal communication interface 21. Next, the server receiving unit 331 of the server device 3 receives User A's output information transmitted from User A's terminal device 2 via the server communication interface 31. Then, the server transmitting unit 332 of the server device 3 refers to the received user IDs of User B (User B1 to User B3) and transmits User A's output information to User B's (User B1 to User B3) terminal devices 2 via the server communication interface 31. When the receiving unit 273 of User B's terminal device 2 receives the output information via the terminal communication interface 21, the display processing unit 274 of User B's terminal device 2 displays a user output image 812a that includes a character object 811a based on User A's motion data, indicated by the user ID included in the output information.

[0148] The character object 811b is an animation (moving image) of user B's character object generated by the display processing unit 274 based on the motion data included in the output information of user B received by the receiving unit 273 and the character object associated with user B's user ID in user table T1. If the motion data of user B received by the receiving unit 273 includes user B's face motion data, a user output image 812b is displayed that includes the character object 811b whose facial expression changes in sync with the movement of user B's facial expression. For example, if user B makes a sound, a user output image 812b is displayed that includes the character object 811b whose lips are moving. Also, if the motion data of user B generated by the generation unit 271 includes user B's body motion data, a user output image 812b is displayed that includes the character object 811b whose corresponding body parts move in sync with the movement of user B's body. If the output information of user B received by the receiving unit 273 includes voice data of user B, the display processing unit 274 displays the user output image 812b, and the voice output unit 275 outputs the voice of user B generated based on the voice data.

[0149] In this way, by displaying the communication screen 810, which includes user output images 812a and 812b, user A can see user B's character object 811b, which appears to be emitting sound, while listening to user B's voice. Furthermore, user A's character object and voice data are output by user B's terminal device 2. This enables user A and user B, who are participating in the communication service, to communicate via their character objects.

[0150] The communication screen 810 includes an add button 813 and a game button 814. In the example shown in Figure 8(b), the add button 813 and the game button 814 are located within the display area of ​​the user output image 812a, but they may also be displayed outside the display area of ​​the user output image 812a. Furthermore, the display control unit 274 may control the display of the add button 813 and the game button 814 so that they are displayed for a predetermined time only when predetermined display conditions are met.

[0151] The add button 813 is an object image, such as a button object, that instructs the display of the selection screen 1200 for selecting additional objects to be included in the communication screen 810. When the add button 813 is selected by user A through the operation of the input device 24, the selection screen 1200 is displayed. Details of the selection screen 1200 will be described later.

[0152] The game button 814 is an object image, such as a button object, used to instruct the start of the game. When the game button 814 is selected by user A through the operation of the input device 24, the input device 24 inputs a game start instruction to the display processing unit 274. When the terminal processing unit 27 receives the game start instruction from the input device 24, it executes various commands contained in the control program (game program) stored in the terminal storage device 22. As a result, the display processing unit 274 of the terminal processing unit 27 starts the game and displays the game display object 1500 (Figure 15(a)), which includes the game screen of the started game, on the display device 23. Details of the game display object 1500, which includes the game screen, will be described later.

[0153] If User A wants to attach a new attachment object to character object 811a, or if User A wants to change an attachment object currently attached to character object 811a to a new attachment object, User A selects character object 811a. When character object 811a is selected by User A operating the input device 24, the display of communication screen 810 ends and communication screen 900 (Figure 9(a)) is displayed. Note that the display of communication screen 900 may also be performed when user output image 812a is selected by User A operating the input device 24.

[0154] Figure 9(a) shows another example of the communication screen 900 displayed on the display device 23 of the terminal device 2. The communication screen 900 is displayed when a character object 811a or a user output image 812a is selected by user A through operation of the input device 24, which is included in the communication screen 810 (Figure 8(b)).

[0155] On the communication screen 900, user output image 812a, which includes user A's character object 811a, is displayed across the entire screen. In addition, user output image 901, which includes the character objects 811b1, 811b2, and 811b3 of users B1, B2, and B3 respectively, is displayed superimposed on user output image 812a. Note that user output image 901 does not necessarily have to be displayed on the communication screen 900. In this case, the voice data of users B1, B2, and B3 may be output.

[0156] Furthermore, the communication screen 900 includes an add button 813 and a game button 814, as well as a change button 902 and a screen exit button 903. However, the communication screen 900 does not necessarily have to include the add button 813 and the game button 814.

[0157] The change button 902 is an object image such as a button object used to instruct the display of the selection screen 910 (Figure 9(b)). When the change button 902 is selected by user A through the operation of the input device 24, the selection screen 910 is displayed for selecting a new attachment object to be attached to the character object 811a. Details of the selection screen 910 will be described later.

[0158] The screen exit button 903 is an object image such as a button object used to indicate the end of the display of the communication screen 900. When the screen exit button 903 is selected by user A through the operation of the input device 24, the display of the communication screen 900 ends and the communication screen 810 (Figure 8(b)) is displayed.

[0159] Figure 9(b) shows an example of a selection screen 910 displayed on the display device 23 of the terminal device 2. On the selection screen 910, similar to the communication screen 900 (Figure 9(a)), a user output image 812a including the character object 811a of user A is displayed across the entire screen. The selection screen 910 also includes placement candidate objects 911 and a screen exit button 912.

[0160] As shown in object table T2, the placement location for each attachment object is one of a predetermined set of placement locations. In the example shown in Figure 9(b), the selection screen 910 includes multiple placement candidate objects 911, each representing one of the multiple placement locations. Note that the multiple placement candidate objects 911 may correspond to multiple categories (types) to which each attachment object belongs. The number of placement candidate objects 911 displayed on the selection screen 910 is not limited to five. If the display processing unit 274 of the terminal device 2 cannot include part of the image showing the multiple placement candidate objects 911 within the selection screen 910, it may display the placement candidate objects 911 in a scrollable manner. In this case, in response to a swipe operation by user A (for example, an operation in which a finger touches the screen and moves from the bottom to the top of the screen), placement candidate objects 911 that are not currently displayed on the selection screen 910 will be scrolled and displayed within the selection screen 910.

[0161] User A first selects the placement location for the attachment object they want to attach to the character object 811a. That is, when User A selects one of the multiple placement candidate objects 911 through the operation of the input device 24, a selection screen 1000 is displayed for User A to select the attachment object associated with the placement candidate corresponding to the selected placement candidate object 911.

[0162] The screen exit button 912 is an object image such as a button object used to indicate the end of the display of the selection screen 910. When the screen exit button 912 is selected by user A through the operation of the input device 24, the display of the selection screen 910 ends and the communication screen 810 (Figure 8(b)) is displayed.

[0163] Figure 10(a) shows an example of a selection screen 1000 displayed on the display device 23 of the terminal device 2. On the selection screen 1000, a user output image 812a including the user A's character object 811a is displayed across the entire screen, similar to the communication screen 900 (Figure 9(a)). The selection screen 1000 also includes a candidate attachment object 1001, which is a candidate attachment object to be attached to the character object 811a, and a screen exit button 1002.

[0164] The mounting candidate object 1001 is the selected object corresponding to the object ID of the object owned by user A that is associated with the placement candidate corresponding to the placement candidate object 911 selected on the selection screen 910 (Figure 9(b)). In the example shown in Figure 10(a), the selection screen 1000 includes three mounting candidate objects 1001, each corresponding to the object ID of three owned objects. The number of mounting candidate objects 1001 displayed on the selection screen 1000 is not limited to three. If the display processing unit 274 of the terminal device 2 cannot include part of the image showing multiple mounting candidate objects 1001 within the selection screen 1000, it may display the mounting candidate objects 1001 in a scrollable manner. In this case, in response to a swipe operation by user A (for example, an operation in which a finger touches the screen and moves from the bottom to the top of the screen), mounting candidate objects 1001 that are not currently displayed on the selection screen 1000 are scrolled and displayed within the selection screen 1000.

[0165] When user A selects one of the multiple mounting candidate objects 1001 through operation of the input device 24, the input device 24 inputs a change instruction to the terminal processing device 27. Along with the change instruction, the input device 24 inputs the object ID corresponding to the selected mounting candidate object 1001 to the terminal processing device 27. An example of the change processing in response to the change instruction is described below.

[0166] First, the display processing unit 274 acquires the change instruction and object ID input from the input device 24, and stores the acquired object ID in the user table T1 as a used object, associating it with user A's user ID. If the user table T1 already stores the object IDs of other used objects at the same mounting location as the acquired object ID, the display processing unit 274 deletes the object IDs of those other used objects from the user table T1 and stores the acquired object ID in the user table T1 as a used object, associating it with user A's user ID. Based on the motion data of user A generated by the generation unit 271 and the character object and used object associated with user A's user ID in the user table T1, the display processing unit 274 generates a user output image 1012a including the animation (moving image) of user A's character object 1011a, and displays the selection screen 1010 (Figure 10(b)) including the user output image 1012a. As a result, the user output image 1012a including the character object 1011a with the new mounting object attached is displayed.

[0167] Next, when the transmission unit 272 receives the change instruction and object ID input from the input device 24, it transmits information indicating the change instruction, along with the user ID of user A stored in the terminal storage device 22, the acquired object ID, and destination information (user IDs of users B (users B1 to B3)) to the server device 3 via the terminal communication interface 21. Next, the server receiving unit 331 of the server device 3 receives the information indicating the change instruction, user ID of user A, object ID, and destination information via the server communication interface 31. Next, the server transmission unit 332 of the server device 3 transmits the information indicating the change instruction, user ID of user A, and object ID to the terminal device 2 of users B (users B1 to B3) via the server communication interface 31. Then, the receiving unit 273 of user B's terminal device 2 receives information indicating a change instruction, user A's user ID, and object ID via the terminal communication I / F 21. The received object ID is then associated with user A's user ID as a used object and stored in user table T1. The display processing unit 274 of user B's terminal device 2 generates a user output image 1012a, which includes the animation (moving image) of user A's character object 1011a, based on the received motion data of user A and the character object and used object associated with user A's user ID in user table T1. The communication screen 1110 (Figure 11(b)) including the user output image 1012a is then displayed. As a result, user B's terminal device 2 also displays the user output image 1012a, which includes the character object 1011a with the newly attached object. With this, the change process is completed.

[0168] The screen exit button 1002 is an object image such as a button object used to indicate the end of the display of the selection screen 1000. When the screen exit button 1002 is selected by user A through the operation of the input device 24, the display of the selection screen 1000 ends and the communication screen 810 (Figure 8(b)) is displayed. Alternatively, if one of the multiple attachment candidate objects 1001 is selected by user A through the operation of the input device 24, the above-described change process may be executed, the display of the selection screen 1000 ends, and the communication screen 1110 (Figure 11(b)) including a user output image 1012a that includes a character object 1011a with the new attachment object attached may be displayed.

[0169] Figure 10(b) shows an example of a selection screen 1010 displayed on the display device 23 of the terminal device 2. Similar to the selection screen 1000 (Figure 10(a)), the selection screen 1010 displays a user output image 1012a, including the character object 1011a of user A, across the entire screen. Also, similar to the selection screen 1000, the selection screen 1010 includes candidate attachment objects 1013, which are candidates for attachment objects to be attached to the character object 1011a, and a screen exit button 1014.

[0170] Character object 1011a is the user's character object after a change instruction has been entered on the selection screen 1000 (Figure 10(a)). Mounting candidate object 1013 is a selected object that corresponds to the object ID of an object owned by user A, which is associated with the same placement position as mounting candidate object 1001 on the selection screen 1000. Since character object 1011a has an mounting object attached that corresponds to one of the mounting candidate objects 1001 on the selection screen 1000 (Figure 10(a)), mounting candidate objects corresponding to the mounting object attached to character object 1011a are not included in mounting candidate object 1013. However, mounting candidate objects corresponding to the mounting object attached to character object 1011a may be included in mounting candidate object 1013. In this case, mounting candidate objects corresponding to the mounted mounting object may be grayed out so that user A cannot select them.

[0171] The screen exit button 1014 is an object image such as a button object used to indicate the end of the display of the selection screen 1010. When the screen exit button 1014 is selected by user A through the operation of the input device 24, the display of the selection screen 1010 ends, and a communication screen 1110 (Figure 11(b)) is displayed, which includes a user output image 1012a that includes a character object 1011a with a new attachment object attached. A confirmation button (not shown) may also be displayed on the selection screen 1010. When the confirmation button is selected by user A through the operation of the input device 24, the display of the selection screen 1010 ends, and a communication screen 1110 (Figure 11(b)) is displayed, which includes a user output image 1012a that includes a character object 1011a with a new attachment object attached. In this case, when the screen exit button 1014 is selected by user A through the operation of the input device 24, the display of the selection screen 1010 ends, and a communication screen 810 (Figure 8(b)) is displayed, which includes a user output image 812a that includes a character object 811a before the change process.

[0172] When the change button 902 on the communication screen 900 is selected by user A operating the input device 24, the screen displayed is not limited to the selection screen 910; the selection screen 1100 may also be displayed. Figure 11(a) shows an example of the selection screen 1100 displayed on the display device 23 of the terminal device 2.

[0173] The selection screen 1100 shown in Figure 11(a) is displayed superimposed on the communication screen 900. The selection screen 1100 may also be displayed after the display of the communication screen 900 has ended.

[0174] The selection screen 1100 includes multiple placement candidate objects 1101, each representing one of several placement locations, and attachment candidate objects 1102, which are candidates for attachment objects to be attached to the character object 811a. When user A selects one of the multiple placement candidate objects 1101 through operation of the input device 24, the selection screen 1100 includes attachment candidate objects 1102 corresponding to the object ID of the object owned by user A that is associated with the placement candidate corresponding to the selected placement candidate object 1101. When user A selects one of the multiple attachment candidate objects 1102 through operation of the input device 24, the input device 24 inputs a change instruction to the terminal processing device 27, the above-described change process is executed, and the display processing device 274 displays a communication screen 1110 (Figure 11(b)) including a user output image 1012a that includes the character object 1011a with the new attachment object attached.

[0175] Figure 11(b) shows another example of the communication screen 1110 displayed on the display device 23 of the terminal device 2. The communication screen 1110 includes at least a user output image 1012a which includes a character object 1011a of user A wearing a new attachment object.

[0176] Furthermore, the communication screen 1110, like the communication screen 810 (Figure 8(b)), includes user output images 812b1, 812b2, and 812b3, which contain the respective character objects 811b1, 811b2, and 811b3 of users B1, B2, and B3, who are participating in the communication service.

[0177] When a change instruction entered by user B, who is participating in the communication service, is input to user B's terminal device 2, the change processing corresponding to the change instruction entered by user B is executed on both user B's terminal device 2 and user A's terminal device 2. As a result, a user output image including user B's character object with the newly attached object is displayed on both user B's terminal device 2 and user A's terminal device 2.

[0178] As explained with reference to Figures 8(b) to 11(b), at least a portion of User A's character object is modified in response to a modification instruction from User A, or at least a portion of User B's character object is modified in response to a modification instruction from User B. This prevents Users A and B from becoming bored during communication via the character objects and increases User A's motivation to continue communicating with User B.

[0179] Furthermore, in conventional information systems, users had to interrupt the communication service to change the output image containing a character object after initiating communication with other users via the character object. After the communication service was interrupted, it was necessary to activate a function to change the appearance of the character object and / or the background image within the output image containing the character object. Then, after changing the character object and / or output image using the change function, the user had to rejoin the communication service—a cumbersome procedure. In contrast, the information system 1 of this embodiment allows users to change their character object without interrupting communication via the character object or resetting the user's character object. As a result, the user interface for changing the user's character object during communication in the information system 1 of this embodiment is not complex, thus improving usability compared to conventional information systems. Also, because the user's character object can be changed without interrupting communication in the information system 1 of this embodiment, the number and volume of communications between the server device 3 and the terminal device 2 can be reduced. Thus, the information system 1 of this embodiment makes it possible to improve communication between the server device 3 and the terminal device 2.

[0180] Returning to Figure 8(b), if User A wants to modify the user output image 812b of at least one User B by including additional objects in the user output image 812b of at least one User B, User A selects the add button 813. When the add button 813 is selected by User A operating the input device 24, the selection screen 1200 is displayed.

[0181] Figure 12(a) shows an example of a selection screen 1200 displayed on the display device 23 of the terminal device 2. The selection screen 1200 shown in Figure 12(a) is displayed superimposed on the communication screen 810. The selection screen 1200 may also be displayed after the display of the communication screen 810 has ended.

[0182] In the example shown in Figure 12(a), images and names representing eight types of additional candidate objects are included within the display area of ​​the selection screen 1200. The number of additional candidate objects included within the display area of ​​the selection screen 1200 is not limited to eight. If the display processing unit 274 of the terminal device 2 cannot include some of the images representing multiple additional candidate objects within the display area of ​​the selection screen 1200, it may display the images representing the additional candidate objects in a scrollable manner. In this case, in response to a swipe operation by user A (for example, an operation in which a finger touches the screen and moves from right to left on the screen), images representing additional candidate objects not currently included within the display area of ​​the selection screen 1200 are scrolled and appear within the display area of ​​the selection screen 1200.

[0183] Each additional candidate object represents an additional object, and the image representing the additional candidate object is, for example, an icon image or thumbnail image representing a gift object. Furthermore, on the selection screen 1200, the image and / or name of each additional candidate object may be displayed categorized by the type of additional object (free gift, paid gift, accessory, cheering goods, appeal, variety, etc.). Additional objects belonging to paid gifts are those that can be purchased by the user by consuming "coins," which represent virtual currency. Additional objects belonging to free gifts are those that can be obtained without consuming monetary value, but instead by consuming points earned by watching broadcasts, etc. Additional objects may also be objects that user A possesses by playing a predetermined mission executed on terminal device 2 and clearing that mission.

[0184] When User A selects one of the candidate objects to be added through operation of the input device 24, the input device 24 inputs an add command to the terminal processing device 27. If, around the same time as the selection of a candidate object to be added, User A selects one of User B's user output images 812b that they want to include the add object in through operation of the input device 24, the User ID of User B corresponding to the selected User Output Image 812b is identified. Along with the add command, the input device 24 inputs the object ID corresponding to the selected candidate object to the terminal processing device 27, along with the identified User ID. If none of User B's user output images 812b are selected, the input device 24 inputs the object ID corresponding to the selected candidate object to the terminal processing device 27 along with the add command. In this case, all of User B's user output images 812b will include the add object. The selection of User B's user output images 812b that they want to include the add object in may also be the selection of a character object 811b included in the user output image 812b. Furthermore, user output images 812b or character objects 811b of two or more users B may be selected. Below, an example of additional processing in response to additional instructions when user output image 812b1 of user B1 is selected will be described.

[0185] First, the display processing unit 274 obtains the additional instruction and object ID input from the input device 24, then refers to the object table T2 to extract still or moving images (image information) of the additional object associated with the obtained object ID from the terminal storage device 22, and generates a user output image including the extracted still or moving images. If the user ID of user B1 is obtained along with the additional instruction, the user output image including the extracted still or moving images of the additional object is the user output image corresponding to the obtained user ID of user B1. If none of the user IDs of users B1 to B3 are obtained along with the additional instruction, the user output image including the extracted still or moving images is the user output image corresponding to each of the users (user A's user output image and users B1 to B3).

[0186] Next, when the transmitting unit 272 receives the additional instruction and object ID from the input device 24, it transmits information indicating the additional instruction, along with the user ID of user A stored in the terminal storage device 22, the acquired object ID, and destination information (the user IDs of each user B (users B1 to B3)) to the server device 3 via the terminal communication interface 21. Next, the server receiving unit 331 of the server device 3 receives the information indicating the additional instruction, the user ID of user A, and the object ID via the server communication interface 31. Next, the server transmitting unit 332 of the server device 3 transmits the information indicating the additional instruction, the user ID of user A, and the object ID to each of the terminal devices 2 of each user B (users B1 to B3) via the server communication interface 31. Then, when the receiving unit 273 of User B's terminal device 2 receives information indicating a change instruction, User A's User ID, and Object ID via the terminal communication I / F 21, the display processing unit 274 of User B's terminal device 2 refers to the object table T2 and extracts a still image or moving image (image information) of the additional object associated with the received Object ID from the terminal storage device 22, and generates a user output image including the extracted still image or moving image of the additional object. Note that if the transmitting unit 272 of User A's terminal device 2 transmits User B1's User ID along with the additional instruction, User B1's User ID is also transmitted by the server device 3. If User B's terminal device 2 receives User B1's User ID along with the additional instruction, the user output image including the additional object displayed on User B's terminal device 2 is only the user output image corresponding to the received User B1's User ID. If User B's terminal device 2 does not obtain User B's User ID along with the additional instruction, the user output image including the additional object displayed on User B's terminal device 2 is the user output image corresponding to all users (User A and Users B1 to B3). This concludes the additional processing.

[0187] Figure 12(b) shows another example of the communication screen 1210 displayed on the display device 23 of terminal device 2 after an additional instruction from user A. Figure 12(b) shows the communication screen 1210 displayed on the display device 23 of terminal device 2 after an additional instruction is received along with the user ID of user B1. As shown in Figure 12(b), the only user output image that includes the additional object 1211 is the user output image 812b1 corresponding to the user ID of user B1.

[0188] Figure 13(a) shows another example of the communication screen 1300 displayed on the display device 23 of terminal device 2 after user A gives an additional instruction. Figure 13(a) shows the communication screen 1300 displayed on the display device 23 of terminal device 2 after an additional instruction, when user B's user ID is not received along with the additional instruction. As shown in Figure 13(a), the user output image including the additional object 1301 is user A's user output image 812a and user output images 812b1 to 812b3 corresponding to all user IDs of user B. The user output image including the additional object 1301 after user A gives an additional instruction may also be user output images 812b1 to 812b3 corresponding to each of user IDs of user B (user A's user output image 812a may not be included).

[0189] In the examples shown in Figures 12(a) to 13(a), additional objects are automatically added to the user output images of one or more users in response to additional instructions from the user. However, the system may be configured so that additional objects are added to the user output images of one or more users only after an additional instruction has been made by a user, if an approval instruction has been received from another user other than the user who made the additional instruction.

[0190] Figure 13(b) shows another example of the communication screen 1310 displayed on the display device 23 of the terminal device 2 after an additional instruction is given. In the example shown in Figure 13(b), the additional instruction is input by user B1 through the operation of the input device 24 of the terminal device 2. In the example shown in Figure 13(b), the additional instruction is input by user B1 through the input device 24 of the terminal device 2 when one of the additional candidate objects representing the attached object is selected, and user A's user output image 812a is selected as the user output image that is to include the attached object.

[0191] When user A's terminal device 2 receives an add command along with the object ID of the attached object corresponding to the selected add candidate object, it displays a user output image 612a including the add object 1311 as shown in Figure 13(b). The add object 1311 displays information indicating that the add object was sent by user B1, who issued the add command. The add object 1311 is not displayed on the terminal devices 2 of users B1 to B3. However, the add object 1311 may be displayed on the terminal devices 2 of users B1 to B3.

[0192] When User A performs a predetermined operation on the communication screen 1310 displaying the user output image 812a including the additional object 1311 (for example, a tap operation on the user output image 812a, a swipe operation in a predetermined direction, etc.), the input device 24 of User A's terminal device 2 inputs an acceptance instruction to the terminal processing device 27, along with the object ID of the attached object transmitted from User B1's terminal device 2. An example of acceptance processing in response to an acceptance instruction is described below.

[0193] First, the display processing unit 274 obtains the acceptance instruction and object ID input from the input device 24, and then refers to the object table T2 to generate a user output image that includes the character object of user A wearing the additional object (attached object) associated with the obtained object ID. Then, the display processing unit 274 displays a communication screen 1400 that includes the user output image 812a, which includes the character object 811a of user A wearing the additional object (attached object) 1401.

[0194] Next, when the transmission unit 272 receives the acceptance instruction and object ID from the input device 24, it transmits the information indicating the acceptance instruction, along with the user ID of user A stored in the terminal storage device 22, the acquired object ID, and the destination information (user IDs of users B (users B1 to B3)) to the server device 3 via the terminal communication interface 21. Next, the server receiving unit 331 of the server device 3 receives the information indicating the acceptance instruction, the user ID and object ID of user A, and the destination information via the server communication interface 31. Next, the server transmission unit 332 of the server device 3 transmits the information indicating the acceptance instruction, the user ID and object ID of user A to each of the terminal devices 2 of all users B (users B1 to B3) via the server communication interface 31. Finally, the receiving unit 273 of user B's terminal device 2 receives the information indicating the acceptance instruction, the user ID and object ID of user A via the terminal communication interface 21. Next, the display processing unit 274 of user B's terminal device 2 refers to the object table T2 and generates a user output image 812a that includes user A's character object 811a wearing the additional object (attached object) 1401 associated with the received object ID. Then, the display processing unit 274 displays the communication screen 1400, which includes the user output image 812a that includes user A's character object 811a wearing the additional object (attached object) 1401. With this, the acceptance process is completed.

[0195] Furthermore, the character object modified by the additional object (attached object) related to the additional instruction by user B1 may also be the character object of all users other than user B1 (in the example shown in Figure 13(b), users A, B2, and B3). In this case, user output images 812a, 812b2, and 812b3 including the additional object 1311 will be displayed on the terminal device 2 of all users other than user B1. When an acceptance instruction regarding the additional object 1311 is entered by users A, B2, and B3, user output image 812a including the character object 811a of user A with the additional object (attached object) 1401 attached, user output image 812b2 including the character object 811b2 of user B2 with the additional object (attached object) 1401 attached, and user output image 812b3 including the character object 811b3 of user B3 with the additional object (attached object) 1401 attached will be displayed. In this case, the additional object (attached object) 1401 may be attached to the character object 811b1 of user B1, who issued the additional instruction.

[0196] Figure 14(a) shows another example of the communication screen 1400 displayed on the display device 23 of the terminal device 2 after acceptance instructions. As shown in Figure 14(a), the communication screen 1400 displays a user output image 812a that includes a character object 811a with an additional object (attached object) 1401 attached.

[0197] When a new attachment object is attached to a user's character object, it may be possible to allow other users to evaluate it. Figure 14(b) shows another example of the communication screen 1410 displayed on the display device 23 of user A's terminal device 2 when a new attachment object 1411 is attached to user B2's character object 811b2.

[0198] The communication screen 1410 includes an evaluation button 1412, along with the additional button 813 and the game button 814. In the example shown in Figure 14(b), the evaluation button 1412 is located within the display area of ​​the user output image 812a, but it may also be displayed outside the display area of ​​the user output image 812a. Furthermore, the display control unit 274 may control the display of the evaluation button 1412 so that it is displayed only for a predetermined time (for example, 10 seconds) after the attachment object 1411 is attached to the user B2's character object 811b2.

[0199] The evaluation button 1412 is an object image for inputting an evaluation instruction to send evaluation data indicating User A's evaluation of the character object 811b2 to which the newly attached object 1411 has been attached. When User A selects the evaluation button 1412 by operating the input device 24, the input device 24 inputs the evaluation instruction to the terminal processing device 27. Along with the evaluation instruction, the input device 24 inputs User A's user ID, stored in the terminal storage device 22, and User B2's user ID related to the character object 811b2 to which the newly attached object has been attached, to the terminal processing device 27. An example of the evaluation process in response to the evaluation instruction will be described below.

[0200] First, the display processing unit 274 of user A's terminal device 2 receives the evaluation instruction, user A's user ID, and user B2's user ID from the input device 24, and then displays information indicating a positive evaluation of the character object 811b2 to which the newly attached object has been attached. For example, the display processing unit 274 extracts the name of user B2 associated with user B2's user ID and displays a string praising user B2's name as information indicating a positive evaluation. Alternatively, for example, the display processing unit 274 may display a predetermined mark image as information indicating a positive evaluation within the display area of ​​the user output image 812b2 of the character object 811b2.

[0201] Next, the transmitting unit 272 of user A's terminal device 2 receives information indicating the evaluation instruction, user A's user ID, and user B2's user ID from the input device 24. It then transmits the acquired information, along with destination information (user IDs of users B (users B1 to B3)), to the server device 3 via the terminal communication interface 21. Next, the server receiving unit 331 of the server device 3 receives the information indicating the evaluation instruction, user A's user ID, user B2's user ID, and destination information via the server communication interface 31. Finally, the server transmitting unit 332 of the server device 3 transmits the information indicating the evaluation instruction, user A's user ID, and user B2's user ID to user B's (users B1 to B3) terminal device 2 via the server communication interface 31. Then, when the receiving unit 273 of user B's terminal device 2 receives information indicating a change evaluation instruction, user A's user ID, and user B2's user ID via the terminal communication I / F 21, the display processing unit 274 of user B's terminal device 2 displays information indicating a positive evaluation regarding the character object 811b2 to which the newly attached object has been attached. The display processing unit 274 may, for example, extract the name of user B2 associated with user B2's user ID and display a string praising user B2's name as information indicating a positive evaluation. Alternatively, for example, the display processing unit 274 may display a predetermined mark image as information indicating a positive evaluation within the display area of ​​the user output image 812b2 of the character object 811b2. With this, the evaluation process is completed.

[0202] This allows User A to react with simple operations when an attachment object is attached to User B's character object, thereby increasing User A's motivation to continue communicating with User B. Furthermore, the display screen size of the terminal device 2's display device 23 is limited, and displaying a large amount of information on the display device 23 makes it difficult for the user to recognize the displayed information. In this embodiment, User A's terminal device 2 displays an evaluation button 1412 when a character object or user output image of a user other than User A changes. User A can then input an evaluation of the changed character object or user output image of the other user by operating the evaluation button 1412. Thus, in this embodiment, User A's terminal device 2 can provide User A with the evaluation button 1412 in accordance with the timing of the evaluation of the character object or user output image, thereby improving usability. Note that the information displayed in response to User A's evaluation instruction is not limited to positive information; negative information is also acceptable. Additionally, there may be multiple types of evaluation buttons 1412 displayed. For example, two types of evaluation buttons may be displayed: one for submitting positive evaluation data and another for submitting negative evaluation data.

[0203] Returning to Figure 8(b), a game display object including a game screen showing the progress of the game being played by User A may be displayed in place of, or together with, User A's user output image 812a. The game display object may also be a so-called filter. For example, when User A selects the game button 814 by operating the input device 24, the input device 24 inputs a game start command to the terminal processing device 27. In addition, the selection screen 1200 may include an additional candidate object that has the same function as the game button 814. In this case, when User A selects an additional candidate object for inputting a game start command by operating the input device 24, the input device 24 inputs a game start command to the terminal processing device 27.

[0204] First, when the display processing unit 274 receives a game start command from the input device 24, it starts the game by executing various commands contained in the control program (game application program, etc.) stored in the terminal storage device 22, and displays a game display object 1500, including the game screen of the started game, on the display device 23. Figure 15(a) shows an example of a game display object 1500, including the game screen, that is displayed on the display device 23 of user A's terminal device 2 in response to a game start command from user A.

[0205] When the game display object 1500 is displayed, the display processing unit 274 advances the game of the game display object 1500 in accordance with various input information from user A and various commands contained in the control program (game application program, etc.). Then, when predetermined game termination conditions are met, the display processing unit 274 terminates the game of the game display object 1500.

[0206] Furthermore, when the game display object 1500 is displayed, the generation unit 271 generates game image data showing the game screen of the game being played by user A as the game progresses. Next, the generation unit 271 generates output information including the generated game image data and the user ID stored in the terminal storage device 22, and outputs the generated output information to the transmission unit 272.

[0207] Next, the transmitting unit 272 of User A's terminal device 2 transmits User A's output information, along with destination information (User IDs of Users B (Users B1 to B3)), to the server device 3 via the terminal communication interface 21. Next, the server receiving unit 331 of the server device 3 receives User A's output information transmitted from User A's terminal device 2 via the server communication interface 31. Next, the server transmitting unit 332 of the server device 3 transmits User A's output information to User B's (Users B1 to B3) terminal devices 2 via the server communication interface 31. When the receiving unit 273 of User B's terminal device 2 receives the output information via the terminal communication interface 21, the display processing unit 274 of User B's terminal device 2 displays a game display object 1511 containing a game image based on the game image data included in the output information, instead of User A's user output image 812a, which is indicated by the User ID included in the output information (Figure 15(b)).

[0208] In the example shown in Figure 15(a), the game displayed in the game screen of the game display object 1500 is a fishing game. The game screen included in the game display object 1500 contains a game object 1501, a target object 1502, a parameter cursor 1503, and a parameter gauge frame 1504.

[0209] Game object 1501 is an object image resembling a fish, and its display is controlled by the display processing unit 274 so that it automatically moves along a predetermined path or a randomly determined path within the game screen. The terminal storage device 22 stores initial values ​​associated with game object 1501, and the display processing unit 274 sets these initial values ​​as parameter values ​​for game object 1501 at the start of the game.

[0210] The target object 1502 consists of a first target object 1502a, a second target object 1502b, and a third target object 1502c. The first target object 1502a is an object image having a roughly circular shape, the second target object 1502b is an object image having a shape that surrounds the first target object 1502a, and the third target object 1502c is an object image having a shape that surrounds the second target object 1502b.

[0211] If the input device 24 is a touch panel, the input device 24 inputs a game input instruction to the display processing unit 274 when user A taps within the game object 1501. The display processing unit 274 determines whether a predetermined location on the game object 1501 (for example, the location corresponding to the fish's mouth) at the time it receives the game input instruction from the input device 24 is located in one of the first target object 152a, the second target object 1502b, or the third target object 1502c.

[0212] If the display processing unit 274 determines that a predetermined location of the game object 1501 is located at any of the first target object 1502a, the second target object 1502b, or the third target object 1502c, it changes (increases or decreases) the parameter value of the game object 1501.

[0213] The display processing unit 274 modifies the amount of change in the parameter value of the game object 1501 according to the position of a predetermined location of the game object 1501 at the time the game input instruction is received from the input device 24. For example, if the predetermined location of the game object 1501 at the time the game input instruction is received is located within the first target object 1502a, the display processing unit 274 decreases the parameter value of the game object 1501 by a first decrease value. If the predetermined location of the game object 1501 at the time the game input instruction is received is located within the first target object 1502a, the display processing unit 274 may increase the parameter value of the game object 1501 by a first increase value. Furthermore, if the predetermined location of the game object 1501 at the time the game input instruction is received is outside the first target object 1502a but within the second target object 1502b, the display processing unit 274 decreases the parameter value of the game object 1501 by a second decrease value that is smaller than the first decrease value. Furthermore, if the designated location of the game object 1501 at the time of game input instruction is outside the first target object 1502a and inside the second target object 1502b, the display processing unit 274 may increase the parameter value of the game object 1501 by a second increase value that is smaller than the first increase value. Also, if the designated location of the game object 1501 at the time of game input instruction is outside the second target object 1502b and inside the third target object 1502c, the display processing unit 274 may decrease the parameter value of the game object 1501 by a third decrease value that is smaller than the second decrease value. Furthermore, if the designated location of the game object 1501 at the time of game input instruction is outside the second target object 1502b and inside the third target object 1502c, the display processing unit 274 may increase the parameter value of the game object 1501 by a third increase value that is smaller than the second increase value. Furthermore, if a predetermined location of the game object 1501 is located outside the target object 1502 at the time of game input instruction, the display processing unit 274 does not change the parameter value of the game object 1501.

[0214] The parameter cursor 1503 is a cursor object that indicates the parameter value of the game object 1501. The display processing unit 274 positions the parameter cursor 1503 within the parameter gauge frame 1504 at a position corresponding to the parameter value of the game object 1501. In the example shown in Figure 13(a), the parameter cursor 1503 can move left and right within the parameter gauge frame 1504, and the larger the parameter value of the game object 1501, the further to the right it is positioned within the parameter gauge frame 1504.

[0215] The display processing unit 274 determines that user A has successfully cleared the game and terminates the game if the parameter value of the game object 1501 becomes less than or equal to a predetermined value (for example, "0" (zero)). The display processing unit 274 may also determine that user A has successfully cleared the game if the parameter value of the game object 1501 becomes greater than or equal to a predetermined value (a value greater than the parameter value at the start of the game). The display processing unit 274 may continue the game until the game is successfully cleared, or it may terminate the game if a predetermined game period (for example, 30 seconds) has elapsed from the start of the game, even if user A has not successfully cleared the game.

[0216] When the game screen included in the game display object 1500 ends, the game display object may be displayed on either user B's terminal device 2. For example, the transmission unit 272 of user A's terminal device 2 sends information indicating the end of the game to the server device 3, and the server device 3 sends information indicating the end of the game relating to user A to user B1's terminal device 2. When the display processing unit 274 of user B1's terminal device 2 receives the information regarding the end of the game, it starts a game implemented by executing various commands included in the control program (game application program, etc.) stored in the terminal storage device 22, and displays the game display object 1500, including the game screen of the started game, on the display device 23. When the game played by user B1 ends, similarly, information indicating the end of the game is sent from user B1's terminal device 2 to user B2's terminal device 2 via the server device 3, and the display processing unit 274 of user B1's terminal device 2 displays the game display object 1500. Furthermore, when user B2 finishes playing a game, information indicating the end of the game is similarly sent from user B2's terminal device 2 to user B3's terminal device 2 via server device 3, and the display processing unit 274 of user B1's terminal device 2 displays the game display object 1500. Note that terminal devices 2 of users not currently playing a game will display the screen shown in Figure 13(b), allowing them to see the game status of other users. In this way, each user in the communication group can take turns playing games, which can increase users' motivation to continue using the communication system.

[0217] Terminal device 2 may also execute the game program by calling a browser function incorporated into the control program in order to run the game. For example, terminal device 2 uses the called browser function to receive a web page (a document written in a markup language such as HTML (Hyper Text Markup Language)) from server device 3. Then, terminal device 2 executes the game program incorporated into the received web page. For example, by using HTML5 as the markup language, terminal device 2 can easily run new games. By adopting the above-described web application in information system 1, game program creators can provide new games to the client side (terminal device 2) simply by incorporating the new game program into the web page sent by server device 3. This makes it possible for game program creators to significantly reduce the effort required to create new game programs. Furthermore, on the client side, new games can be provided simply by receiving a web page without downloading the new game program. As a result, in information system 1, it is possible to reduce the load on the communication network, communication costs and / or communication time compared to downloading game programs, and to simplify the user interface.

[0218] The game input instructions from user A via the input device 24 are not limited to those input in response to tap operations. For example, the input device 24 may input game input instructions to the display processing unit 274 in response to a swipe operation by user A in a predetermined direction. Furthermore, game input instructions are not limited to those input by the input device 24. For example, the display processing unit 274 may acquire face motion data generated as time progresses by the generation unit 271 based on continuously acquired imaging data, and generate game input instructions when it determines that user A has performed a predetermined action based on the face motion data. Predetermined actions include, for example, user A closing their eyes, user A turning their face in a predetermined direction, or user A's hand performing a predetermined action. When game input instructions are generated based on face motion data, a game display object 1500 including a game screen showing the progress of the game being played by user A may be displayed along with user A's user output image 812a. The simultaneous display of the user output image 812a and the game display object 1500 may be in the form of side-by-side display of the user output image 812a and the game display object 1500, or the user output image 812a may be superimposed on the game display object 1500. For example, the user output image 812a may be displayed as the background image of the game display object 1500. This allows the user to play the game while checking the movement of their character object, thereby improving the operability of the game. Furthermore, if user A shakes the terminal device 2 having an acceleration sensor (not shown), and the acceleration sensor measures an acceleration of a predetermined value or higher, the display processing unit 274 may generate a game input instruction.

[0219] The game displayed in the game screen of game display object 1500 does not have to be a fishing game. For example, it could be a game in which a character object of user A moves a predetermined distance each time user A closes their eyes, or a game in which a character object of any user B moves a predetermined distance each time user A closes their eyes.

[0220] Furthermore, if User B can input a predetermined game input command to the game image within the game display object 1511, the game shown in the game image within the game display object 1511 may progress based on the game input command corresponding to any of User B's operations. Note that if User B can input a predetermined game input command to the game image within the game display object 1511, the input device 24 of User A's terminal device 2 may be controlled to prevent User A from inputting a game input command. Also, if User B can input a predetermined game input command to the game image within the game display object 1511, the game shown in the game image within the game display object 1511 may progress based on the game input commands from User A and all of User B's input commands. In the screen of Figure 15(b), not only the game display object 1511 played by User A, but also the game display objects 1511 played by Users B1, B2, and B3 may be displayed. In this case, each user will see four game display objects 1511 on the screen shown in Figure 15(b), and each user will play the game corresponding to the game display object 1511 for that user.

[0221] In this way, User A can provide the game screen during communication with User B. This makes it possible for both User A and User B to increase their motivation to continue communicating.

[0222] (Operation sequence of Information System 1) Figure 16 shows an example of the operation sequence of information system 1. This operation sequence is executed mainly by the terminal processing unit 27 and the server processing unit 33 in cooperation with each element of terminal device 2 and server device 3, based on control programs stored in advance in terminal storage device 22 and server storage device 32. Hereinafter, it will be explained that user A operates terminal device 2a, user B1 operates terminal device 2b1, and user B2 operates terminal device 2b2. Note that the terminal devices of users other than user A are not limited to terminal devices 2b1 to 2b2 of users B1 and B2, but terminal devices 2 of three or more users may be connected to server device 3.

[0223] First, the transmission unit 272 of the terminal device 2a transmits output information to the server device 3 via the terminal communication interface 21, including character video data containing motion data of user A generated by the generation unit 271 based on imaging data continuously output by the imaging device 25, voice data of user A output from the microphone 26, and user ID of user A (step S101). The output information may also include game image data showing the game screen of the game the user is playing and user ID of user A. In addition, destination information may be included in the transmission process of the transmission unit 272. The process in step S101 is executed continuously at predetermined time intervals (for example, every 2 seconds) until the distribution event ends, so the processes related to steps S101 to S110 are executed intermittently.

[0224] Next, the server transmission unit 332 of the server device 3 transmits the output information received from terminal device 2a to terminal device 2b1 via the server communication interface 31 (step S102), and also transmits it to terminal device 2b2 (step S103).

[0225] Furthermore, the transmission unit 272 of terminal device 2b1 transmits output information including user B1's character video data, user B1's voice data, and user B1's user ID to server device 3 via terminal communication I / F 21 (step S104). Next, the server transmission unit 332 of server device 3 transmits the output information received from terminal device 2b1 to terminal device 2a via server communication I / F 31 (step S105), and also to terminal device 2b2.

[0226] Furthermore, the transmission unit 272 of terminal device 2b2 transmits output information including user B2's character video data, user B2's voice data, and user B2's user ID to server device 3 via terminal communication I / F 21 (step S106). Next, the server transmission unit 332 of server device 3 transmits the output information received from terminal device 2b2 to terminal device 2a via server communication I / F 31 (step S107), and also to terminal device 2b1.

[0227] The display processing unit 274 of the terminal device 2a displays and outputs a communication screen to the display device 23 of the terminal device 2a, which includes a user output image including the character object of user A, a user output image including the character object of user B1, and a user output image including the character object of user B2, based on the output information of user A, user B1, and user B2, and outputs the voices of user B1 and user B2 (step S108).

[0228] Similar to step S108, the display processing unit 274 of the terminal device 2b1 displays and outputs a communication screen on the display device 23 of the terminal device 2b1, which includes a user output image including the character object of user A, a user output image including the character object of user B1, and a user output image including the character object of user B2, based on the output information of user A, user B1, and user B2, and outputs the voices of user A and user B2 (step S109).

[0229] Similar to step S108, the display processing unit 274 of the terminal device 2b2 displays and outputs a communication screen to the display device 23 of the terminal device 2b2, which includes a user output image including the character object of user A, a user output image including the character object of user B1, and a user output image including the character object of user B2, based on the output information of user A, user B1, and user B2, and outputs the voices of user A and user B1 (step S110).

[0230] Next, the transmission unit 272 of the terminal device 2a transmits information indicating one of the following—a change instruction, an addition instruction, or an evaluation instruction—which has been input in response to the operation of the input device 24 by user A, to the server device 3 via the terminal communication I / F 21 (step S111).

[0231] Next, the server transmission unit 332 of the server device 3 transmits information indicating one of the change instruction, addition instruction, or evaluation instruction received from the terminal device 2a to the terminal device 2b1 via the server communication I / F 31 (step S112), and also transmits it to the terminal device 2b2 (step S113).

[0232] The display processing unit 274 of the terminal device 2a modifies at least a portion of the user output image of user A, the user output image of user B1, and the user output image of user B2, in response to any of the change instruction, addition instruction, and evaluation instruction input by user A in response to the operation of the input device 24, and displays the modified image on the display device 23 of the terminal device 2a (step S114).

[0233] The display processing unit 274 of terminal device 2b1 modifies at least a portion of the user output image of user A, the user output image of user B1, and the user output image of user B2, in accordance with the information transmitted from user A's terminal device 2a indicating one of the following: a change instruction, an addition instruction, or an evaluation instruction, and displays the modified image on the display device 23 of terminal device 2b1 (step S115).

[0234] The display processing unit 274 of terminal device 2b2 modifies at least a portion of the user output image of user A, the user output image of user B1, and the user output image of user B2, in accordance with the information transmitted from user A's terminal device 2a indicating a change instruction, an addition instruction, or an evaluation instruction, and displays the modified image on the display device 23 of terminal device 2b2 (step S116).

[0235] Next, the transmission unit 272 of the terminal device 2b1 transmits information indicating an instruction to add an object to be attached, which was input in response to the operation of the input device 24 by user B1, to the server device 3 via the terminal communication I / F 21 (step S117).

[0236] Next, the server transmission unit 332 of the server device 3 transmits information indicating an additional instruction for an attached object, received from the terminal device 2b1, to the terminal device 2a via the server communication interface 31 (step S118).

[0237] The display processing unit 274 of terminal device 2a determines whether a predetermined operation related to acceptance has been performed by user A for the additional object displayed in response to information indicating an instruction to add an object transmitted from user B1's terminal device 2a (step S119).

[0238] If it is determined that User A has not performed the prescribed operation related to acceptance (step S119-No), step S123 is not executed. If it is determined that User A has performed the prescribed operation related to acceptance (step S119-Yes), the transmission unit 272 of terminal device 2a transmits information indicating acceptance instruction to server device 3 via terminal communication I / F 21 (step S120).

[0239] Next, the server transmission unit 332 of the server device 3 transmits information indicating the acceptance instruction received from terminal device 2a to terminal device 2b1 via the server communication interface 31 (step S121), and also transmits it to terminal device 2b2 (step S122). Along with the information indicating the acceptance instruction, an object ID indicating the installed object related to the additional instruction information transmitted from user B1's terminal device 2a in step S117 is also transmitted.

[0240] In step S123, the display processing unit 274 of the terminal device 2a displays a user output image on the display device 23 of the terminal device 2a, which includes a character object of user A wearing an attachment object related to the attachment information indicating additional instructions transmitted from user B1's terminal device 2a in step S117.

[0241] In step S117, the display processing unit 274 of terminal device 2b1 outputs a user output image to the display device 23 of terminal device 2b1, which includes a character object of user A wearing an attachment object related to the attachment object related to the information indicating additional instructions transmitted from terminal device 2a of user B1 (step S124).

[0242] In step S117, the display processing unit 274 of terminal device 2b2 outputs a user output image to the display device 23 of terminal device 2b2, which includes a character object of user A wearing an attachment object related to the information indicating additional instructions transmitted from terminal device 2a of user B1 (step S125).

[0243] As detailed above, in the information system 1 of this embodiment, in response to instructions from user A, user output images of user A, which have been modified in at least part, and / or user output images of at least one user B, which have been modified in at least part, are displayed. In this way, the information system 1 of this embodiment makes it possible to improve the user's motivation to continue using the communication service by making the output from terminal device 2 modified by user A or at least one user B during the communication service.

[0244] (Variation 1) It should be noted that the present invention is not limited to this embodiment. For example, if certain conditions are met, the display processing unit 274 of user A's terminal device 2 may add new candidate objects to the display area of ​​the selection screen 1200 (Figure 12(a)). Also, if certain conditions are met, the display processing unit 274 of user A's terminal device 2 may delete at least some of the candidate objects included in the display area of ​​the selection screen 1200.

[0245] The specified condition is that the time that User A's character object or user output image is displayed on the communication screen exceeds a predetermined time. Alternatively, the specified condition may be that the time that User A's character object or user output image is displayed on the communication screen does not exceed a predetermined time. Alternatively, the specified condition may be that the time that any User B's character object or user output image is displayed on the communication screen exceeds a predetermined time or does not exceed a predetermined time. Alternatively, the specified condition may be that the total time that User A's and all Users B's character objects or user output images are displayed on the communication screen exceeds a predetermined time or does not exceed a predetermined time. Alternatively, the specified condition may be that the total time that User A's and at least two of all Users B's character objects or user output images are displayed on the communication screen exceeds a predetermined time or does not exceed a predetermined time.

[0246] Furthermore, the predetermined condition may be that the number of times any of the candidate objects for addition are selected by User A through the operation of the input device 24 exceeds a predetermined number of times. Alternatively, the predetermined condition may be that the number of times any of the candidate objects for addition are selected by User A through the operation of the input device 24 does not exceed a predetermined number of times. Alternatively, the predetermined condition may be that the number of times any of the candidate objects for addition are selected by any of the Users B through the operation of the input device 24 exceeds a predetermined number of times or does not exceed a predetermined number of times. Alternatively, the predetermined condition may be that the total number of times any of the candidate objects for addition are selected by User A and all Users B through the operation of the input device 24 exceeds a predetermined number of times or does not exceed a predetermined number of times.

[0247] The new candidate objects added to the display area of ​​the selection screen 1200 when certain conditions are met may correspond to, for example, a specific wallpaper object placed behind a character object in the user output image, or a specific wearable object.

[0248] Thus, by fulfilling certain conditions, the range of additional objects that can be selected is broadened (or narrowed), making it possible to add variety to the output in response to user A's actions during the communication service. Therefore, it becomes possible to increase user A's motivation to continue using the communication service with user B.

[0249] (Modification 2) As shown in Figure 14(b), when a new attachment object 1411 is attached to user B2's character object 811b2, user A can operate the evaluation button 1412 included in the newly displayed communication screen 1410. User A's terminal device 2 may be configured to allow user A to perform operations other than operating the evaluation button 1412, either in place of or in conjunction with the evaluation button 1412.

[0250] For example, if a new attachment object 1411 is attached to user B2's character object 811b2, the communication screen 1410 may include an action button for a predetermined period (e.g., 10 seconds) to automatically cause user A's character object 811a to perform a predetermined action. When user A selects the action button by operating the input device 24, the generation unit 271 suspends the use of user A's motion data for a predetermined period (e.g., 5 seconds), extracts information about the predetermined action previously stored in the terminal storage device 22, and causes user A's character object 611a to automatically perform the predetermined action for a predetermined period (e.g., 5 seconds). The predetermined action could be, for example, clapping or showing surprise. Note that when user A selects the action button by operating the input device 24, the generation unit 271 does not need to suspend the use of some of user A's motion data. For example, if the action corresponding to the action button is an action related to the character object 611a's arm, the use of face motion data may continue. As a result, information system 1 can go beyond uniform movements controlled by action buttons and enable richer expressions for character objects, thereby increasing the user's motivation to continue using the system for communication.

[0251] Furthermore, information regarding a predetermined action to be automatically performed by User A's character object 811a may be associated with each of several predetermined operations performed by User A and stored in the terminal storage device 22. For example, information regarding a peace sign gesture may be stored in the terminal storage device 22 in association with an upward swipe operation, and information regarding a jumping gesture may be stored in the terminal storage device 22 in association with a downward swipe operation. For example, when the input device 24 detects that User A has performed an upward swipe operation, the generation unit 271 stops using User A's motion data for a predetermined period (e.g., 5 seconds), extracts the information regarding a peace sign gesture previously stored in the terminal storage device 22, and causes User A's character object 811a to automatically perform a peace sign gesture for a predetermined period (e.g., 5 seconds). In this case, the information regarding the predetermined action associated with the predetermined operation may be changed for a predetermined period (e.g., 10 seconds) after a new attachment object 1411 is attached to User B2's character object 811b2. For example, after a new attachment object 1411 is attached to user B2's character object 811b2, for a predetermined period (e.g., 10 seconds), information regarding a clapping motion (by modifying information regarding a peace sign motion) may be stored in the terminal storage device 22 in association with an upward swipe operation. As a result, when the input device 24 detects that user A has performed an upward swipe operation during a predetermined period (e.g., 10 seconds) after the new attachment object 1411 is attached to user B2's character object 811b2, the generation unit 271 stops using user A's motion data for a predetermined period (e.g., 5 seconds), extracts the information regarding a clapping motion previously stored in the terminal storage device 22, and causes user A's character object 811a to automatically perform a clapping motion for a predetermined period (e.g., 5 seconds).In this way, even with the same operation, the character object can be made to perform different actions depending on whether it is during normal communication or when a new attachment object is attached to another user's character object, making it possible to add diversity to the output in response to user A's operations during the communication service. Therefore, it is possible to increase user A's motivation to continue using the communication service with user B. Furthermore, by making the character object perform a predetermined action associated with a predetermined operation, the information system 1 can reduce the processing load on the terminal device 2 compared to when motion data is used, and prevent an increase in the communication load between the terminal device 2 and the server device 3.

[0252] (Variation 3) The display processing unit 274 may include a selection button on the communication screen for selecting additional objects to be added to the communication screen. For example, when a selection button is selected by user A through the operation of the input device 24, the display processing unit 274 selects one or more additional objects from a group containing multiple additional objects stored in the terminal storage device 22 based on a predetermined selection method. The predetermined selection method may be a method of randomly selecting from a group containing multiple additional objects, or a method of selecting from a group containing multiple additional objects in a predetermined order. The display processing unit 274 and the transmission unit 272 then execute an additional processing using the object ID and additional instruction of the selected additional object. As a result, user A's character object can change the user output information based on the selected additional object.

[0253] In this case, the attachment button may be included in the communication screen of User B, whose character object is wearing an attachment object associated with the selected additional object. An attachment object associated with an additional object is, for example, an attachment object with the same attributes as the additional object, or an attachment object of the same color as the additional object. In this case, in the object table T2, information indicating the attributes and / or color of each object is stored in association with each object ID. Also, in the user table T1, information indicating the attributes and / or color of each user's character object is stored in association with each user's user ID. Attributes are, for example, the type of object (male or female, etc.). Attributes may also be the type of lottery game from which the object can be obtained. When the attachment button is selected by User B through the operation of the input device 24, the display processing unit 274 identifies the object ID of the selected additional object, and the display processing unit 274 and the transmission unit 272 execute the additional processing using the object ID of the selected additional object and the additional instruction. This allows users to attach attachment objects related to additional objects included in user A's user output image to user B's character object, thereby increasing user A's motivation to continue using the communication feature with user B. Alternatively, an attachment button may be included in user B's communication screen in the user output image where the wallpaper object related to the selected additional object is applied.

[0254] (Modification 4) If user A does not perform a predetermined operation on the communication screen 1310 displaying the user output image 812a including the additional object 1311, the additional object 1311 may continue to be displayed on the communication screen 1310 of user A's terminal device 2. In this case, the additional object 1311 may be changed to a mark image. This allows user A to change their user character with an attachment object or the like corresponding to the additional object 1311 at their preferred timing.

[0255] Furthermore, if the number of additional objects 1311 for which User A has not performed the prescribed operation exceeds a predetermined number, information regarding the additional objects 1311 for which User A has not performed the prescribed operation may be transmitted to each of the terminal devices 2 of other Users B (Users B1 to B3) via the server device 3. In this case, the additional objects 1311 for which User A has not performed the prescribed operation will be displayed on each of the terminal devices 2 of Users B (Users B1 to B3). When User B performs the prescribed operation on an additional object 1311, the input device 24 of User B's terminal device 2 inputs an acceptance instruction, along with the object ID of the additional object 1311, to the terminal processing device 27, on behalf of the input device 24 of User A's terminal device 2. The acceptance process is then executed. As a result, even if User A does not attach the attachment object corresponding to the additional object sent by another User B, another user other than User A can attach the attachment object corresponding to the additional object.

[0256] (Variation 5) When a character object is generated based on 3D model data for generating 3D animation, the user output image for each user may be an image of the character object, which is a three-dimensional object (3D object) placed in a virtual space defined for each user, viewed from a predetermined viewpoint within that virtual space. For example, the display processing unit 274 places a 2D screen in the line of sight direction of a predetermined viewpoint, projects the 3D coordinates of various three-dimensional objects placed in the virtual space onto the 2D screen, and displays the image of the 2D screen surface on which the various objects are projected as the user output image. For example, the line of sight direction of a predetermined viewpoint for displaying the user output image of user A displayed on user A's terminal device 2 is the direction facing user A's character object. The 2D screen for displaying the user output image of user A displayed on user A's terminal device 2 is controlled to move and rotate in accordance with the movement of the predetermined viewpoint and the rotation of the line of sight direction.

[0257] For example, when user A's character object 811a attaches a new attachment object, the communication screen 1110 shown in Figure 11(b) is displayed. Before or after the display of the communication screen 1110, the display processing unit 274 may automatically move a predetermined viewpoint.

[0258] For example, if user A's character object 811a attaches a new attachment object, the display processing unit 274 may move a predetermined viewpoint closer to user A's character object 811a. For example, the display processing unit 274 calculates a path connecting the position of the predetermined viewpoint when user A's character object 811a attaches the new attachment object to the position of the new attachment object, and automatically moves along that path. The display processing unit 274 displays an image showing the virtual space as seen from the predetermined viewpoint that automatically moves within the virtual space as the user output image.

[0259] Figure 17(a) shows an example of a communication screen 1700 where the user output image 1012a is an image showing the virtual space as viewed from a predetermined viewpoint that automatically moves within the virtual space. In the example shown in Figure 17(a), the hairstyle of user A's character object 811a has been changed to a new attachment object, so the user output image 1012a, which includes the enlarged head of the character object 1011a, is displayed.

[0260] For example, when User A's character object 811a attaches a new attachment object, the display processing unit 274 sets a path around the character object 1011a after the attachment. The display processing unit 274 then automatically moves a predetermined viewpoint along the path, aligning the viewpoint's line of sight with the direction facing the character object 1011a. The display processing unit 274 displays an image of the virtual space as seen from the predetermined viewpoint that automatically moves within the virtual space as the user output image.

[0261] Figure 17(b) shows another example of the communication screen 1710, where the user output image 1012a is an image showing the virtual space as viewed from a predetermined viewpoint that automatically moves within the virtual space. In the example shown in Figure 17(a), the user output image 1012a is displayed when moving around the character object 1011a of user A, and the user output image 1012a including the back of the character object 1011a is displayed.

[0262] Furthermore, the display processing unit 274 may control the movement of a predetermined viewpoint and / or the rotation of the line of sight of a predetermined viewpoint in response to a predetermined operation of the input device 24 by user A on the communication screen. The display processing unit 274 may also include an automatic movement button on the communication screen to automatically move a predetermined viewpoint. In this case, for example, if the automatic movement button is selected by user A through an operation of the input device 24, the display processing unit 274 may automatically move the predetermined viewpoint according to a predetermined movement rule.

[0263] (Experimental variation 6) When a character object is generated based on 3D model data for generating 3D animation, each user's user output image may be an image of the virtual space viewed from the user's viewpoint, which is positioned at a predetermined location (for example, the eye position of the character object) within the character object, which is a three-dimensional object (3D object) placed in a virtual space common to each user. The user's viewpoint may move from the predetermined location of the character object, in response to user operations and / or automatically, to be positioned behind, above, or in front of the character object, etc.

[0264] An example of how to use a communication service in a virtual space common to all users will be described. For example, as shown in Figure 18, the display processing unit 274 makes a virtual device object S, such as a tablet PC, appear within the field of view of the user A's character object C in the virtual space. Various screens described in Figures 6-15 and 17 are displayed on the display device portion of this device object S. In this case, the user output image of user A displayed on the display device portion of the device object S is a two-dimensional image of the virtual space as seen from a virtual camera installed at a predetermined position on the device object S. User A's voice data is voice data acquired by the microphone 26 of the terminal device 2.

[0265] The transmission unit 272 of terminal device 2 transmits User A's user output image, User A's voice data, and User A's user ID to all User B's terminal devices 2 via server device 3. Alternatively, the transmission unit 272 may transmit the virtual camera's position and gaze direction in the virtual space, the position and body orientation of User A's character object in the virtual space, and User A's motion data instead of User A's user output image. Furthermore, User A's terminal device 2 receives each User B's user output image, each User B's voice data, and each User B's user ID from each User B's terminal device 2 via server device 3. The display processing unit 274 of User A's terminal device 2 then displays each User B's user output image on the display device portion of the device object S, and outputs each User B's voice data from the terminal device 2. This makes it possible to realize a communication service using the virtual device object S.

[0266] (Example 7) At least a portion of the functions implemented by the terminal processing unit 27 of terminal device 2 may be implemented by a processor other than the terminal processing unit 27 of terminal device 2. At least a portion of the functions implemented by the terminal processing unit 27 may be implemented, for example, by the server processing unit 33 of server device 3. Specifically, some of the functions of the generation unit 271 and the display processing unit 274 may be executed in server device 3. For example, terminal device 2 transmits continuously acquired image data and audio data to server device 3 along with the user ID of the user operating terminal device 2. Server device 3 then uses the functions of the generation unit 271 and the display processing unit 274 to generate character video data and generate display information for displaying a communication screen including user output images for all users on the display device 23. Server device 3 then transmits the display information along with audio information (audio data) to each terminal device 2 of all users. Each terminal device 2 outputs the information received from server device 3 (so-called "server rendering").

[0267] (Variation 8) Terminal device 2 has the function of displaying communication screens, etc., by executing various commands contained in the control program stored in terminal device 2. In order to realize communication services, terminal device 2 may call a browser function embedded in a web application, and using this browser function, receive a web page (a document written in a markup language such as HTML (Hyper Text Markup Language)) from the server device and execute the control program embedded in this web page (so-called "browser rendering"). For example, by using HTML 5 as the markup language, terminal device 2 can easily perform new information processing. By adopting such a web application to realize communication services in the terminal device, the program creator can provide new information processing to the client side (terminal device side) simply by embedding the new program in the web page transmitted by the server device, and the man-hours required to create a new program can be greatly reduced. Furthermore, on the client side, new services can be provided simply by receiving a web page without downloading a new control program, which reduces the load on the communication network, communication costs and / or communication time compared to downloading a control program, and also simplifies the user interface.

[0268] (Extreme variation 9) The generation unit 271 of the terminal device 2 of User A may generate output information including face motion data without using imaging data. For example, the terminal storage device 22 of the terminal device 2 stores face motion data corresponding to voice data, and the generation unit 271 extracts the face motion data corresponding to the acquired voice data of User A from the terminal storage device 22, and generates output information including the extracted face motion data, the voice data, and the user ID of User A. Further, the generation unit 271 of the terminal device 2 of User A may generate output information including the voice data of User A output from the microphone 26 and the user ID stored in the terminal storage device 22 without including the face motion data. In this case, the display processing unit 274 of the terminal device 2 of User B may extract the face motion data corresponding to the voice data of User A included in the output information of User A received via the server device 3, and generate an animation of the character object of User A. Note that the generation process of the face motion data corresponding to the voice data of the user may be a process using a known lip sync algorithm.

[0269] In this way, in the terminal device 2 of User A using the HMD as the display device 23, it is possible to communicate with User B via the character object without acquiring the imaging data of User A.

Explanation of Signs

[0270] 1 Information system 2 (2a, 2b1, 2b2) Terminal device 21 Terminal communication I / F 22 Terminal storage device 23 Display device 24 Input device 25 Imaging device 26 Microphone 27 Terminal processing device 271 Generation unit 272 Transmission unit 273 Reception unit 274 Display processing unit 275 Voice output unit 3 Server equipment 31 Server Communication Interface 32 Server Storage Devices 33 Server Processing Units 331 Server receiving unit 332 Server Transmission Unit 4 base station 5 Mobile communication network 6 Gateways 7. Internet

Claims

1. A control program for the terminal device of the first user, On the terminal device of the first user, At least two images are displayed: a first image showing the virtual space as viewed from a predetermined viewpoint within the virtual space, which includes a first object representing the first user; and a second image including a second object representing a second user different from the first user. Information regarding the display of the first image is transmitted to the terminal device of the second user. When the first voice of the first user is acquired, the first voice is transmitted to the terminal device of the second user. The first image is displayed, which includes the first object that changes in response to the actions of the first user or the first sound. When the second voice of the second user is received, the second voice is output. The second image is displayed, which includes the second object that changes in response to the actions of the second user or the second voice. Displaying the first image or the second image, which has been modified in at least part, in response to instructions from the first user or the second user. Make it run, In the display of the first image, in which at least a part has been modified, the terminal device of the first user, Displaying the first image showing the virtual space as seen from the viewpoint, which has been moved in accordance with a change in a part of the first object. A control program to execute the program.

2. The control program according to claim 1, wherein the predetermined viewpoint is controlled to move around the first object while aligning the line of sight from the predetermined viewpoint with the direction toward the first object.

3. The control program according to claim 1, wherein the viewpoint movement control calculates a path connecting the predetermined viewpoint position and the modified part, and moves the viewpoint along the said path.

4. The control program according to claim 1, wherein the viewpoint movement control involves setting a path for the viewpoint around the first object and moving the viewpoint along the path while aligning the line of sight direction with the direction toward the first object.

5. In the display of the first image, in which at least a part has been modified, the terminal device of the first user, If predetermined conditions are met, including conditions relating to the display time of the first image and / or the display time of the second image, a specific second candidate object is added to a selection group containing multiple second candidate objects, or some of the second candidate objects are removed from the selection group. Display the first image which includes a third object that corresponds to the second candidate object selected from the selection group in accordance with the instructions of the second user. A control program according to claim 1, which causes the following to be executed.

6. In the display of the first image, in which at least a part has been modified, the terminal device of the first user, Displaying a first image that includes a modified first object based on a first candidate object selected from one or more first candidate objects associated with the first user in accordance with the instructions of the first user, A control program according to claim 1, which causes the following to be executed.

7. The first user can instruct the selection of the third object, On the terminal device of the first user, In response to the selection instruction of the third object by the first user, the first image is displayed, which includes the first object that has been partially modified. The control program according to claim 5, which causes the following to be executed.

8. In the display of the first image, in which at least a part has been modified, the terminal device of the first user, Displaying an image showing the progress of the game played by the first user and / or the second user in place of at least a portion of the first image. A control program according to any one of claims 1 to 7, which causes the following to be executed.

9. On the terminal device of the first user, If the second candidate object selected from a selection group containing multiple second candidate objects in response to instructions from the second user corresponds to a game, then an image showing the progress of the game played by the first user and / or the second user is displayed in place of at least a portion of the second image. A control program according to any one of claims 1 to 7, which causes the following to be executed.

10. The control program according to claim 1, wherein displaying the first image means displaying the first image showing the virtual space as seen from a viewpoint that has been controlled to move closer to the modified part in response to a modification of the part of the first object.

11. A control program for a server device capable of communicating with a terminal device of a first user and a terminal device of a second user different from the first user, Information regarding the display of a first image, which shows the virtual space as viewed from a predetermined viewpoint within the virtual space and includes a first object representing the first user, is received from the terminal device of the first user, and information regarding the display of a second image, which includes a second object representing the second user, is received from the terminal device of the second user. Information for displaying the first image is transmitted to at least the terminal device of the second user, and information for displaying the second image is transmitted to at least the terminal device of the first user. When the first voice of the first user is received, the first voice is transmitted to the terminal device of the second user. Information for displaying the first image, which includes the first object that changes in response to the actions of the first user or the first voice, is transmitted to at least the terminal device of the second user. When the second voice of the second user is received, the second voice is transmitted to the terminal device of the first user. Information for displaying the second image, which includes the second object that changes in response to the actions of the second user or the second voice, is transmitted to at least the terminal device of the first user. In response to instructions from the first user, information for displaying the first image, which has been modified in at least part, is transmitted to at least the terminal device of the second user. In response to instructions from the second user, information for displaying the second image, which has been modified in at least part, is transmitted to at least the first user's terminal device. Make it run, In the display of the first image, in which at least a part has been modified, the server device, Information for displaying the first image showing the virtual space as seen from the viewpoint, which has been moved in response to a change in a part of the first object, is transmitted to at least the first user's terminal device. A control program characterized by causing the execution of a certain action.

12. The terminal device of the first user, At least two images are displayed: a first image showing the virtual space as viewed from a predetermined viewpoint within the virtual space, which includes a first object representing the first user; and a second image including a second object representing a second user different from the first user. Information regarding the display of the first image is transmitted to the terminal device of the second user. When the first voice of the first user is acquired, the first voice is transmitted to the terminal device of the second user. The first image is displayed, which includes the first object that changes in response to the actions of the first user or the first sound. When the second voice of the second user is received, the second voice is output. The second image is displayed, which includes the second object that changes in response to the actions of the second user or the second voice. A processing unit displays the first image or the second image, which has been modified in at least part, in response to instructions from the first user or the second user. Equipped with, The processing unit, in displaying the first image which has been modified in at least part, The first image is displayed showing the virtual space as seen from the viewpoint, which has been moved in accordance with a change in a part of the first object. A terminal device characterized by the following features.

13. A server device capable of communicating with the terminal device of a first user and the terminal device of a second user different from the first user, Information regarding the display of a first image, which shows the virtual space as viewed from a predetermined viewpoint within the virtual space and includes a first object representing the first user, is received from the terminal device of the first user, and information regarding the display of a second image, which includes a second object representing the second user, is received from the terminal device of the second user. Information for displaying the first image is transmitted to at least the terminal device of the second user, and information for displaying the second image is transmitted to at least the terminal device of the first user. When the first voice of the first user is received, the first voice is transmitted to the terminal device of the second user. Information for displaying the first image, which includes the first object that changes in response to the actions of the first user or the first voice, is transmitted to at least the terminal device of the second user. When the second voice of the second user is received, the second voice is transmitted to the terminal device of the first user. Information for displaying the second image, which includes the second object that changes in response to the actions of the second user or the second voice, is transmitted to at least the terminal device of the first user. In response to instructions from the first user, information for displaying the first image, which has been modified in at least part, is transmitted to at least the terminal device of the second user. A processing unit transmits information for displaying the second image, which has been modified in part or in response to instructions from the second user, to at least the terminal device of the first user. Equipped with, The processing unit, in displaying the first image which has been modified in at least part, Information for displaying the first image showing the virtual space as seen from the viewpoint, which has been moved in response to a change in a part of the first object, is transmitted to at least the terminal device of the first user. A server device characterized by the following features.

14. A method for controlling a terminal device of a first user, The terminal device of the first user, At least two images are displayed: a first image showing the virtual space as viewed from a predetermined viewpoint within the virtual space, which includes a first object representing the first user; and a second image including a second object representing a second user different from the first user. Information regarding the display of the first image is transmitted to the terminal device of the second user. When the first voice of the first user is acquired, the first voice is transmitted to the terminal device of the second user. The first image is displayed, which includes the first object that changes in response to the actions of the first user or the first sound. When the second voice of the second user is received, the second voice is output. The second image is displayed, which includes the second object that changes in response to the actions of the second user or the second voice. Displaying the first image or the second image, which has been modified in at least part, in response to instructions from the first user or the second user. Includes, In the display of the first image, in which at least a part has been modified, the terminal device of the first user, Displaying the first image showing the virtual space as seen from the viewpoint, which has been moved in accordance with a change in a part of the first object. A control method characterized by further comprising the following.

15. A control method for a server device capable of communicating with a terminal device of a first user and a terminal device of a second user different from the first user, Information regarding the display of a first image, which shows the virtual space as viewed from a predetermined viewpoint within the virtual space and includes a first object representing the first user, is received from the terminal device of the first user, and information regarding the display of a second image, which includes a second object representing the second user, is received from the terminal device of the second user. Information for displaying the first image is transmitted to at least the terminal device of the second user, and information for displaying the second image is transmitted to at least the terminal device of the first user. When the first voice of the first user is received, the first voice is transmitted to the terminal device of the second user. Information for displaying the first image, which includes the first object that changes in response to the actions of the first user or the first voice, is transmitted to at least the terminal device of the second user. When the second voice of the second user is received, the second voice is transmitted to the terminal device of the first user. Information for displaying the second image, which includes the second object that changes in response to the actions of the second user or the second voice, is transmitted to at least the terminal device of the first user. In response to instructions from the first user, information for displaying the first image, which has been modified in at least part, is transmitted to at least the terminal device of the second user. In response to instructions from the second user, information for displaying the second image, which has been modified in at least part, is transmitted to at least the first user's terminal device. Includes, In the display of the first image, in which at least a part has been modified, the server device, Information for displaying the first image showing the virtual space as seen from the viewpoint, which has been moved in response to a change in a part of the first object, is transmitted to at least the first user's terminal device. A control method characterized by including

Images