Program, information processing device and system

Abstract

A program, an information processing device, and a system are provided that can display an avatar in an appropriate display mode depending on the overlapping state of parts related to the avatar. [Solution] The program causes a computer to function as a storage means, an acquisition means, a display processing means, and a change means. The storage means stores a plurality of image data corresponding to each part of an avatar, and aspect setting information in which the display aspect of the image data is set according to a combination of the plurality of image data. The acquisition means acquires the plurality of image data and the aspect setting information from the storage means. The display processing means performs display processing of the avatar. When execution of the display processing is started, if parts of the avatar overlap each other in the virtual space, the change means changes the display aspect of the image data corresponding to at least one of the overlapping parts based on the aspect setting information.

Description

[Technical Field]

[0001] The present invention relates to a program, an information processing device, and a system. [Background technology]

[0002] Conventionally, techniques for displaying an avatar having a plurality of body parts have been known.

[0003] For example, Patent Document 1 discloses an avatar that includes a body layer (parts) and a plurality of clothing layers (parts) arranged on the body layer. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2011-129001 Summary of the Invention [Problem to be solved by the invention]

[0005] In the technology described in Patent Document 1, avatar parts may overlap each other. Depending on the design of the avatar, changing the display mode of the overlapping parts may improve the appearance of the avatar. However, in the technology described in Patent Document 1, even when avatar parts overlap each other, the avatar is displayed in the same display mode. As a result, it was not possible to display the avatar in an appropriate display mode depending on the overlapping state of the avatar parts.

[0006] The present invention has been made in consideration of such problems, and its purpose is to provide a program, information processing device, and system that can display an avatar in an appropriate display mode depending on the overlapping state of the avatar's parts. [Means for solving the problem]

[0007] In order to solve the above problem, the program of the present invention causes a computer to function as: a storage means for storing a plurality of image data corresponding to each part of an avatar and aspect setting information in which the display aspect of the image data is set according to the combination of the plurality of image data; an acquisition means for acquiring the plurality of image data and the aspect setting information from the storage means; a display processing means for performing display processing of the avatar; and a change means for, when execution of the display processing is started and parts of the avatar overlap each other in virtual space, changing the display aspect of image data corresponding to at least one of the overlapping parts based on the aspect setting information.

[0008] In another aspect of the program of the present invention, the aspect setting information includes information regarding the anteroposterior order of the placement of the parts in the virtual space, and further functions as a selection means that selects the multiple parts in response to a selection request from a user, and an arrangement means that arranges multiple image data corresponding to the multiple parts selected by the selection means in the virtual space in a anteroposterior order based on the information regarding the anteroposterior order.

[0009] In another aspect of the program of the present invention, the mode setting information further functions as an adjustment means for adjusting parameters related to the display mode when the display mode is set for a combination of the plurality of image data combinations that is the target of changing the display mode, and when a combination for which the mode setting information is set is included in a plurality of image data corresponding to a plurality of parts selected by the selection means.

[0010] In another aspect of the program according to the present invention, the aspect setting information includes an initial value of the parameter, and the adjusting means adjusts the parameter based on the initial value.

[0011] In another aspect of the program according to the present invention, when the adjustment means executes the parameter adjustment process, the display processing means displays an adjustment object that can be operated by the user's operation input, and the adjustment means adjusts the parameter based on the user's operation input to the adjustment object.

[0012] In another aspect of the program of the present invention, the aspect setting information includes information on whether the user can perform the adjustment process, and the display processing means displays the adjustment object related to a combination of the multiple image data for which the user can perform the adjustment process.

[0013] In another aspect of the program of the present invention, the storage means stores part sets composed of the multiple parts for each body part of the avatar, the acquisition means acquires the multiple part sets for each body part of the avatar from the storage means, the selection means selects one part set from the multiple part sets for each body part in response to the selection request, and the placement means places multiple image data corresponding to the multiple parts included in the multiple part sets selected by the selection means in the virtual space in a front-to-back order based on information regarding the front-to-back order.

[0014] In another aspect of the program of the present invention, the modification means modifies the display mode so that, when the parts overlap each other and at least one of the overlapping parts is located closer to the front in the virtual space than the other parts of the overlapping parts, a transparency process is performed on the portion of the image data corresponding to the one part that overlaps with the image data corresponding to the other part based on the mode setting information.

[0015] In another aspect of the program of the present invention, when the parts overlap each other and at least one of the overlapping parts is located closer to the front in the virtual space than another part of the overlapping parts, the change means changes the display mode based on the mode setting information so that the other part is located closer to the front in the virtual space than the one part.

[0016] In another aspect of the program of the present invention, the storage means stores movement data of the avatar that corresponds to the user's movement, the acquisition means acquires video data of the user and acquires the movement data from the storage means, and the display processing means extracts information regarding the user's posture and facial expression from the video data and performs the display processing to move multiple parts of the avatar based on the extracted information and the movement data.

[0017] In another aspect of the program according to the present invention, the change means determines whether the parts overlap each other depending on the movement of the parts, and executes a process for changing the display mode based on the result of the determination.

[0018] The information processing device of the present invention comprises a storage means for storing a plurality of image data corresponding to each part of an avatar and aspect setting information in which the display aspect of the image data is set according to the combination of the plurality of image data, an acquisition means for acquiring the plurality of image data and the aspect setting information from the storage means, a display processing means for performing display processing of the avatar, and a change means for, when execution of the display processing is started, changing the display aspect of image data corresponding to at least one of the overlapping parts based on the aspect setting information if parts of the avatar overlap in virtual space.

[0019] The system of the present invention comprises a storage means for storing a plurality of image data corresponding to each part of an avatar and aspect setting information in which the display aspect of the image data is set according to the combination of the plurality of image data; an acquisition means for acquiring the plurality of image data and the aspect setting information from the storage means; a display processing means for performing display processing of the avatar; and a change means for, when execution of the display processing is started, changing the display aspect of image data corresponding to at least one of the overlapping parts of the avatar based on the aspect setting information, if parts of the avatar overlap in virtual space. [Effects of the Invention]

[0020] According to the present invention, the program, information processing device, and system can display an avatar in an appropriate display mode depending on the overlapping state of parts related to the avatar. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a diagram illustrating an example of the overall configuration of an avatar display system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a server device illustrated in FIG. [Figure 3] FIG. 2 is a diagram illustrating an example of a hardware configuration of the user terminal illustrated in FIG. [Figure 4] 2 is a diagram illustrating an example of a functional configuration of the user terminal illustrated in FIG. 1. FIG. [Figure 5A] 5 is a diagram showing an example of a parts set for each body part of an avatar in the parts information shown in FIG. 4. FIG. [Figure 5B] FIG. 5 is a diagram showing an example of a hair part set among the part sets shown in FIG. 4. [Figure 5C] FIG. 5C is a diagram showing an example of parts constituting the part set shown in FIG. 5B. [Figure 5D] FIG. 5 is an explanatory diagram showing an example of the order in which the parts shown in FIG. 4 are arranged in the virtual space. [Figure 6]5 is a table showing an example of the mode setting information shown in FIG. 4. [Figure 7] 5 is a flowchart showing an example of a processing flow when each functional unit shown in FIG. 4 performs an avatar creation process. [Figure 8] 5 is a flowchart showing an example of a processing flow when each functional unit shown in FIG. 4 changes the display mode of a part. [Figure 9] 5 is a flowchart showing an example of a processing flow when each functional unit shown in FIG. 4 performs avatar distribution processing. [Figure 10A] FIG. 10 is a diagram showing an example of display content when adjusting the front-to-back order of parts when creating an avatar. [Figure 10B] FIG. 10 is a diagram showing an example of display content when adjusting the front-to-back order of parts when creating an avatar. [Figure 10C] FIG. 10 is a diagram showing an example of display content when adjusting the transparency of parts when creating an avatar. [Figure 10D] FIG. 10 is a diagram showing an example of display content when adjusting the transparency of parts when creating an avatar. [Figure 11A] FIG. 10 is a diagram showing an example of display content when an avatar is distributed. [Figure 11B] FIG. 10 is a diagram showing an example of display content when an avatar is distributed. DETAILED DESCRIPTION OF THE INVENTION

[0022] Hereinafter, an embodiment of the present invention (hereinafter referred to as "the present embodiment") will be described with reference to the accompanying drawings. To facilitate understanding of the description, the same components and steps in each drawing will be denoted by the same reference numerals as much as possible, and duplicated descriptions will be omitted.

[0023] <Overall structure> FIG. 1 illustrates an example of the overall configuration of an avatar display system 1 according to this embodiment. The avatar display system 1 is a system capable of creating and distributing avatars that reflect user preferences. The avatar display system 1 displays avatars that are currently being created or have already been created during the avatar creation and distribution processes. An avatar is modeling data for video distribution. An avatar is composed of multiple image data items associated with multiple body parts. As illustrated in FIG. 1, the avatar display system 1 includes, for example, a server device 10, a user terminal 20, a distributor user terminal 30, a distributor server 40, and multiple viewer terminals 50. These devices are communicatively connected via a communication network NT such as an intranet, the Internet, or a telephone line. Note that the user terminal 20 and the distributor user terminal 30 are communicatively connected via wired or wireless communication, separate from the communication network NT. Wired communication is, for example, communication via a Universal Serial Bus (USB) or a wired Local Area Network (LAN) cable. Wireless communication is, for example, communication via a wireless LAN.

[0024] The server device 10 is an information processing device of a service provider that provides a service for generating avatars and distributing videos to one or more viewers using the generated avatars. The server device 10 provides a program for displaying, creating, and distributing avatars to the user terminal 20 via the communication network NT. The server device 10 also provides various data required for executing the program (e.g., avatar data, body part image data, and personal information for identifying the user) to the user terminal 20 via the communication network NT. The generated avatars express facial expressions and movements according to the user's preferences. Examples of videos using avatars include live streaming and on-demand streaming, in which a user who distributes a video has an avatar displayed on a screen perform a performance for one or more viewers. Examples of performances include game commentary, singing, playing an instrument, and dancing. In this embodiment, the video distribution service provided by the distributor of the distributor server 40 is used to distribute the video.

[0025] The user terminal 20 is an information processing device owned by a user. After installing a program received from the server device 10, the user terminal 20 executes the program to provide each user with a program for displaying, creating, and distributing an avatar. Examples of these user terminals 20 include portable communication devices such as mobile phones, smartphones, tablets, and laptops. In this embodiment, the user terminal 20 performs an avatar creation process based on user input, and then generates video data using the created avatar. Furthermore, the user terminal 20 transmits the video data to the distribution user terminal 30.

[0026] The distributing user terminal 30 is an information processing device owned by a user, and performs distribution processing of the avatar created by the user terminal 20. Examples of these user terminals 20 include communication devices such as tablets, notebook computers, and desktop computers equipped with input devices and display devices. Examples of the input devices include keyboards, touch panels, cameras, microphones, etc. Examples of display devices include monitors and touch panels, etc. The distributing user terminal 30 transmits avatar video data transmitted from the user terminal 20 to the distributor server 40 via the communication network NT.

[0027] The distributor server 40 is an information processing device of a distributor that provides a service of distributing video data provided by a user to one or more viewing users. Services provided by the distributor include, for example, a video distribution platform, SNS (Social Networking Service), video calling service, etc. The distributor server 40 distributes the video data sent from the distribution user terminal 30 to the viewer terminal 50 via the communication network NT.

[0028] The viewer terminal 50 is an information processing device of a viewer that views video data distributed by a user. The viewer terminal 50 displays and plays video data distributed from the distributor server 40 via the communication network NT. Examples of these viewer terminals 50 include communication devices such as mobile phones, smartphones, tablets, laptops, and desktop computers.

[0029] In this embodiment, in the avatar creation process, the avatar display system 1 first installs a program from the server device 10 to the user terminal 20 via the communication network NT. Then, the user terminal 20 creates an avatar by causing the installed program to execute the avatar creation process based on user input. However, the flow of the avatar creation process in the avatar display system 1 is not limited to the above.

[0030] In the avatar creation and distribution processes of the avatar display system 1, it is not necessary to install all of the programs in advance from the server device 10 to the user terminal 20 and transmit all of the data in advance. In the avatar creation and distribution processes, the user terminal 20 may receive parts of the programs and various data from the server device 10 via the communication network NT whenever necessary. In other words, the functional configuration of the avatar display system 1, which will be described later, may be realized by a plurality of information processing devices, including the server device 10 and the user terminal 20.

[0031] In this embodiment, in the process of distributing avatar video data, avatar display system 1 first transmits avatar video data from user terminal 20 to distributor user terminal 30. Distributor user terminal 30 transmits the avatar video data to distributor server 40 via communications network NT. Distributor server 40 then transmits the received video data to viewer terminal 50 via communications network NT.

[0032] <Hardware configuration> Fig. 2 is a diagram schematically illustrating an example of a hardware configuration of the server device 10 shown in Fig. 1. As shown in Fig. 2, the server device 10 includes a storage device 11, a CPU (Central Processing Unit) 12, a memory 13, and a communication device 14.

[0033] The storage device 11 stores various programs and information required for the CPU 12 to execute processes, as well as information on the processing results. The CPU 12 functions as various functional means by executing predetermined programs stored in the memory 13 or the storage device 11. The memory 13 temporarily stores predetermined programs and data required for the CPU 12 to execute the predetermined programs. The communication device 14 is composed of a communication interface and the like for communicating with external devices. The communication device 14 transmits and receives various information to and from the user terminal 20, the viewer terminal 50, and the like via the communication network NT, for example.

[0034] The server device 10 can be realized using an information processing device such as a dedicated or general-purpose server computer. The server device 10 may be configured from a single information processing device or from multiple information processing devices distributed over a communication network NT. Fig. 2 shows only a portion of the main hardware configuration of the server device 10, and the server device 10 may also have other configurations that are generally included in a server.

[0035] Fig. 3 is a diagram showing an example of the hardware configuration of the user terminal 20 shown in Fig. 1. As shown in Fig. 3, the user terminal 20 includes a main control unit 21, a touch panel (touch screen) 22, a camera 23, a microphone 24, and a speaker 25. Furthermore, the user terminal 20 includes a mobile communication unit 26, a wireless LAN (Local Area Network) communication unit 27, a GPS (Global Positioning System) receiving unit 28, and a storage device 29.

[0036] The main control unit 21 is configured to include a CPU 21A, a memory 21B, etc. The CPU 21A functions as various functional means by executing predetermined programs stored in the memory 21B or the storage device 29, etc. Details of these functional means will be described later. The memory 21B temporarily stores predetermined programs and data required when the CPU 21A executes the predetermined programs. The main control unit 21 is also connected to a touch panel 22, a camera 23, a microphone 24, a speaker 25, a mobile communication unit 26, a wireless LAN communication unit 27, a GPS receiving unit 28, and a storage device 29. The main control unit 21 has a function to control these connection destinations.

[0037] The touch panel 22 has both a display function and an input function. The touch panel 22 includes a display 22A that performs the display function and a touch sensor 22B that performs the input function. The display 22A can display a screen showing a character avatar, an operation input screen for creating an avatar, a distribution screen for distributing the avatar, etc. The touch sensor 22B is a sensor that can detect the input position of the player relative to the screen.

[0038] The camera 23 has a function of capturing still images, moving images, and the like.

[0039] The microphone 24 has a function of receiving input of sound such as the user's voice.

[0040] The speaker 25 has a function of outputting sounds such as voice, music, and sound effects from the user terminal 20.

[0041] The mobile communication unit 26 is connected to a mobile communication network (not shown) via an antenna 26 A. The mobile communication unit 26 has a function of communicating with other communication devices connected to the mobile communication network.

[0042] The wireless LAN communication unit 27 is connected to the communication network NT via an antenna 27A. The wireless LAN communication unit 27 has a function of communicating with other devices connected to the communication network NT, such as the server device 10, the distribution user terminal 30, the distributor server 40, and the viewer terminal 50.

[0043] The GPS receiver 28 receives radio waves from GPS satellites. The GPS receiver 28 has a function of acquiring location information indicating the current location of the user terminal 20. The location information is, for example, latitude and longitude.

[0044] The storage device 29 stores various programs and information required for the CPU 21A to execute processes, as well as information on the results of processing.

[0045] <Functional configuration> FIG. 4 is a block diagram showing an example of the functional configuration of the user terminal 20 shown in FIG.

[0046] 4, the user terminal 20 includes, as functional components, for example, a storage unit 200, an acquisition unit 210, a display processing unit 220, a change unit 230, and a selection unit 240. The user terminal 20 further includes, as functional components, for example, an arrangement unit 250, an adjustment unit 260, a control unit 270, and a transmission unit 280. The functional components other than the storage unit 200 are realized by the CPU 21A executing a program stored in the storage device 29 or the like. Note that all or part of the above functional units may be included in the server device 10 rather than in the user terminal 20.

[0047] The storage means 200 stores various programs and data required for executing processing by the functional configuration, data related to processing results, images, etc. Specifically, the storage means 200 stores part information 201, appearance setting information 202, created avatars 203, movement data 204, video data 205, and display content 206.

[0048] The part information 201 is information about parts 2011 that constitute an avatar. The part information 201 has a plurality of part sets 2010, each of which includes at least one part 2011, for each part of the avatar. An avatar part is a part of the character represented by the avatar. Each part 2011 is associated with image data for constituting the avatar. A single image data item is not necessarily associated with a single part 2011. A plurality of image data items may be associated with a part 2011 depending on the direction from which the part 2011 is viewed. Specifically, a plurality of image data items may be associated with a part 2011, such as image data items when the part 2011 is viewed from the front, image data items when viewed from the right front, and image data items when viewed from the left front. In this example, three image data items are associated with the part 2011, but the present invention is not limited to this. Two or more image data items may be associated with the part 2011. In addition to image data, the part 2011 may be associated with an identification number for the part 2011 and information indicating which part set 2010 the part 2011 belongs to.

[0049] Furthermore, image data may be associated with the part 2011 for each predetermined angle within the range in which the avatar can move in the virtual space. Specifically, the avatar can move from a 45° angle in a direction turning rightward from the front (rear side) to a 45° angle in a direction turning leftward from the front side, based on the direction from the rear (rear side) to the front (near side). It is also assumed that the predetermined angle is 15°. In this case, a total of seven pieces of image data viewed from angles of 45°, 30°, 15°, 0°, −15°, −30°, and −45° are associated with the part 2011, assuming that the turning direction to the right is positive. The virtual space is the space in which the avatar is placed. A virtual camera is provided in the virtual space to project images within the virtual space. The avatar is displayed on the display content 206 (described later) through the virtual camera. That is, the user of the user terminal 20 views the avatar in the virtual space through the virtual camera. Additionally, the front (near side) is the direction indicating the virtual camera side (i.e., the user's viewpoint side) in the virtual space, or the side expressed as the front side in an image composed of multiple layers. In contrast, the rear (deep side) is the direction indicating the opposite side from the virtual camera, or the side expressed as the back side in an image composed of multiple layers.

[0050] Avatar body parts and part sets 2010 will be described with reference to FIG. 5A. FIG. 5A is a diagram showing an example of part sets 2010 for each avatar body part in part information 201 shown in FIG. 4. As shown in FIG. 5A, examples of avatar body parts include "hair," "face," "eyes," "eyebrows," "nose and mouth," "body," "arms," ​​and "legs." Part sets 2010A to 2010H are examples of part sets 2010 for "hair," "face," "eyes," "eyebrows," "nose and mouth," "body," "arms," ​​and "legs," respectively. Note that part set 2010 shown in FIG. 5A is merely an example, and part set 2010 may be classified into body parts not listed in FIG. 5A. Examples of body parts not listed in FIG. 5A include hats, glasses, and hair ornaments.

[0051] Further, with reference to FIG. 5B , a specific example of part set 2010 will be described. FIG. 5B is a diagram showing an example of hair part set 2010 among part sets 2010 shown in FIG. 4. As shown in FIG. 5B , in this embodiment, part information 201 has a total of four types of part sets 2010 classified as "hair," for example, part sets 2010A, 2010L, 2010M, and 2010N. As shown in FIG. 5B , part information 201 can include part sets 2010 of various hairstyles as hair part sets 2010. Note that while FIG. 5B shows a specific example of hair part set 2010, various types of part sets 2010 are similarly classified for avatar parts other than hair. Note that multiple part sets 2010 do not necessarily have to be classified for each avatar part; only one part set 2010 may be classified for some avatar parts.

[0052] Returning to FIG. 4 , the part set 2010 is a group of parts made up of at least one part 2011. The part set 2010 is classified to belong to one of a plurality of predetermined avatar body parts. At least one part set 2010 belongs to an avatar body part. The part set 2010 includes information about the parts 2011 that make up the part set 2010. Furthermore, when the part set 2010 includes a plurality of parts 2011, the part set 2010 may further include information about the relative positional relationships between the parts 2011 in the virtual space. Specifically, the part set 2010 may include a reference part 2011 among the plurality of parts 2011. Furthermore, in the part set 2010, coordinate values ​​indicating the position of the parts 2011 relative to the reference part 2011 may be defined for the parts 2011 other than the reference part 2011. Furthermore, the number of parts 2011 included in a part set 2010 may tend to be greater when the part of the avatar to which the part set 2010 belongs is on the upper body side than when it is on the lower body side. Specifically, the part set 2010 tends to include more parts 2011 in the "face," "arms and torso," and "legs" order of the avatar parts to which the part set 2010 belongs. The "face" is a part of the avatar's upper body, and includes the avatar parts "hair," "head," "eyes," "eyebrows," and "nose and mouth." The "arms and torso" is a part of the avatar's upper body other than the "face," and includes the avatar parts "body" and "arms." The legs are a part of the avatar's lower body side. In other words, the part set 2010 tends to include more parts 2011 for the "face" part, which is more likely to attract the viewer's gaze, and to include fewer parts 2011 for parts that are less likely to attract the viewer's gaze.

[0053] Here, the relationship between part set 2010 and part 2011 will be described with reference to FIG. 5C. FIG. 5C is a diagram showing an example of part 2011 constituting part set 2010A shown in FIG. 5B. For ease of explanation, FIG. 5C does not show information on the relative positional relationships between parts 2011 in virtual space. As shown in FIG. 5C, part set 2010A, which is classified as the "hair" region, is made up of four parts, 2011A to 2011D. Image data showing bangs is associated with part 2011A. Image data showing back hair is associated with part 2011B. Image data showing hair on the right side of the head is associated with part 2011C. Image data showing hair on the left side of the head is associated with part 2011D. Although part set 2010A is given as an example in FIG. 5C, the same applies to other part sets 2010 such as part set 2010L in FIG. 5B and part set 2010B of a body part other than "hair" in FIG. 5A.

[0054] Returning to FIG. 4 , the appearance setting information 202 is information that is set regarding the display appearance of image data associated with the parts 2011. The appearance setting information 202 includes setting information regarding the display appearance for each combination of image data associated with the parts 2011. In addition to the setting information regarding the display appearance, the appearance setting information 202 further includes information regarding the front-to-back order of the placement positions of the parts 2011 in the virtual space for each image data associated with the parts 2011. Note that the appearance setting information 202 is provided for each created avatar 203 (described later), or the value (current value) of a parameter related to the display appearance is determined for each created avatar 203. Here, details of the appearance setting information 202 will be described with reference to FIG. 6. FIG. 6 is a diagram illustrating an example of the appearance setting information 202 shown in FIG. 4. As shown in FIG. 6 , the appearance setting information 202 includes, for example, at least four pieces of setting information for each combination of parts 2011 (combination of image data). A combination of parts 2011 is composed of two parts 2011. In a combination of parts 2011, information about the first part 2011 is defined for the first part, and information about the second part 2011 is defined for the second part. In this embodiment, the display mode is set in the mode setting information 202 for combinations of parts 2011 that are subject to a change in display mode. That is, in this embodiment, the mode setting information 202 does not set four pieces of setting information for combinations that are not subject to a change in display mode. Information about the front-to-back order of the arrangement positions of the parts 2011 is set for each image data corresponding to the parts 2011, regardless of whether the parts 2011 belong to a combination that is not subject to a change in display mode.

[0055] The four pieces of setting information are, for example, whether or not the user can adjust the parameter, the method for changing the display mode, the initial value of the parameter, and the current value of the parameter. Whether or not the user can adjust the parameter is information indicating whether or not the user can adjust the parameter related to changing the display mode. Specifically, whether or not the user can adjust the parameter indicates "adjustable" or "unadjustable." The method for changing the display mode is information regarding how to change the display mode of the image data. The method for changing the display mode indicates, for example, "transparent" or "change of display order." The initial value of the parameter is the initial value of the parameter adjusted when changing the display mode. The current value of the parameter is the current value of the parameter. The parameter is a value indicating the degree of change of the display mode, and is determined for each method for changing the display mode. When the method for changing the display mode is "transparent," the parameter is, for example, a value between 0 "%" and 100 "%." When the method for changing the display mode is "change of display order," the parameter is, for example, a value indicating "change" or "do not change." In this example, the value indicating "change" is "1." Furthermore, the value indicating "do not change" is "0." The display mode parameters are not limited to the values ​​and contents described above, and may be any information that can be used as a parameter.

[0056] The information regarding the front-to-back order of the arrangement positions of the parts 2011 in the virtual space, which is included in the aspect setting information 202, will be described. As described above, the information regarding the front-to-back order is set for each image data corresponding to the part 2011. The information regarding the front-to-back order is information indicating which part 2011 should be displayed in front (closer) and which part 2011 should be displayed behind (farther back) when multiple parts 2011 overlap each other in the virtual space. The information regarding the front-to-back order is a numerical value in this example. In this embodiment, for example, the smaller the number, the further forward (closer) the part 2011 is displayed, and the larger the number, the further back (farther back) the part 2011 is displayed. To give an example, since the information regarding the front-to-back order of part A is "204", it is displayed behind (farther back) than part B, whose information regarding the front-to-back order is "22". Furthermore, part A is displayed ahead (closer) than part D, whose information regarding the front-to-back order is "860". The information regarding the front-to-back order may not be an absolute value as described above, but may be determined as a relative value between the parts 2011. Specifically, the information regarding the front-to-back order may simply be determined as a value indicating which part 2011 should be displayed in front for each combination of parts 2011.

[0057] Here, with reference to Fig. 5D, the anteroposterior order of the placement positions of the parts 2011 will be described with a specific example. Fig. 5D is an explanatory diagram showing an example of the anteroposterior order when the parts 2011 shown in Fig. 4 are placed in virtual space. In Fig. 5D, the parts 2011 are placed in virtual space from the front (near side) to the rear (deep side) in the anteroposterior order of part 2011A, part sets 2010C and 2010B, and part 2011B. Note that although part sets 2010B and 2010C are each made up of multiple parts 2011, for convenience of explanation, they are treated as part sets 2010B and 2010C in Fig. 5D.

[0058] As shown in FIG. 5D , part 2011A representing bangs is positioned in front (closer) of part 2011B representing back hair in virtual space. In this way, the multiple parts 2011 constituting part set 2010A are positioned in accordance with information relating to the front-to-back order. Furthermore, part set 2010C is positioned in front (closer) of part set 2010B in virtual space. This is an example of a case where a front-to-back order relationship is determined between part sets 2010. Furthermore, part 2011A constituting part set 2010A is positioned in front (closer) of part sets 2010C and 2010B. In contrast, part 2011B constituting part set 2010A is positioned behind (deeper) of part sets 2010C and 2010B. In this way, even if parts 2011 are classified into the same part set 2010, a case where a different front-to-back order is determined relative to other part sets 2010 is also an example. That is, one example is a case where the parts 2011 constituting a part set 2010 are arranged in a chronological order such that parts 2011 constituting other parts sets 2010 are positioned between the parts 2011 constituting the other parts sets 2010.

[0059] Returning to FIG. 4 , created avatar 203 is an avatar created by a user. Created avatar 203 includes information about part sets 2010 that constitute created avatar 203. Specifically, created avatar 203 is configured from image data of part sets 2010 selected as the avatar's body parts: "hair," "face," "eyes," "eyebrows," "nose / mouth," "body," "arms," ​​and "legs." In other words, created avatar 203 includes information about which part set 2010 has been selected. Information about the display mode of created avatar 203 is stored as mode setting information 202 described above.

[0060] The motion data 204 is data used when moving the avatar in accordance with the user's motion captured in the video data 205 (described later). The motion data 204 includes information about the motion of the avatar, and information about the position and shape of each part 2011 constituting the avatar. The information about the motion of the avatar includes, for example, information about facial expression control of the avatar, information about physical simulation, and information about the range of motion of each part 2011 constituting the avatar. The information about facial expression control of the avatar includes, for example, parameters such as joy, anger, sadness, and happiness, and the intensity and duration of facial expressions. The information about physical simulation includes parameters of the physical movements (such as swinging) of clothing and accessories. The information for adjusting the position, shape, etc. of each part 2011 constituting the avatar includes, for example, parameters for adjusting the texture of the avatar and parameters related to the virtual camera. The avatar texture refers to the color, texture, pattern, etc. of the avatar's skin, hair, clothing, etc. The parameters related to the virtual camera include the position and angle of the virtual camera in the virtual space, the viewing angle of the virtual camera, and the tracking ability of the virtual camera.

[0061] The video data 205 is a moving image captured by the user. The video data 205 is captured by the camera 23 and stored in the storage means 200.

[0062] Display content 206 is content that performs avatar creation and distribution processes. Display content 206 is a program or application that runs on the operating system of user terminal 20. In the avatar creation process, display content 206 accepts user operation input to select part set 2010 and adjust the display mode of image data corresponding to part 2011. In the avatar distribution process, display content 206 accepts user operation input to operate created avatar 203 in accordance with video data 205 captured by the user. In the avatar distribution process, display content 206 also performs processing to transmit created avatar 203 currently being displayed to distribution user terminal 30, distributor server 40, etc.

[0063] Here, the display content 206 when creating an avatar will be described in detail with reference to Fig. 10A. Fig. 10A is a diagram showing an example of the display content 206 when adjusting the front-to-back order of parts 2011 when creating an avatar. As shown in Fig. 10A, the display content 206 displays content made up of an avatar display area 2071, buttons 2072 and 2074, and an adjustment object 2073 during the avatar creation process.

[0064] The avatar display area 2071 is an area where an avatar being created is displayed. A virtual space is displayed in the avatar display area 2071. Image data corresponding to avatar parts 2011 is placed or removed in the virtual space of the avatar display area 2071 in response to user input operations, processing by the display content 206, and the like.

[0065] Button 2072 is an object that allows the user to select part set 2010. Button 2072 is provided in display content 206 for each body part of the avatar. In FIG. 10A , a plurality of buttons 2072 are provided corresponding to the avatar's body parts, namely, "hair," "face," "eyes," "eyebrows," "mouth," "body," and "clothes." When button 2072 is pressed by a user's operation input, part set 2010 for the corresponding avatar body part is displayed in display content 206. In response to a selection operation by the user on displayed part set 2010, display content 206 arranges the selected part set 2010 in avatar display area 2071. Note that in FIG. 10A , button 2072 corresponding to the eyebrow body part has been pressed.

[0066] Adjustment object 2073 is an object for adjusting parameters of the display mode of avatar part 2011. In FIG. 10A, adjustment object 2073 adjusts the display position of part 2011 belonging to part set 2010D, which is classified as the "eyebrow" region and corresponds to pressed button 2072. Adjustment object 2073 is provided in display content 206. Also in FIG. 10A, adjustment object 2073 has two check boxes for adjusting the display position of part set 2010D. In FIG. 10A, the check boxes of adjustment object 2073 are checked so that part 2011 belonging to part set 2010D is displayed below (i.e., on the far side of) bangs part 2011A. As a result, part set 2010D is displayed so as to be farther back than part 2011A.

[0067] Button 2074 is an object for the user to request completion of the avatar he or she is currently creating. When button 2074 is pressed by the user's operation input, the avatar currently being created is stored in storage means 200 as created avatar 203. In addition, current appearance setting information 202 is stored in storage means 200 in association with created avatar 203.

[0068] Further, with reference to Fig. 11A and Fig. 11B, details of display content 206 when an avatar is distributed will be described. Fig. 11A and Fig. 11B are diagrams showing an example of display content 206 when an avatar is distributed. As shown in Fig. 11A, display content 206 displays avatar display area 2071 and buttons 2075 to 2077 during the avatar distribution process. Note that in Fig. 11A, the avatar includes at least parts 2011L, 2011M, 2011N, 2011P, and 2011Q.

[0069] The avatar to be distributed is displayed in avatar display area 2071. Each part 2011 constituting the avatar displayed in avatar display area 2071 moves in accordance with motion data 204 and video data 205 captured of the user during tracking processing.

[0070] Button 2075 is an object for switching between starting and stopping the tracking process described above. Pressing button 2075 switches between starting and stopping the user's image capture process and the tracking process in which the avatar moves in conjunction with video data 205 captured by the user. As shown in FIG. 11B , the image data and movement data 204 of each part 2011 of the avatar are set so that the avatar character can turn left or right when viewed from the front. As shown in FIG. 11B , during the tracking process, each part 2011 of the avatar moves in conjunction with video data 205 captured by the user. During the tracking process, the overlapping portions of each part 2011 change in accordance with the movement of the parts 2011. Therefore, the display mode of the image data of each part 2011 is updated at predetermined intervals, for example, in accordance with mode setting information 202.

[0071] Button 2076 is an object for switching the content displayed by display content 206 from content related to avatar distribution to content related to avatar creation. When button 2076 is pressed, the display of content related to the distribution process shown in Fig. 11A is stopped, and the display of content related to the avatar creation process as shown in Fig. 10A above is started.

[0072] The button 2077 is an object for switching between starting and stopping the distribution of the avatar. When the button 2077 is pressed, the distribution of the avatar displayed in the avatar display area 2071 is switched between starting and stopping.

[0073] Returning to FIG. 4 , the acquisition means 210 acquires various data stored in the storage means 200 from the storage means 200. Specifically, in the avatar creation process, the acquisition means 210 acquires from the storage means 200 a plurality of image data corresponding to each part 2011 of the avatar, appearance setting information 202, and display content 206. In addition, in the avatar distribution process, the acquisition means 210 acquires the created avatar 203, movement data 204, video data 205, and display content 206. Note that the acquisition means 210 may acquire the video data 205 captured by the camera 23 of the user terminal 20 from the camera 23, rather than from the storage means 200. In addition, the acquisition means 210 acquires control commands, processing requests, and the like included in a user's operation input received by an input device such as the touch sensor 22B or the microphone 24, from the input device.

[0074] The display processing means 220 executes a process for displaying an avatar. The display processing means 220 also executes a process for displaying the display content 206. The display processing means 220 also executes a process for displaying an adjustment object 2073 for adjusting parameters of the display mode of a part 2011 selected by the selection means 240 (described later). Specifically, the display processing means 220 refers to the mode setting information 202 and determines whether or not parameters related to the display mode of the part 2011 selected by the selection means 240 are adjustable by the user. If the determination is affirmative, the display processing means 220 displays an adjustment object 2073 for adjusting the display mode of the part 2011. As described with reference to FIG. 10A , the adjustment object 2073 is an object that is provided on the display content 206 and can be turned on or off or whose value can be adjusted. Examples of objects that can be switched on or off include a button or a check box that can be switched between two states. Examples of objects with adjustable values ​​include a select button, a seek bar, a dial-like object that can be rotated, a list box, a text box, etc. The display processing means 220 displays these avatars and display content 206 on the display 22A of the user terminal 20 through display processing.

[0075] Furthermore, in the avatar distribution process, the display processing means 220 moves each part 2011 of the avatar in conjunction with video data 205 captured by the user. Specifically, the display processing means 220 extracts information about the user's posture and display from the video data 205 captured by the user. Next, the display processing means 220 determines the angle at which the part 2011 is viewed from the front in the virtual space, based on the extracted information about the user's posture and display and the movement data 204. Furthermore, the display processing means 220 selects one piece of image data corresponding to the determined angle from multiple pieces of image data previously associated with the part 2011. Then, the display processing means 220 displays the selected piece of image data.

[0076] When parts 2011 relating to an avatar overlap one another in virtual space, the change means 230 changes the display mode of image data associated with at least one of the overlapping parts 2011. The change means 230 executes the process of changing the display mode at the start of the execution of the avatar display process. The change means 230 also changes the display mode of the image data based on the mode setting information 202 set in the image data of the overlapping parts 2011. As described in the description of the mode setting information 202, examples of the process of changing the display mode include making the foreground part 2011 transparent and changing the order of the parts in front.

[0077] Furthermore, when the parts 2011 overlap in the virtual space, the change means 230 changes the display mode so as to perform transparency processing on the portion of the front part 2011 that overlaps with the back part 2011. Furthermore, when the parts 2011 overlap in the virtual space, the change means 230 changes the display mode so that the back part 2011 is positioned closer to the front than the front part 2011. Furthermore, the change means 230 determines whether the parts 2011 overlap with each other in accordance with the movement of the parts 2011, and executes processing to change the display mode based on the result of the determination. The patterns of the processing to change the display mode are as described above. Here, a specific example will be given of a pattern in which the overlapping of the parts 2011 changes in accordance with the movement of the parts 2011. For example, it is assumed that the user is facing forward, and the "eyebrow" part 2011 and the "bangs" part 2011 of the avatar do not overlap. Here, when the user moves their head to look to the right, each of the avatar's features 2011 moves in conjunction with the video data 205 captured by the user, as described above. Specifically, the display processing means 220 selects one image data item for each of the avatar's features 2011, namely, "hair," "face," "eyebrows," "eyes," and "nose and mouth," that corresponds to the case where the user is looking to the right. Next, the display processing means 220 displays the newly selected image data item for each of the avatar's features 2011, namely, "hair," "face," "eyebrows," "eyes," and "nose and mouth." At this time, it is assumed that the avatar's "bangs" feature 2011 changes to a state where a part of the avatar's feature 2011 overlaps with the "brows" feature 2011. Next, the change means 230 determines that the avatar's "bangs" feature 2011 and "brows" feature 2011 overlap with each other. The modification means 230 extracts an area of ​​the "bangs" part 2011 of the avatar that overlaps with the "eyebrows" part 2011. Then, the modification means 230 executes a process of modifying the display mode of the extracted area of ​​the "bangs" part 2011, for example, to increase the transparency of the "eyebrows" part 2011.

[0078] The selection means 240 selects a plurality of parts 2011 in response to a selection request from the user in the avatar creation process. Specifically, when selecting parts 2011, the selection means 240 selects one part set 2010 from a plurality of part sets 2010 for each body part of the avatar. At least one part 2011 belongs to the selected part set 2010. That is, the selection means 240 selects a part set 2010 in response to the selection request from the user, thereby selecting the part 2011 belonging to the part set 2010. The selection request from the user is acquired by the acquisition means 210 from an operation input by the user to the touch sensor 22B.

[0079] In the avatar creation process, the arrangement means 250 arranges the image data associated with the parts 2011 selected by the selection means 240 in the virtual space in a chronological order based on the chronological order information. Specifically, the arrangement means 250 references the chronological order information corresponding to the parts 2011 belonging to the part set 2010 in accordance with the selection process of the part set 2010 by the selection means 240. Furthermore, if there is a part 2011 that has already been arranged, the arrangement means 250 also references the chronological order information corresponding to the already arranged part 2011. As described above, the chronological order information is included in the aspect setting information 202. The arrangement means 250 arranges the image data corresponding to each part 2011 belonging to the part set 2010 in the virtual space in the chronological order indicated by the chronological order information that it has referenced.

[0080] When the image data corresponding to the part 2011 selected by the selection means 240 includes a combination for which the aspect setting information 202 is set, the adjustment means 260 adjusts parameters related to the display aspect of the image data. The parameters related to the display aspect are as described above. The adjustment means 260 first adjusts the parameters related to the display aspect based on the initial values ​​of the parameters included in the aspect setting information 202. Next, when the adjustment object 2073 is being displayed, the adjustment means 260 adjusts the parameters of the display aspect of the part 2011 based on the user's operation input to the adjustment object 2073.

[0081] Here, adjustment of display mode parameters will be described with reference to Fig. 10B to Fig. 10D. Fig. 10B is a diagram showing an example of display content 206 when adjusting the front-to-back order of parts 2011 when creating an avatar. In Fig. 10B, the checkbox of adjustment object 2073 is checked so that part 2011 belonging to part set 2010D is displayed above (i.e., in front of) bangs part 2011A. As a result, the display mode of part set 2010D is changed so that it is displayed in front of part 2011A.

[0082] FIG. 10C is a diagram illustrating an example of display content 206 when adjusting the transparency of part 2011 during avatar creation. FIG. 10D is a diagram illustrating an example of display content 206 when adjusting the transparency of part 2011 during avatar creation. Specifically, FIGS. 10C and 10D illustrate an example of adjusting the transparency of part set 2010I representing glasses relative to part set 2010C representing eyes. In FIGS. 10C and 10D, it is assumed that part 2011 belonging to part set 2010I representing glasses is positioned closer to the viewer than part 2011 belonging to part set 2010C representing eyes. In FIGS. 10C and 10D, it is assumed that button 2072 representing clothing has been pressed, and part set 2010C representing glasses, which is classified as an accessory among clothing, has been selected. In FIGS. 10C and 10D, adjustment object 2073 is provided as a seek bar in display content 206. 10C and 10D, the adjustment object 2073 adjusts the transparency, which indicates the degree of transparency of image data corresponding to the parts 2011 belonging to the part set 2010I, by the position of the seek bar. In FIG. 10C, the seek bar of the adjustment object 2073 is positioned near the left end so that the transparency of the eyeglasses part set 2010I is low relative to the eye part set 2010C. This adjusts the display mode of the eyeglasses part set 2010I so that it is almost opaque to the eye part set 2010C and does not reflect the eye part set 2010C. In contrast, in FIG. 10D, the seek bar of the adjustment object 2073 is positioned near the right end so that the transparency of the eyeglasses part set 2010I is high relative to the eye part set 2010C. This adjusts the display mode of the eyeglasses part set 2010I so that it is transparent to the eye part set 2010C and reflects the eye part set 2010C.

[0083] 10A to 10D, the adjustment object 2073 displays a check box when adjusting the front-to-back order and a seek bar when adjusting the transparency, but this is not limited to this. The adjustment object 2073 may display an object that allows selection of adjustment values ​​from predetermined values ​​for the front-to-back order and transparency. Examples of selectable objects include radio buttons and pull-down menus. A case in which the adjustment object 2073 is a radio button will be described using a specific example of a part 2011 in the part set 2010 belonging to the "hair" region and a part 2011 in the part set 2010 belonging to the "eyebrows." The adjustment object 2073 may display a radio button that allows selection of one of three options: "transparent," "not transparent," and "displayed over hair." When "transparent" is selected from the radio buttons, the adjustment unit 260 adjusts the transparency parameter of the part 2011 in the "hair" region for the part 2011 in the "eyebrows" region to, for example, "70%." As a result, the part 2011 of the "eyebrow" part becomes visible through the part 2011 of the "hair" part. Furthermore, when "No transparency" is selected from the radio buttons, the adjustment means 260 adjusts the transparency parameter of the part 2011 of the "hair" part with respect to the part 2011 belonging to the "eyebrow" part to a value such as "0%". As a result, the part 2011 of the "eyebrow" part is hidden by the part 2011 of the "hair" part and becomes invisible. Furthermore, when "Display above hair" is selected from the radio buttons, the adjustment means 260 changes the front-to-back order so that the part 2011 of the "eyebrow" part is displayed in front of (closer to) the part 2011 belonging to the "hair" part. As a result, the part 2011 of the "eyebrow" part is displayed above (closer to) the part 2011 of the "hair" part.

[0084] 4, the control means 270 controls the operation of the camera 23, microphone 24, and touch panel 22 in the user terminal 20. The control means 270 also controls the display process of the display content 206 and the flow of the processes executed by the above-mentioned functional means.

[0085] The transmission means 280 transmits the created avatar 203. Specifically, the transmission means 280 transmits the created avatar 203 displayed in the display content 206 to the distribution user terminal 30. Here, the created avatar 203 may be in a state where the tracking process is being executed and each part 2011 is moving, or may be in a state where the tracking process is being stopped and each part 2011 is not moving.

[0086] <Processing flow> The functional configuration of the user terminal 20 has been described above. Next, a specific processing flow in the user terminal 20 will be described in detail. FIG. 7 is a flowchart showing an example of the processing flow when the functional means shown in FIG. 4 perform avatar creation processing. In FIG. 7, it is assumed that the user terminal 20 displays the display content 206 on the display 22A of the user terminal 20 using the display processing means 220. Note that the content and order of the processing of the following steps can be changed as appropriate.

[0087] (Step SP10) The acquiring means 210 acquires the part information 201 and the mode setting information 202 from the storage means 200. As described above, the various data such as the part information 201 and the mode setting information 202 are not limited to being stored in the storage means 200. The acquiring means 210 may acquire these various data from the server device 10 via the communication network NT, instead of from the storage means 200. Then, the processing proceeds to the processing of step SP12.

[0088] (Step SP12) The selection means 240 determines whether or not there is a request from the user to select a part 2011. Specifically, the selection means 240 determines whether or not there is a request to select a part 2011 from the user's operation input acquired by the acquisition means 210. The operation input for selecting a part 2011 is executed, for example, by selecting an avatar body part and selecting one part set 2010 from multiple part sets 2010 belonging to the selected avatar body part. As described above, this is because the part set 2010 is classified into one of multiple defined avatar body parts, and at least one part 2011 belongs to the part set 2010. If the determination is affirmative, the process proceeds to step SP14. On the other hand, if the determination is negative, the process proceeds to step SP24.

[0089] (Step SP14) The selection means 240 refers to the parts information 201 and selects one parts set 2010 that complies with the user's selection request from the multiple parts sets 2010 included in the parts information 201. Then, the process proceeds to step SP16.

[0090] (Step SP16) The arrangement means 250 arranges in the virtual space the parts 2011 belonging to the part set 2010 selected by the selection means 240. When arranging the parts 2011, the arrangement means 250 refers to information in the aspect setting information 202 regarding the order of the already arranged parts 2011 and the parts 2011 to be arranged. The arrangement means 250 arranges the parts 2011 so that the order conforms to the information regarding the order that it has referred to. After arranging the parts 2011 belonging to the selected part set 2010 in the virtual space, the arrangement means 250 may change the order of the already arranged parts 2011 so that the order conforms to the information regarding the order. The display processing means 220 displays the parts 2011 that have been arranged in the virtual space by the arrangement means 250. Then, the process proceeds to step SP18.

[0091] (Step SP18) The change means 230 determines whether or not the image data corresponding to the parts 2011 already arranged in the virtual space includes a combination of image data that is the target of a change in the display mode. Specifically, the change means 230 extracts a pattern of the combination of image data that corresponds to the parts 2011 already arranged in the virtual space. The change means 230 references the mode setting information 202 and determines whether or not the extracted pattern of the combination of image data is set in the mode setting information 202. If the determination is affirmative, the process proceeds to step SP20. On the other hand, if the determination is negative, the process proceeds to step SP24.

[0092] (Step SP20) The adjustment means 260 adjusts the display mode parameters for each combination of image data that is the target of a change in display mode, among the image data corresponding to the parts 2011 already arranged in the virtual space. The adjustment means 260 refers to the mode setting information 202 and adjusts the display mode parameters so that the parameters correspond to the combination of image data that is the target of the change are the initial values. Then, the processing proceeds to step SP22.

[0093] (Step SP22) The change means 230 performs a process of changing the display mode for each combination of parts 2011 that have been placed in the virtual space. Then, the process proceeds to the process of step SP24. Details of the process of step SP22 will now be described with reference to FIG. 8. FIG. 8 is a flowchart showing an example of the process flow when each functional means shown in FIG. 4 changes the display mode of the parts 2011. Note that the content and order of the processes of the following steps can be changed as appropriate.

[0094] (Step SP100) The change means 230 selects one combination of image data whose overlap state has not been confirmed from the image data corresponding to the parts 2011 arranged in the virtual space, and then the process proceeds to step SP102.

[0095] (Step SP102) The change means 230 refers to the mode setting information 202 and determines whether or not to execute a change process of the display mode for the selected combination of image data. Specifically, the change means 230 determines to execute a change process of the display mode if the selected combination of image data is included in the mode setting information 202. Furthermore, the change means 230 determines not to execute a change process of the display mode if the selected combination of image data is not included in the mode setting information 202. If the determination is affirmative, the process proceeds to step SP104. On the other hand, if the determination is negative, the process proceeds to step SP106.

[0096] (Step SP104) The change means 230 changes the display mode of the area where the image data overlaps each other in the virtual space in the selected combination of image data. Specifically, the change means 230 first extracts the area where the image data overlaps each other when viewed from the front side of the virtual space. Next, the change means 230 changes the display mode of the overlapping area so that it becomes the display mode set for the image data combination in the mode setting information 202. Note that if the selected combination of image data does not have any overlapping areas in the virtual space, the change means 230 may or may not change the display mode of the image data. Then, the processing proceeds to the processing of step SP106.

[0097] (Step SP106) The change means 230 determines whether there is a combination of image data for which the overlap state has not been confirmed for the image data corresponding to the parts 2011 arranged in the virtual space. If the determination is affirmative, the process returns to the process of step SP100. On the other hand, if the determination is negative, the series of processes shown in FIG. 8 ends.

[0098] The details of the process for changing the display mode have been explained above. Returning to Fig. 7, the process going to step SP24 will now be explained. (Step SP24) The display processing means 220 determines whether the image data corresponding to the parts 2011 already arranged in the virtual space includes a combination whose display mode can be adjusted by the user. The display processing means 220 makes this determination by referring to the mode setting information 202. If the determination is affirmative, the process proceeds to step SP26. On the other hand, if the determination is negative, the process proceeds to step SP34.

[0099] (Step SP26) The display processing means 220 displays adjustment objects 2073 for parts 2011 whose display mode of image data can be adjusted by the user, among the parts 2011 already arranged in the virtual space. If there are multiple parts 2011 whose display mode can be adjusted by the user, the display processing means 220 also simultaneously displays a switching object that switches the display of the adjustment objects 2073 for each piece of image data. Then, the processing proceeds to step SP28.

[0100] (Step SP28) The adjustment means 260 determines whether or not a user operation input has been made to the adjustment object 2073. Specifically, the adjustment means 260 refers to the user operation input acquired by the acquisition means 210, and determines whether or not an operation has been made to the adjustment object 2073. If the determination is affirmative, the process proceeds to step SP30. On the other hand, if the determination is negative, the process proceeds to step SP36.

[0101] (Step SP30) The adjustment means 260 adjusts the parameters relating to the representation of the image data corresponding to the adjustment object 2073 based on the user's operation input to the adjustment object 2073. Then, the process proceeds to step SP32.

[0102] (Step SP32) The change means 230 performs a process of changing the display mode for each combination of parts 2011 that have been placed in the virtual space. Details of the process of step SP32 are the same as those of step SP20 and have been explained with reference to Figure 8, so a detailed explanation will be omitted here. Then, the process proceeds to the process of step SP36.

[0103] (Step SP34) The display processing means 220 hides the adjustment object 2073. Specifically, if the adjustment object 2073 or the switching object is displayed, the display processing means 220 hides the displayed adjustment object 2073 and the switching object. If neither the adjustment object 2073 nor the switching object is displayed, the display processing means 220 omits the execution of the process for hiding the adjustment object 2073. Then, the process proceeds to step SP36.

[0104] (Step SP36) Control means 270 determines whether or not there is a completion request from the user to complete the avatar in the process of being created as created avatar 203. The completion request is made, for example, by operating an input such as pressing a button provided on display content 206. If the determination is affirmative, the process proceeds to step SP38. On the other hand, if the determination is negative, the process returns to step SP12.

[0105] (Step SP38) Control means 270 considers the avatar being created to be complete, and stores the information on parts set 2010 selected in the processing of steps SP10 to SP36 in storage means 200 as created avatar 203. Control means 270 also associates current appearance setting information 202 with created avatar 203. Then, the series of processing flows shown in FIG. 7 ends.

[0106] The avatar creation process has been described above. Next, the avatar distribution process will be described with reference to FIG. 9. FIG. 9 is a flowchart showing an example of the processing flow when the functional means shown in FIG. 4 perform the avatar distribution process. In FIG. 9, it is assumed that the user terminal 20 is displaying the display content 206 on the display 22A of the user terminal 20 using the display processing means 220. Note that the content and order of the processing of the following steps can be changed as appropriate.

[0107] (Step SP200) The acquiring means 210 acquires the created avatar 203, the behavior setting information 202, and the motion data 204 from the storage means 200. Note that the various data such as the created avatar 203, the behavior setting information 202, and the motion data 204 are not limited to being stored in the storage means 200. The acquiring means 210 may acquire these various data from the server device 10 via the communication network NT, instead of from the storage means 200. Then, the process proceeds to step SP202.

[0108] (Step SP202) The control means 270 determines whether or not an imaging process of imaging the user with the camera 23 is being executed among the processes executed for the display content 206. If the determination is negative, the process proceeds to step SP204. On the other hand, if the determination is positive, the process proceeds to step SP212.

[0109] (Step SP204) The control means 270 determines whether or not there is a request from the user to start imaging the user. The request to start imaging is made, for example, by an operation input such as pressing a button provided on the display content 206. If the determination is affirmative, the process proceeds to step SP206. On the other hand, if the determination is negative, the process proceeds to step SP220.

[0110] (Step SP206) Control means 270 controls the operation of camera 23 to start capturing an image of the user. The moving image of the user captured by camera 23 is stored in storage means 200 as moving image data 205. Furthermore, acquisition means 210 acquires moving image data 205 of the user captured by camera 23 from storage means 200. Then, the process proceeds to the process of step SP208.

[0111] (Step SP208) The display processing means 220 starts extracting information about the user's posture and facial expression from the video data 205. Then, the process proceeds to step SP210.

[0112] (Step SP210) The display processing means 220 starts the movement of each part 2011 constituting the avatar. Specifically, the display processing means 220 starts the process of moving each part 2011 constituting the avatar in accordance with the information on the user's posture and facial expression extracted from the video data 205. Then, the process proceeds to step SP220.

[0113] (Step SP212) The control means 270 determines whether or not there is a request from the user to stop capturing images of the user. The request to stop capturing images is made, for example, by an operation input such as pressing a button provided on the display content 206. If the determination is affirmative, the process proceeds to step SP216. On the other hand, if the determination is affirmative, the process proceeds to step SP214.

[0114] (Step SP214) The control means 270 controls the operation of the camera 23 so as to stop capturing images of the user, and then the process proceeds to step SP216.

[0115] (Step SP216) The display processing means 220 stops extracting information about the user's posture and facial expression from the video data 205. Then, the process proceeds to step SP218.

[0116] (Step SP218) The display processing means 220 stops the movement of each part 2011 that constitutes the avatar, and the process then proceeds to step SP220.

[0117] (Step SP220) The change means 230 performs processing to change the display mode of each part 2011 constituting the currently displayed avatar. Details of the processing in step SP224 are the same as those in steps SP20 and SP32 in Fig. 7 and have been explained with reference to Fig. 8, so explanation thereof will be omitted here. Then, the processing proceeds to the processing in step SP222.

[0118] (Step SP222) The control means 270 determines whether or not the avatar distribution process is being executed among the processes executed for the display content 206. If the determination is negative, the process proceeds to step SP224. On the other hand, if the determination is positive, the process proceeds to step SP228.

[0119] (Step SP224) The control means 270 determines whether or not there is a distribution start request from the user to start distribution of the avatar. The distribution start request is made, for example, by operating an input such as pressing a button provided on the display content 206. If the determination is affirmative, the process proceeds to step SP226. On the other hand, if the determination is negative, the process proceeds to step SP232.

[0120] (Step SP226) The transmission means 280 starts distributing the avatar. The transmission means 280 transmits the avatar displayed in the display content 206 to the distributor user terminal 30. The avatar displayed in the display content 206 is transmitted by the distributor user terminal 30 to the distributor server 40 via the communications network NT. Furthermore, the avatar displayed in the display content 206 is distributed from the distributor server 40 to the viewer terminal 50 via the communications network NT. Then, the processing proceeds to step SP232.

[0121] (Step SP228) The control means 270 determines whether or not there is a distribution stop start request from the user to stop the distribution of the avatar. The distribution stop request is made, for example, by operating an input such as pressing a button provided on the display content 206. If the determination is affirmative, the process proceeds to step SP230. On the other hand, if the determination is affirmative, the process proceeds to step SP232.

[0122] (Step SP230) The transmission means 280 stops the distribution of the avatar. That is, the transmission means 280 stops the transmission of the avatar displayed in the display content 206 to the distribution user terminal 30. This stops the distribution of the avatar to the viewer terminal 50. Then, the processing proceeds to step SP232.

[0123] (Step SP232) The control means 270 determines whether there is an end command to end the execution process of the display content 206. The end command is, for example, an instruction to turn off the power of the user terminal 20, or a command to end the display content 206 by the operating system of the user terminal 20. If the determination is positive, the process proceeds to the process of step SP234. On the other hand, if the determination is positive, the process returns to the process of step SP202.

[0124] (Step SP234) The control means 270 ends the execution process of the display content 206. Then, the series of processing flows shown in FIG.

[0125] <Effects> As described above, in this embodiment, the program causes a computer (user terminal 20) to function as a storage means 200 that stores multiple image data corresponding to each part 2011 of an avatar and behavior setting information 202 in which the display behavior of the image data is set according to the combination of the multiple image data, an acquisition means 210 that acquires the multiple image data and behavior setting information 202 from the storage means 200, a display processing means 220 that performs display processing of the avatar, and a change means 230 that, when execution of the display processing is started and parts 2011 of the avatar overlap each other in the virtual space, changes the display behavior of image data corresponding to at least one of the overlapping parts 2011 based on the behavior setting information 202.

[0126] With this configuration, when the program executes the avatar display process, if the parts 2011 overlap each other in the virtual space, the program changes the display mode of the image data of the parts 2011. Therefore, the program can display the avatar in an appropriate display mode depending on the overlapping state of the parts 2011 related to the avatar.

[0127] In addition, in this embodiment, the aspect setting information 202 includes information regarding the anteroposterior order of arrangement of parts 2011 in the virtual space, and the program further causes the computer to function as a selection means 240 that selects multiple parts 2011 in response to a selection request from a user, and an arrangement means 250 that arranges multiple image data corresponding to the multiple parts 2011 selected by the selection means 240 in the virtual space in a anteroposterior order based on the information regarding the anteroposterior order.

[0128] With this configuration, the program arranges the parts 2011 selected by the user based on information about the front-to-back order. Therefore, when arranging the parts 2011 in the avatar display process, the program can display the avatar in an appropriate display mode depending on the overlapping state of the parts 2011 related to the avatar.

[0129] In addition, in this embodiment, the display mode is set in the mode setting information 202 for a combination of multiple image data that is the target of changing the display mode, and the program further causes the computer to function as an adjustment means 260 that adjusts parameters related to the display mode when a combination for which the mode setting information 202 is set is included in the multiple image data corresponding to the multiple parts 2011 selected by the selection means 240.

[0130] According to this configuration, the display mode is set for the combination to be changed in the mode setting information 202, and the parameters related to the display mode are adjustable. Therefore, the program can adjust the parameters of the display mode according to the type of each part 2011 related to the avatar and the user's preferences, and therefore the avatar can be displayed in a more appropriate display mode.

[0131] In this embodiment, the aspect setting information 202 includes initial values ​​of the parameters, and the adjusting means 260 adjusts the parameters based on the initial values.

[0132] With this configuration, the program adjusts the parameters using the initial values, which saves the user the trouble of considering an appropriate display mode for each combination of parts 2011 and adjusting the parameters.

[0133] In addition, in this embodiment, when the adjustment means 260 performs parameter adjustment processing, the display processing means 220 displays an adjustment object 2073 that can be operated by a user's operation input, and the adjustment means 260 adjusts the parameter based on the user's operation input to the adjustment object 2073.

[0134] According to this configuration, the program adjusts the display mode parameters in response to user input, thereby enabling the program to adjust the display mode parameters in accordance with the user's preferences, thereby enabling the avatar to be displayed in a more appropriate display mode.

[0135] In this embodiment, the mode setting information 202 includes information on whether or not the user can perform adjustment processing, and the display processing means 220 displays an adjustment object 2073 relating to a combination of multiple image data for which the user can perform adjustment processing.

[0136] With this configuration, the program adjusts parameters for parts 2011 that require user instructions or operations in response to user input. Therefore, the program can reduce the user's effort of considering an appropriate display mode for each combination of parts 2011 and adjusting parameters. Furthermore, the program can adjust display mode parameters according to the user's preferences.

[0137] Furthermore, in this embodiment, the storage means 200 stores a part set 2010 consisting of a plurality of parts 2011 for each body part of the avatar, the acquisition means 210 acquires the plurality of part sets 2010 for each body part of the avatar from the storage means 200, the selection means 240 selects one part set 2010 from the plurality of part sets 2010 in response to a selection request for each body part, and the arrangement means 250 arranges in the virtual space a plurality of image data corresponding to a plurality of parts 2011 included in the plurality of part sets 2010 selected by the selection means 240 in an order based on information regarding the anteroposterior order.

[0138] According to this configuration, the program selects and arranges parts 2011 included in part set 2010 in response to a user's request to select part set 2010. Therefore, even when there are an enormous number of parts 2011, the program can easily select and arrange parts 2011 because parts 2011 are grouped into each part set 2010. Furthermore, even when parts 2011 are grouped into part set 2010, the program can display the avatar in an appropriate display mode depending on the overlapping state of parts 2011.

[0139] Furthermore, in this embodiment, when the parts 2011 overlap each other and at least one of the overlapping parts 2011 is located closer to the front in the virtual space than the other parts 2011 of the overlapping parts 2011, the modification means 230 modifies the display mode based on the mode setting information 202 so as to perform transparency processing on the portion of the image data corresponding to one part 2011 that overlaps with the image data corresponding to the other parts 2011.

[0140] With this configuration, when parts 2011 overlap, the program processes the image data in the foreground to be transparent and to reflect the part 2011 in the background. Therefore, the program can display the avatar in an appropriate display mode depending on the overlapping state of parts 2011.

[0141] Furthermore, in this embodiment, when parts 2011 overlap each other and at least one of the overlapping parts 2011 is positioned closer to the front in the virtual space than the other parts 2011 of the overlapping parts 2011, the change means 230 changes the display mode based on the mode setting information 202 so that the other parts 2011 are positioned closer to the front in the virtual space than the one part 2011.

[0142] According to this configuration, when parts 2011 overlap, modification means 230 performs processing to position the image data located at the back in front of the part 2011 located at the front. Therefore, the program can display the avatar in an appropriate display mode depending on the overlapping state of parts 2011.

[0143] Furthermore, in this embodiment, the storage means 200 stores avatar movement data 204 associated with the user's movement, the acquisition means 210 acquires video data 205 of the user being captured and acquires the movement data 204 from the storage means 200, and the display processing means 220 extracts information relating to the user's posture and facial expression from the video data 205 and executes display processing to move multiple parts 2011 relating to the avatar based on the extracted information and the movement data 204.

[0144] Therefore, even when the parts 2011 are moving, the program can display the avatar in an appropriate display mode depending on the overlapping state of the parts 2011.

[0145] Furthermore, in this embodiment, the change means 230 determines whether the parts 2011 overlap each other in accordance with the movement of the parts 2011, and executes a process for changing the display mode based on the result of the determination.

[0146] Therefore, even when the overlapping state of the parts 2011 changes as the parts 2011 move, the program can display the avatar in an appropriate display mode depending on the overlapping state of the parts 2011.

[0147] <Modification> The present invention is not limited to the above-described embodiments. In other words, variations of the above-described embodiments, which are appropriately modified by a person skilled in the art, are also included within the scope of the present invention as long as they include the features of the present invention. Furthermore, the elements of the above-described embodiments and the modifications described below can be combined to the extent technically possible, and such combinations are also included within the scope of the present invention as long as they include the features of the present invention.

[0148] In the above embodiment, the avatar display system 1 transmits the video data 205 from the user terminal 20 to the viewer terminal 50 via the distributor user terminal 30 and the distributor server 40 during the distribution process, but this is not limited to this. In the distribution process, the avatar display system 1 may transmit the video data 205 from the user terminal 20 to the distributor server 40 via the communication network NT, rather than transmitting the video data 205 from the user terminal 20 to the distributor server 40.

[0149] With this configuration, the avatar display system 1 can display the avatar in an appropriate display mode according to the overlapping state of the avatar-related parts 2011, even if the user does not have the distributing user terminal 30. Furthermore, the avatar display system 1 can distribute the avatar after displaying it in an appropriate display mode.

[0150] Furthermore, in the distribution process, the avatar display system 1 does not necessarily need to transmit the video data 205 from the user terminal 20 to either the distributor user terminal 30 or the distributor server 40. The avatar display system 1 may transmit the video data 205 from the user terminal 20 to the viewer terminal 50 via the communication network NT. In other words, the avatar display system 1 does not necessarily need to include the distributor user terminal 30 or the distributor server 40. Note that, when the video data 205 is transmitted directly from the user terminal 20 to the viewer terminal 50, a program for receiving and viewing the video data 205 from the server device 10 is assumed to have been installed in advance in the viewer terminal 50.

[0151] According to this configuration, avatar display system 1 can display and distribute avatars in an appropriate display mode depending on the overlapping state of parts 2011 related to the avatar, without going through distributor server 40.

[0152] Furthermore, in the above embodiment, when two parts 2011 overlap, the change means 230 changes the display mode of at least one of the overlapping parts 2011, but this is not limited to this. When three or more parts 2011 overlap, the change means 230 may change the display mode of at least one of the overlapping parts 2011. Specifically, the mode setting information 202 may set display modes not only for combinations of image data of two parts 2011, but also for combinations of image data of three or more parts 2011. When three or more parts 2011 overlap, the change means 230 changes the display mode of the image data to a display mode corresponding to the combination of three or more image data in the mode setting information 202.

[0153] With this configuration, the program can display the avatar in an appropriate display mode depending on the overlapping state of three or more parts 2011.

[0154] Furthermore, when the display mode is set in the mode setting information 202 for a combination of image data of three or more parts 2011, the change means 230 may change the display mode of overlapping parts 2011. A specific example will be described taking as an example a case where parts α, β, and γ are overlapping in order from the front side in the virtual space. For example, the change means 230 may change the order of part γ to be in front of part α in accordance with the mode setting information 202, and then further adjust the transparency of part α to the initial value or a value adjusted by the user.

[0155] With this configuration, the program can display the avatar by adjusting the display mode in a complex and detailed manner according to the overlapping state of three or more parts 2011.

[0156] Furthermore, in the above embodiment, the change means 230 executes the process of changing the display mode at the start of the execution of the avatar display process, but this is not limited to this. The change means 230 may execute the process of changing the display mode before or after the execution of the avatar display process. That is, the change means 230 may execute the change of the display mode in advance, so that the change of the display mode is reflected when the avatar or the parts 2011 that make up the avatar are displayed. Furthermore, the change means 230 may execute the change of the display mode after first displaying the avatar or the avatar.

[0157] With this configuration, the program can display and distribute the avatar in an appropriate display mode depending on the overlapping state of the parts 2011 related to the avatar, regardless of the timing of execution of the avatar display process.

[0158] Furthermore, the avatar display system 1 executes the processes of steps SP10 to SP38 in the avatar creation process as described with reference to FIG. 7, but is not limited to this. For example, after executing the process of step SP22, the avatar display system 1 may proceed to the process of step SP26 instead of the process of step SP24. Specifically, the avatar display system 1 displays the adjustment object 2073 when the image data corresponding to the part 2011 already placed in the virtual space includes a combination of image data whose display mode is subject to change. In this example, if the combination of image data is subject to change, the user can also adjust the display mode. Note that in this example, if a negative determination is made in the processes of steps SP12 and SP18, the avatar display system 1 proceeds to the process of step SP36 instead of the process of step SP24.

[0159] With this configuration, the program does not distinguish between whether a combination of image data is subject to change in display mode and whether it is adjustable by the user. This allows the program to reduce the number of processes, and to display the avatar in an appropriate display mode according to the overlapping state of the avatar parts 2011 with low load.

[0160] Furthermore, for example, if a positive determination is made in the processing of step SP18, the avatar display system 1 may transition to the processing of step SP26 instead of the processing of step SP20. That is, the avatar display system 1 does not have to perform the processing of steps SP20, SP22, and SP24. In this example, the avatar display system 1 first displays the adjustment object 2073 without adjusting the display mode parameters based on the initial values ​​or changing the display mode. Note that, also in this example, if a negative determination is made in the processing of steps SP12 and SP18, the avatar display system 1 transitions to the processing of step SP36 instead of the processing of step SP24.

[0161] With this configuration, the program adjusts the display mode parameters and changes the display mode only when an operation is performed on the displayed adjustment object 2073. Therefore, the program can reduce the number of processes, and can display the avatar in an appropriate display mode according to the overlapping state of the avatar parts 2011 with low load. [Explanation of symbols]

[0162] 1... avatar display system (system), 20... user terminal (computer, information processing device), 200... storage means, 210... acquisition means, 220... display processing means, 230... change means

Claims

1. Computer, a storage means for storing a plurality of image data corresponding to each part of an avatar, and display mode setting information in which a display mode of the image data is set in accordance with a combination of the plurality of image data; an acquisition means for acquiring the plurality of image data and the aspect setting information from the storage means; a display processing means for performing display processing of the avatar; a change means for changing a display mode of image data corresponding to at least one of the overlapping parts of the avatar based on the mode setting information when the execution of the display process is started and the parts of the avatar overlap each other in the virtual space; an adjustment means for adjusting parameters relating to the display mode; It functions as the mode setting information includes information on whether or not a user can perform an adjustment process of the parameter; the display processing means, when a combination of the plurality of image data for which the adjustment process can be performed is included among a plurality of image data corresponding to a plurality of parts related to the avatar, displays an adjustment object related to the combination; the adjustment means adjusts the parameter based on an operation input by the user to the adjustment object. program.

2. the aspect setting information includes information regarding the order of placement of the parts in the virtual space, a selection means for selecting the plurality of parts in response to a selection request from the user; an arrangement means for arranging, in the virtual space, a plurality of image data items corresponding to a plurality of parts selected by said selection means in a front-to-back order based on the information about said front-to-back order; The program according to claim 1, further functioning as:

3. the display mode setting information is set for a combination of the plurality of image data that is to be subject to a change of the display mode, the adjusting means adjusts the parameters related to the display mode when a combination for which the mode setting information is set is included in a plurality of image data corresponding to a plurality of parts selected by the selecting means. The program according to claim 2, further functioning as:

4. the behavior setting information includes initial values ​​of the parameters, The adjusting means adjusts the parameters based on the initial values. The program according to claim 3.

5. the storage means stores a part set configured by the plurality of parts for each body part of the avatar; the acquiring means acquires a plurality of the part sets for each body part of the avatar from the storage means; the selection means selects one of the plurality of part sets for each of the portions in response to the selection request, the arranging means arranges, in the virtual space, a plurality of image data items corresponding to a plurality of parts included in a plurality of the part sets selected by the selecting means in a front-to-back order based on the information about the front-to-back order. The program according to claim 2.

6. when the parts overlap each other and at least one of the overlapping parts is located closer to the front in the virtual space than the other parts of the overlapping parts, the change means changes the display mode based on the mode setting information so as to perform transparency processing on a portion of the image data corresponding to the one part that overlaps with the image data corresponding to the other part. The program according to claim 1.

7. when the parts overlap each other and at least one of the overlapping parts is located closer to the viewer in the virtual space than another of the overlapping parts, the change means changes the display mode based on the mode setting information so that the other part is located closer to the viewer in the virtual space than the one part; The program according to claim 1.

8. the storage means stores the avatar's motion data associated with the user's motion; the acquiring means acquires video data of the user being captured, and acquires the motion data from the storage means; the display processing means extracts information relating to the user's posture and facial expression from the video data, and executes the display processing to move a plurality of parts relating to the avatar based on the extracted information and the motion data. The program according to claim 1.

9. the change means determines whether the parts overlap each other in accordance with the movements of the parts, and executes a process of changing the display mode based on a result of the determination. The program according to claim 8.

10. a storage means for storing a plurality of image data corresponding to each part of an avatar, and display mode setting information in which a display mode of the image data is set in accordance with a combination of the plurality of image data; an acquisition means for acquiring the plurality of image data and the aspect setting information from the storage means; a display processing means for performing a display process of the avatar; a change means for changing a display mode of image data corresponding to at least one of the overlapping parts of the avatar based on the mode setting information when the display process is started and parts of the avatar overlap each other in the virtual space; an adjustment means for adjusting parameters relating to the display mode; Equipped with the mode setting information includes information on whether or not a user can perform an adjustment process of the parameter; the display processing means, when a combination of the plurality of image data corresponding to a plurality of parts of the avatar is included in the plurality of image data for which the user can perform the adjustment process, displays an adjustment object related to the combination; the adjustment means adjusts the parameter based on an operation input by the user to the adjustment object. Information processing device.

11. a storage means for storing a plurality of image data corresponding to each part of an avatar, and display mode setting information in which a display mode of the image data is set in accordance with a combination of the plurality of image data; an acquisition means for acquiring the plurality of image data and the aspect setting information from the storage means; a display processing means for performing a display process of the avatar; a change means for changing a display mode of image data corresponding to at least one of the overlapping parts of the avatar based on the mode setting information when the display process is started and parts of the avatar overlap each other in the virtual space; an adjustment means for adjusting parameters relating to the display mode; Equipped with the mode setting information includes information on whether or not a user can perform an adjustment process of the parameter; the display processing means, when a combination of the plurality of image data corresponding to a plurality of parts of the avatar is included in the plurality of image data for which the user can perform the adjustment process, displays an adjustment object related to the combination; the adjustment means adjusts the parameter based on an operation input by the user to the adjustment object. system.

Images