Information processing system, information processing method, and program

Abstract

Utilize data on other users' movements. [Solution] An information processing system is disclosed, comprising: a storage unit that stores master motion information representing a time series of movements of part or all of a person or thing; a motion acquisition unit that acquires first motion information relating to the movements of part or all of a user; a calculation unit that calculates a value of a predetermined parameter relating to the similarity between the first motion information acquired by the motion acquisition unit and the master motion information stored in the storage unit; and an extraction processing unit that, based on the value of the predetermined parameter calculated by the calculation unit, extracts one or more master motion information corresponding to the first motion information acquired by the motion acquisition unit from among a plurality of master motion information stored in the storage unit.

Description

Technical Field

[0001] The present disclosure relates to an information processing system, an information processing method, and a program.

Background Art

[0002] There is known a technique for analyzing a user's motion based on sensor data of a detection position detected by a sensor device worn by the user and controlling the display of an image of the user based on the motion analysis result.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the prior art as described above, obtaining data on the movements of other users is not assumed. Therefore, in one aspect, the present disclosure aims to utilize data on the movements of other users.

Means for Solving the Problems

[0005] In one aspect, a storage unit that stores master motion information representing a time series of the movement of a part or the whole of a person or an object, a motion acquisition unit that acquires first motion information related to the movement of a part or the whole of a user, A calculation unit calculates a value for a predetermined parameter relating to the similarity between the first motion information acquired by the motion acquisition unit and the master motion information stored in the storage unit. An information processing system is provided, which includes an extraction processing unit that, based on the value of the predetermined parameter calculated by the calculation unit, extracts one or more master motion information corresponding to the first motion information acquired by the motion acquisition unit from among a plurality of master motion information stored in the storage unit, the first motion information acquired by the motion acquisition unit. [Effects of the Invention]

[0006] In one respect, this disclosure allows for the utilization of data on the movements of other users. [Brief explanation of the drawing]

[0007] [Figure 1] This is a block diagram of the search support system according to this embodiment. [Figure 2] This is a schematic diagram showing an example of the hardware configuration of a server device. [Figure 3] This is a schematic diagram showing an example of the hardware configuration of a terminal device. [Figure 4] This flowchart provides a schematic overview of one example of the processing flow performed by the search support system. [Figure 5] Figure 3 is an explanatory diagram of the process shown, illustrating an example of the mounting position of the acceleration sensor. [Figure 6] Figure 3 is an explanatory diagram of the process shown, and is an explanatory diagram of an example of a joint. [Figure 7] Figure 3 is an explanatory diagram of the process shown, illustrating the relationship between input and output. [Figure 8] This flowchart provides a schematic overview of another example of the processing flow performed by the search support system. [Figure 9] This is a schematic block diagram showing an example of the search support system of this embodiment. [Figure 10] This is an explanatory diagram showing an example of data in the avatar information storage unit. [Figure 11] It is an explanatory diagram showing an example of data in the master motion information storage unit. [Figure 12] It is an explanatory diagram of a profile image. [Figure 13A] It is an explanatory diagram (part 1) of animation content for automatic playback. [Figure 13B] It is an explanatory diagram (part 2) of animation content for automatic playback. [Figure 14] It is an explanatory diagram related to the motions of a plurality of avatars. [Figure 15] It is an explanatory diagram of the hierarchical structure of master motion information.

Mode for Carrying Out the Invention

[0008] Hereinafter, each embodiment will be described in detail while referring to the accompanying drawings. In the accompanying drawings, for ease of viewing, some parts having the same attribute and existing in plurality may be partially provided with reference numerals only.

[0009] Referring to FIG. 1, an overview of a search support system 1 according to an embodiment will be described. FIG. 1 is a block diagram of the search support system 1 according to the present embodiment.

[0010] The search support system 1 includes a server device 10 and one or more terminal devices 20. In FIG. 1, for simplicity, one terminal device 20 is illustrated, but the number of terminal devices 20 is arbitrary.

[0011] The server device 10 is an information processing system such as a server managed by one or more operators who provide a search support function according to the present embodiment. The terminal device 20 is a device used by a user, such as a mobile phone, smartphone, tablet terminal, PC (Personal Computer), head-mounted display, or game device. The terminal device 20 may include a wearable device. The wearable device may be a head-mounted display or a glasses-type device. The glasses-type device may be so-called AR (Augmented Reality) glasses or MR (Mixed Reality) glasses. In any case, the wearable device may be separate from the main body of the terminal device 20 or may implement some or all of the functions of the terminal device 20.

[0012] The terminal device 20 is capable of executing a search support application according to the present embodiment. The search support application may be received by the terminal device 20 from the server device 10 or a predetermined application distribution server via the network 3, or may be previously stored in a storage device provided in the terminal device 20 or a storage medium such as a memory card readable by the terminal device 20. The server device 10 and the terminal device 20 are communicably connected via the network 3. For example, the server device 10 and the terminal device 20 cooperate to execute various processes related to the search support function according to the present embodiment.

[0013] Each terminal device 20 is communicably connected to each other via the server device 10. Hereinafter, "one terminal device 20 transmits information to another terminal device 20" means "one terminal device 20 transmits information to another terminal device 20 via the server device 10". Similarly, "one terminal device 20 receives information from another terminal device 20" means "one terminal device 20 receives information from another terminal device 20 via the server device 10". However, in a modified example, each terminal device 20 may be communicably connected without going through the server device 10.

[0014] Network 3 may include wireless communication networks, the Internet, VPNs (Virtual Private Networks), WANs (Wide Area Networks), wired networks, or any combination thereof.

[0015] In the following, the search support system 1 implements an example of an information processing system, but each element of a specific terminal device 20 may implement an example of an information processing system, or multiple terminal devices 20 may cooperate to implement an example of an information processing system. Furthermore, the server device 10 may implement an example of an information processing system by itself, or the server device 10 and one or more terminal devices 20 may cooperate to implement an example of an information processing system.

[0016] Figure 2 is a schematic diagram showing an example of the hardware configuration of server device 10. Figure 2 schematically illustrates peripheral devices 129 in relation to the hardware configuration of server device 10. Peripheral devices 129 are optional and may include displays, etc.

[0017] The server device 10 includes a CPU (Central Processing Unit) 111, RAM (Random Access Memory) 112, ROM (Read Only Memory) 113, auxiliary storage device 114, drive device 115, and communication interface 117 connected by a bus 119, as well as a wired transceiver 125 and a wireless transceiver 126 connected to the communication interface 117.

[0018] The auxiliary storage device 114 is, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive), and is a storage device that stores data related to application software, etc.

[0019] The wired transceiver unit 125 includes a transceiver capable of communicating using a wired network. Peripheral devices 129 are connected to the wired transceiver unit 125. However, some or all of the peripheral devices 129 may be connected to the bus 119 or to the wireless transceiver unit 126.

[0020] The wireless transceiver 126 is a transceiver capable of communicating using a wireless network. The wireless network may include the network 3 described above.

[0021] The server device 10 may also be connectable to the recording medium 116. The recording medium 116 stores a predetermined program. The program stored on this recording medium 116 is installed on the auxiliary storage device 114 of the server device 10 via the drive device 115. The installed predetermined program becomes executable by the CPU 111 of the server device 10. For example, the recording medium 116 may be a recording medium that records information optically, electrically, or magnetically, such as a CD (Compact Disc)-ROM, flexible disk, or magneto-optical disk, or a semiconductor memory that records information electrically, such as a ROM or flash memory.

[0022] Figure 3 is a schematic diagram showing an example of the hardware configuration of terminal device 20. In Figure 3, peripheral devices 260 are schematically illustrated in relation to the hardware configuration of server device 10.

[0023] The terminal device 20 includes a CPU 211, RAM 212, ROM 213, auxiliary storage device 214, drive device 215, and communication interface 217 connected by a bus 219, as well as a wired transceiver 225 and a wireless transceiver 226 connected to the communication interface 217.

[0024] The auxiliary storage device 214 is, for example, an HDD or SSD, and is a storage device that stores data related to application software, etc.

[0025] The wired transceiver unit 225 includes a transceiver capable of communicating using a wired network. Peripheral devices 260 are connected to the wired transceiver unit 225. However, some or all of the peripheral devices 260 may be connected to the bus 219 or to the wireless transceiver unit 226.

[0026] The wireless transceiver 226 is a transceiver capable of communicating using a wireless network. The wireless network may include the network 3 described above.

[0027] The terminal device 20 may also be connectable to the recording medium 216. The recording medium 216 stores a predetermined program. The program stored on this recording medium 216 is installed on the auxiliary storage device 214 of the terminal device 20 via the drive device 215. The installed predetermined program becomes executable by the CPU 211 of the terminal device 20. For example, the recording medium 216 may be a recording medium that records information optically, electrically, or magnetically, such as a CD-ROM, flexible disk, or magneto-optical disk, or a semiconductor memory that records information electrically, such as a ROM or flash memory.

[0028] The peripheral device 260 is optional, but in this embodiment, it includes a display device 2623 and an input device 2624.

[0029] The display device 2623 includes, for example, a display device such as a liquid crystal display or an organic EL (Electro-Luminescence) display. The display device 2623 is capable of displaying a variety of images. The display device 2623 is configured, for example, as a touch panel and functions as an interface for detecting various user operations. The display device 2623 may also be built into a head-mounted display.

[0030] The input device 2624 may include physical keys, or it may further include any input interface, including a pointing device such as a mouse. The input device 2624 may also be capable of accepting non-contact user input, such as voice input, gesture input, or gaze input. For gesture input, sensors for detecting various user states (such as image sensors, acceleration sensors, distance sensors, etc.), dedicated motion capture systems integrating sensor technology and cameras, or controllers such as joysticks may be used. The camera for gaze detection may also be placed inside the head-mounted display. As mentioned above, various user states include, for example, the user's orientation, position, movement, or similar, and in this case, the user's orientation, position, and movement are not limited to the orientation, position, and movement of any part or all of the user's body, such as the face or hands, but also include the orientation, position, and movement of the user's gaze or similar.

[0031] Next, the search support system 1 will be further explained with reference to Figure 4 and subsequent figures.

[0032] Figure 4 is a flowchart illustrating an example of the processing flow performed by the search support system 1. Figures 5 to 7 are explanatory diagrams of the processing shown in Figure 3, with Figure 5 being an explanatory diagram of an example of the mounting position of the acceleration sensor. Figure 6 is an explanatory diagram of an example of a joint, and Figure 7 is a diagram showing the relationship between input and output.

[0033] Figure 4 shows the processing divided between the server device 10 and the terminal device 20. Note that, for convenience, the database function is shown separately from the server device 10, but it may also be implemented by the server device 10. The following description focuses on one terminal device 20, but similar functions can be implemented in other terminal devices 20. Furthermore, in the following, "user" refers to the user of one terminal device 20 unless otherwise specified.

[0034] The user generates input motion information (an example of first motion information) via the terminal device 20 (step S2100) and transmits it to the server device 10 (step S2110). The input motion information is information that represents the movement of a person or object. The input motion information may be information based on the posture at a single point in time, or it may be information based on a series of movements (changes in posture) (time-series data).

[0035] The input motion information typically represents the user's own movements, but it may also represent the movements of other people (e.g., friends), animals such as dogs, or objects such as robots.

[0036] Furthermore, the input motion information may be image information such as videos or still images, or it may be information generated based on image information (secondary information). For example, the information generated based on image information may be information in which only the movement has been extracted based on image processing or image analysis, or information that represents movement in an abstract way.

[0037] Furthermore, the input motion information may be based on actual movements, or it may be artificially generated information such as drawing. For example, the input motion information may be generated based on existing drawings such as those found in comics, or it may be generated based on hand-drawn pictures by the user.

[0038] Furthermore, to generate input motion information, sensors for detecting various user states (such as image sensors, accelerometers, and distance sensors), dedicated motion capture systems integrating sensor technology and cameras, and controllers such as joysticks may be used. For example, Figure 5 shows examples of mounting positions for accelerometers that may be attached to the user. Note that the accelerometer may also have a function to detect gyroscope (angular velocity). In the case of Figure 5, the accelerometers are attached to a total of six locations on the person's body: the head, wrists, buttocks, and ankles (see HD, WL, WR, HP, AKL, and AKR in Figure 5), but they may be attached in other ways.

[0039] Furthermore, the input motion information may be generated as joint movements based on distance images, etc. For example, the example shown in Figure 6 is a model represented by multiple joints and the skeleton (links) between the joints. The joint model shown in Figure 6 is a 16-joint model with one head, and three joints each in the torso, both arms, and both legs. The head is defined as having joints at both ends, while the others are defined as having three joints at both ends and the midpoint. Specifically, the joint model consists of 16 joints A0 to A15 and 15 skeletons B1 to B15 (also referred to as "parts B1 to B15") that connect the joints. To make it easier to understand, for example, joints A4 and A7 are the left and right shoulder joints, and joint A2 is a joint related to the cervical spine. Also, joints A14 and A15 are the left and right hip joints, and joint A0 is a joint related to the lumbar spine. Furthermore, the input motion information may also include the movements of finer joints, such as the joints of the fingers.

[0040] Furthermore, the input motion information may represent the overall movement of the moving object (person or object), or only a part of it. For example, the input motion information may represent the movement of the entire body, or the movement of a specific part of the body. In addition, the movement may represent not only the movement of each part via joints, but also movements related to facial expressions (movement of muscles, etc.). Furthermore, the movement may represent the movement of the user's gaze. In this case, the camera for gaze detection may be placed inside the head-mounted display.

[0041] When the server device 10 acquires input motion information (step S2200), it calculates the values ​​of similar parameters related to the input motion information with respect to the motion information data stored in a given master motion information database (step S2210).

[0042] The similarity parameter for input motion information is a parameter that represents the degree of similarity between the input motion information and other motion information being compared. Here, as mentioned above, the input motion information is information that represents movement. Therefore, the degree of similarity between the input motion information and other motion information being compared relates to the degree of similarity of each movement. The method for calculating the value of the similarity parameter for input motion information is arbitrary, but for example, it may be calculated in a manner in which characteristic movements (e.g., movements that occur frequently or unique movements) are given relatively heavy weights.

[0043] Furthermore, the degree of similarity between the input motion information and other motion information being compared may be evaluated after processing the input motion information and / or the other motion information being compared. For example, the input motion information and the other motion information being compared may be compared using partially extracted or compressed parts as digests. Alternatively, the degree of similarity between the input motion information and the other motion information being compared may be evaluated based on the frequency analysis results (e.g., FFT: Fast Fourier Transform) of each time-series waveform. In this case, if the motion is synchronized to a rhythm, a peak will appear at a frequency corresponding to that rhythm, allowing for an appropriate evaluation of the similarity of the motion information when the motion is synchronized to a rhythm. Other parameters that may be suitably used for evaluation include the total time (time of motion), tempo (e.g., Beats Per Minute), and accuracy to the corresponding rhythm (degree of synchronization).

[0044] Furthermore, the degree of similarity between the input motion information and other motion information being compared may be limited to the degree of similarity in movement, but may also include similarity in other elements (artistic merit, associated music, or background). In this case, the input motion information may include supplementary information other than movement (e.g., music or background). Other elements and supplementary information may include avatar parts, costume type, ID, avatar's location in the virtual space, date, user attributes (country, language, age, gender, etc.).

[0045] In addition, a master motion information database that stores other motion information for comparison may typically store a large amount of motion information for comparison. In this case, the value of the similarity parameter related to the input motion information may be calculated for each of the large amount of motion information for comparison. In this case, the larger the amount of motion information for comparison stored in the master motion information database, the higher the probability that there is motion information for comparison in the master motion information database that will result in a high value of the similarity parameter related to the input motion information. In addition, there may be multiple master motion information databases. Therefore, if, when using one master motion information database, other motion information for comparison that will result in a similarity parameter value related to the input motion information being above a threshold cannot be obtained, another master motion information database may be used.

[0046] The server device 10 calculates the value of the similar parameter related to the input motion information (step S2210), and then extracts one or more master motion information corresponding to the input motion information based on the calculation result (step S2220). The one or more master motion information to be extracted may include, for example, other motion information (motion information in the master motion information database) that is a comparison target and whose similar parameter value related to the input motion information is equal to or greater than a threshold. Alternatively, the one or more master motion information to be extracted may be, for example, only the other motion information that is a comparison target and whose similar parameter value related to the input motion information is equal to or greater than a threshold, and whose similar parameter value is the highest among the other motion information that is a comparison target.

[0047] As mentioned above, the server device 10 may be formed by multiple server computers. Therefore, one server computer forming the server device 10 may cooperate with another server computer forming the server device 10 to extract one or more master motion information corresponding to the input motion information. In this case, the server computers may cooperate with each other via an API (Application Programming Interface).

[0048] The server device 10 extracts one or more master motion information corresponding to the input motion information (step S2220) and outputs the extraction results (step S2230). The destination of the output of the extraction results is arbitrary; for example, the extraction results may be output to the user related to the input motion information, or they may be output for other processing within the server device 10.

[0049] In this embodiment, when input motion information is received, the search support system 1 extracts and outputs one or more master motion information corresponding to the input motion information based on the input motion information.

[0050] Here, master motion information refers to information representing the time series of movements of a person or object, either as a part or as a whole, and may be the same as input motion information. For example, master motion information may include, as a time series of movements, at least one of the following: a time series of movements related to a person's facial expressions, and a time series of movements of joints in a person or object. However, unlike input motion information, master motion information is pre-prepared (managed and stored) information (motion information in the master motion information database). The source of master motion information is arbitrary, but may include, for example, performance data from famous performers or actors, or animations and videos. However, in this case, it may be assumed that usage licenses (e.g., copyright-related usage licenses) have been obtained, or it may be offered as a paid service.

[0051] In the following, one or more master motion information that can be extracted when one input motion information is input will also be referred to as one or more master motion information corresponding to one input motion information. In this case, one input motion information will also be referred to as input motion information corresponding to one or more extracted master motion information.

[0052] As described above, a single input motion information and one or more master motion information corresponding to that input motion information have a relationship such that the value of the similar parameter related to the single input motion information is greater than or equal to a threshold. Furthermore, as described above, a single input motion information and one or more master motion information corresponding to that input motion information do not necessarily represent motions of the same length (temporal length). For example, one or more master motion information corresponding to a single input motion information may represent motions that are significantly longer or shorter than the single input motion information. More specifically, for example, a single input motion information may be based on a relatively long video, while one or more master motion information corresponding to that input motion information may be based on relatively short videos (e.g., short videos like those on TikTok®).

[0053] In this embodiment, the extraction conditions for master motion information for a given input motion information may be customizable by the user. In this case, the customizable extraction conditions are arbitrary and may relate to thresholds, attributes, or performers. For example, extraction conditions may be set so that only master motion information relating to female performers is extracted.

[0054] The search support system 1 may output the master motion information extracted in this manner as is, or it may output it after processing or other modifications. The output destination is arbitrary, but for example, it may be set to the user who entered the input motion information. In this case, a user who wants to obtain master motion information related to a certain movement can achieve their goal by generating input motion information that represents a movement similar to that movement and inputting the generated input motion information into the search support system 1. In other words, the user may be able to obtain the master motion information related to that movement. This eliminates the need for the user to create similar motion information from scratch themselves, improving convenience. For example, Figure 7 shows an example of input motion information on the left. In this case, when input motion information is entered, the corresponding master motion information is obtained, so the user can use the master motion information corresponding to the input motion information.

[0055] According to the search support system 1 of this embodiment, a data search function based on the similarity of motion information can be realized, thereby improving convenience. For example, users can obtain desired master motion information based on movements that are difficult to express in words, thus improving convenience. In addition, the search target can be managed in advance as master motion information, and the processing load can be reduced by limiting the search area.

[0056] By the way, when a user searches for videos or other content containing motion data, the only way to extract results was to use the strings associated with those videos or content (such as the title, song title using hashtags, or talent name). However, there may be cases where videos or other content do not have any strings attached to them.

[0057] In contrast, according to this embodiment, when a user generates input motion information by, for example, moving their arms and legs, that input motion information can be used to search for similar motions themselves or videos containing similar motions. In this embodiment, master motion information may be searched for as a viewing target. That is, the user may use the search support system 1 not to use master motion information, but to view it. In this case, the search support system 1 may have different search targets for viewing purposes and for production purposes. For example, in the case of a viewing-purpose search, the motion data that can be searched as master motion information may be videos produced for viewing (for example, videos viewable on TikTok®, YouTube®, or similar platforms). In this case, the user can efficiently view the desired video by using a new method for searching for videos for viewing. Furthermore, in this embodiment, the input motion information used to search for desired master motion information may be information representing emotional elements (emotes), in which case the information representing emotes may include images of corresponding facial expressions or text information representing corresponding emotions. In this case, it is possible to search for master motion information corresponding to the desired emote, and based on that, it becomes possible to generate animations, etc., as described later. For example, the user can select desired master motion information from among multiple master motion information that may be presented, and based on the selected master motion information, it becomes possible to generate animations, etc., as described later. In this case, since the input motion information is the user's own movement, it may be information representing, for example, awkward movement. However, the master motion information searched based on that may be sharp movement (such as movement generated by a user who is good at movement, as if the user's own movement had been corrected), and in that case, the user can also reflect such sharp movement on their own avatar (see the generation of avatar motion information described later).

[0058] Figure 8 is a flowchart illustrating another example of the processing flow performed by the search support system 1.

[0059] The example shown in Figure 8 differs from the example shown in Figure 4 in that it includes a specific example of the process for outputting the extraction results (step S2230). The example shown in Figure 8 includes the processes of steps S22301 and S22302 as step S2230 of the example shown in Figure 4.

[0060] Specifically, the server device 10 transmits the extraction result (an example of second motion information) to the terminal device 20 (step S22301), and the terminal device 20 outputs the extraction result to the user via the display device 2623 (step S2120). The output method of the extraction result via the display device 2623 is arbitrary; for example, a video based on one or more master motion information may be played back in a video playback format. Alternatively, a link such as a URL that enables access to one or more extracted master motion information may be output. In this case, the extraction result transmitted from the server device 10 to the terminal device 20 may include link information such as a URL that enables access to one or more extracted master motion information. With such a configuration, the communication load can be reduced.

[0061] If a user wishes to reflect the extracted master motion information onto their avatar, they may generate a reflection instruction to that effect and send it to the server device 10 (step S2130). The reflection instruction may include instructions on which of the extracted master motion information to use.

[0062] In this case, the server device 10 generates motion information (hereinafter also referred to as "avatar motion information" for distinction) (an example of second motion information) that reflects the master motion information onto a specific avatar (for example, the user's own avatar). The server device 10 then transmits the generated avatar motion information to the terminal device 20 (step S22302). In this case, the terminal device 20 may output the received avatar motion information to the user via the display device 2623 (step S2140). The output method of the avatar motion information via the display device 2623 is arbitrary; for example, a video based on the avatar motion information may be played back in a video playback format. Alternatively, a link such as a URL that enables access to a website where the generated avatar motion information can be played back may be output. In this case, the avatar motion information transmitted from the server device 10 to the terminal device 20 may include link information such as a URL that enables access to the avatar motion information. With this configuration, the communication load can be reduced compared to transmitting the avatar motion information to the terminal device 20 in video file format. Alternatively, the avatar motion information transmitted from the server device 10 to the terminal device 20 may consist of only a portion of the avatar motion information. For example, only the information necessary to reflect the avatar motion information on the user's avatar (e.g., metadata) may be transmitted from the server device 10 to the terminal device 20. In this case, for example, the terminal device 20 may use information it has on hand (e.g., the user's avatar information) to play a video based on the avatar motion information. In this case as well, the communication load can be reduced compared to transmitting the avatar motion information to the terminal device 20 in video file format. Furthermore, metadata such as the user's avatar information (part IDs, IDs of items worn) and audio data may be stored in association with a timestamp, allowing the terminal device 20 to reproduce this data (client rendering).

[0063] Here, the method for reflecting the master motion information in the user's avatar's movements is arbitrary, but for example, if the master motion information has time-series motion data for each joint or body part, the same time-series motion data may be provided to the corresponding joints or body parts of the avatar. In this case, if the person or object that serves as the model for the master motion information and the user's avatar have different numbers of joints or lengths of body parts, such differences may be corrected as appropriate. For example, if the number of joints in the person that serves as the model for the master motion information to which time-series motion data is associated is less than the number of joints in the user's avatar, the movements of the other joints of the avatar may be corrected as appropriate using techniques such as inverse kinematics.

[0064] Furthermore, the number of master motion information pieces reflected in the user's avatar's movements is arbitrary; for example, it may be possible to combine two or more master motion information pieces and reflect them in the user's avatar's movements. In this case, the two or more master motion information pieces may be combined in a time-division manner, or they may be combined in a manner in which different master motion information is reflected for each joint or body part.

[0065] Next, with reference to Figure 9 and subsequent figures, further details of the search support system 1 of this embodiment will be described.

[0066] Figure 9 is a schematic block diagram showing an example of the search support system 1 of this embodiment. Figure 10 is an explanatory diagram showing an example of data in the avatar information storage unit 142. Figure 11 is an explanatory diagram showing an example of data in the master motion information storage unit 144. In Figures 10 and 11, "***" indicates a state in which some information is stored, and "···" indicates a state in which similar information is repeatedly stored. Figure 12 is an explanatory diagram of a profile image. Figures 13A and 13B are explanatory diagrams of animation content for automatic playback. Figure 14 is an explanatory diagram relating to the motion of multiple avatars. Figure 15 is an explanatory diagram of the hierarchical structure of master motion information.

[0067] The following functions of the search support system 1 may be implemented by the server device 10, one or more terminal devices 20, or any combination thereof.

[0068] In the example shown in Figure 9, the search support system 1 includes a user information storage unit 140, an avatar information storage unit 142, a master motion information storage unit 144, an input motion information acquisition unit 150, a similarity calculation unit 152, an extraction processing unit 154, an output processing unit 156, a display motion generation unit 158, a content generation unit 160, an information management unit 162, a storage processing unit 164, a profile image generation unit 166, an animation generation unit 168, and an animation playback processing unit 170.

[0069] Furthermore, each processing unit, such as the input motion information acquisition unit 150, can be implemented using the CPU 111 shown in Figure 2 and / or the CPU 211 shown in Figure 3, with a storage device (e.g., ROM 113, 213, etc.). In addition, storage units such as the evaluation information storage unit 190 and the user information storage unit 192 can be implemented using the auxiliary storage device 114 shown in Figure 2 and the auxiliary storage device 214 shown in Figure 3.

[0070] The user information storage unit 140 may store user IDs and their associated avatar IDs. If the user information storage unit 140 is implemented by a terminal device 20, it may store only the avatar IDs of the corresponding users of the terminal device 20. If the user information storage unit 140 is implemented by a server device 10, it may store user IDs and avatar IDs for each user.

[0071] Avatar information storage unit 142 stores avatar information related to the user's avatar. In the example shown in Figure 10, the avatar information 700 associates each avatar ID with a face part ID, hairstyle part ID, clothing part ID, etc. Appearance-related part information such as face part ID, hairstyle part ID, and clothing part ID is a parameter that characterizes the avatar and may be selected by each corresponding user. For example, multiple types of appearance-related information such as face part ID, hairstyle part ID, and clothing part ID for an avatar may be provided. Furthermore, for face part IDs, a separate part ID may be provided for each type of face, such as face shape, eyes, mouth, and nose, and the information related to face part IDs may be managed by a combination of IDs of each part that constitutes the face. In this case, based on the appearance-related IDs associated with each avatar ID, it becomes possible to render each avatar not only on the server device 10 but also on the terminal device 20 side.

[0072] Furthermore, in this embodiment, the avatar information may include joint information and body part information of the corresponding avatar.

[0073] The avatar's joint information may include information such as which joints it has (which functions), the positional relationships of the joints (e.g., positional relationships in a normal posture), and the range of motion (degrees of freedom) of the joints, as shown in Figure 6. The body part information may include information such as the shape of the parts that connect each joint.

[0074] Furthermore, if the avatar information storage unit 142 is implemented by a terminal device 20, the avatar information storage unit 142 may store only the avatar information relating to the user of the corresponding terminal device 20. Also, if the avatar information storage unit 142 is implemented by a server device 10, the avatar information storage unit 142 may store avatar information relating to each user.

[0075] The master motion information storage unit 144 corresponds to the master motion information database described above, as shown in Figure 4, etc. The master motion information storage unit 144 stores the master motion information to be extracted, as described above. The storage format of the master motion information is arbitrary; for example, it may be in the format of a video file, or it may be in the format of motion information extracted based on the video file (for example, time-series data of the movement of each joint). Furthermore, the storage format may differ for each piece of master motion information. In the example shown in Figure 11, the master motion information storage unit 144 associates various information items such as motion data, sound information, background information, hardware information, uniqueness information, and management information with each master motion information ID. Motion data may be data that represents movement, as described above. Sound information, background information, hardware information, uniqueness information, etc. may be arbitrary information items that may not be associated. For example, if the master motion information is movement synchronized with music, sound information may be associated with the master motion information. Also, if the master motion information is a video captured against a specific background, that background information may be associated with the master motion information. Hardware information is information that can identify the hardware (e.g., acceleration sensor, depth camera, etc.) used to acquire the master motion information. Hardware information may include frame rate, recording time, location information, etc. Uniqueness information is information that represents the uniqueness (originality) of the corresponding motion data. Hardware information may also represent virtual hardware created by subprocesses or emulations executed within the server device 10 or terminal device 20. Uniqueness information may be automatically evaluated based on artificial intelligence, etc. Uniqueness information may also be generated based on the value of a parameter (e.g., similarity parameter) relating to the similarity with other master motion information already stored. Furthermore, the uniqueness related to uniqueness information may be evaluated based on the motion data alone, or it may be evaluated in combination with acoustic information, background information, hardware information, etc. Management information may include text information for tagging, information such as the creator (performer, etc.) of the corresponding master motion information, and the date and time of creation.Furthermore, the management information may include information on the terms of use when using the corresponding master motion information. The terms of use may include conditions based on fees in the case of paid services, as well as conditions on whether processing is permitted, whether deformation is necessary, etc. The terms of use may also include conditions related to the length of the user's limbs (or the length of the avatar's limbs) and hardware configuration.

[0076] Furthermore, the master motion information storage unit 144 may store additional information that allows for narrowing the search range (the range of calculation targets by the similarity calculation unit 152, which will be described later). For example, the additional information may include attributes of the motion related to the master motion information (e.g., dance, sports, everyday movements, etc.).

[0077] Furthermore, since the master motion information storage unit 144 is suitable for storing a relatively large amount of data, it is preferable to implement it using a server device 10, but it may also be implemented using multiple terminal devices 20.

[0078] The input motion information acquisition unit 150 acquires input motion information. The input motion information is as described above.

[0079] The input motion information acquisition unit 150 may be implemented by the terminal device 20 alone. In this case, the terminal device 20 acquires input motion information from the corresponding user. On the other hand, the input motion information acquisition unit 150 may be implemented by multiple terminal devices 20 and a server device 10, in which case the server device 10 may acquire input motion information from each user via the terminal devices 20.

[0080] In this embodiment, the input motion information acquisition unit 150 acquires input motion information in correspondence with motion information requests from the user. The motion information requests from the user are requests related to the acquisition of display motion information, which will be described later, and may include filtering information, information specifying a particular avatar (when requesting the generation of avatar motion information, which will be described later), etc.

[0081] The similarity calculation unit 152 calculates the values ​​of predetermined parameters relating to the similarity between the input motion information acquired by the input motion information acquisition unit 150 and the master motion information stored in the master motion information storage unit 144. The predetermined parameters may be the similarity parameters described above.

[0082] The similarity calculation unit 152 may calculate similarity parameter values ​​for each of the master motion information stored in the master motion information storage unit 144 with respect to one input motion information, but preferably, from the viewpoint of reducing processing load, the calculation target may be narrowed. For example, if the attributes of one input motion information are discriminable, the calculation target may be narrowed to only master motion information having the same attributes. Also, if one input motion information includes supplementary information other than motion, the calculation target may be narrowed based on such supplementary information. For example, the supplementary information may be filtering information that can be entered by the user. For example, the supplementary information may include text information such as "***dance". Also, the supplementary information may include information about the hardware resources (e.g., an acceleration sensor) used to acquire the input motion information. In this case, the calculation target may be narrowed to only master motion information acquired using the same or similar series of hardware resources. Also, the supplementary information may include background information. Also, the supplementary information may include acoustic information. In this case, the calculation target may be narrowed to only master motion information to which acoustic information of the same series is linked. Also, the supplementary information may include background information. Background information, when input motion information is image information, refers to information about the background of the moving person or object. In this case as well, the calculation target may be limited to master motion information that is associated with the same type of background information.

[0083] In this embodiment, the similarity calculation unit 152 includes a motion similarity evaluation unit 1521 and other similarity evaluation units 1522.

[0084] The motion similarity evaluation unit 1521 calculates the values ​​of parameters related to the similarity between motions. In this case, the motions related to the input motion information and the motions related to the master motion information are compared. As mentioned above, the method for comparing the motions may be the same as the method for calculating the similarity parameter values ​​described above.

[0085] Furthermore, with respect to human movement involving joints, the values ​​of the similarity parameters may be calculated based on at least one of the following: joint angle, manner of change of joint angle, joint position, manner of change of joint position, and inter-joint length. Time-series waveforms of these parameter values ​​and their frequency analysis results may be used for similarity comparison. Joint angles may also be evaluated using other parameters such as the dot product.

[0086] The other similarity evaluation unit 1522 calculates the values ​​of parameters related to similarities other than those between movements. In this case, the attributes and supplementary information mentioned above may be taken into consideration.

[0087] The similarity calculation unit 152 may calculate the final value of a predetermined parameter based on the calculated values ​​of the parameter relating to the similarity between movements and the calculated values ​​of the parameter relating to similarity other than between movements. Artificial intelligence may be used in the calculation of similarity.

[0088] The extraction processing unit 154 extracts one or more master motion information based on the calculation results by the similarity calculation unit 152. The method for extracting master motion information based on the calculation results by the similarity calculation unit 152 is arbitrary, but it is possible to extract the master motion information with the largest value of a predetermined parameter, or to extract a predetermined number of master motion information in descending order of the value of the predetermined parameter. Alternatively, all master motion information with a value of the predetermined parameter equal to or greater than a threshold may be extracted.

[0089] The output processing unit 156 outputs display motion information generated based on one or more master motion information extracted by the extraction processing unit 154 in a manner visible to humans. The output processing unit 156 may output to any display device. For example, it may be displayed on a terminal that is viewed by a specific person, such as the display device 2623 of the terminal device 20, or it may be displayed on a device that is viewed by an unspecified number of people, such as digital signage.

[0090] In this embodiment, the output processing unit 156 may output display motion information if the motion information request satisfies predetermined conditions. The predetermined conditions are arbitrary, but may include conditions related to copyright, public order and morals, etc. The predetermined conditions may also include conditions to ensure the uniqueness of the display motion information. In this case, uniqueness may be evaluated based on the relationship between the display motion information and the corresponding original master motion information.

[0091] The display motion generation unit 158 ​​generates display motion information to be output by the output processing unit 156 based on one or more master motion information extracted by the extraction processing unit 154.

[0092] The display motion information may be information that displays two or more master motion information extracted by the extraction processing unit 154 together, or it may be generated for each master motion information. The display motion information related to one master motion information may be the one master motion information itself, or it may be new information generated based on the one master motion information. In the latter case, the display motion information related to one master motion information may be generated by processing the one master motion information. In this case, the processing method is arbitrary, but may include processing to shorten it into a digest version (summary version), or processing to change the background or music rhythm, etc.

[0093] Furthermore, the display motion generation unit 158 ​​may generate the avatar motion information described above. That is, the display motion generation unit 158 ​​may generate display motion information (avatar motion information) by reflecting one or more master motion information extracted by the extraction processing unit 154 onto a specific avatar. In this case, the specific avatar may be specified by the user (for example, the user who requested the motion information), and may be, for example, the user's own avatar. For example, the display motion generation unit 158 ​​may reflect the master motion information retrieved in relation to the emote information onto the movement of the corresponding user's avatar based on a reflection instruction from the user.

[0094] The display motion generation unit 158 ​​may be provided on the server device 10 side, on the terminal device 20 side, or on both sides in a manner in which the functions are shared between the server device 10 side and the terminal device 20 side, as described above in relation to avatar motion information.

[0095] The content generation unit 160 generates video content or animation based on the display motion information generated by the display motion generation unit 158. The video content or animation may be the display motion information generated by the display motion generation unit 158 ​​itself, or it may be video content or animation different from the display motion information generated by the display motion generation unit 158. In the latter case, for example, the content generation unit 160 may incorporate the display motion information generated by the display motion generation unit 158 ​​into a part of the video content or animation. In this case, the video content or animation to which it is incorporated may be created based on different display motion information, although this is optional.

[0096] In this embodiment, the content generation unit 160 may generate video content or animation by associating background information, sound information, emote information, or text information with the display motion information generated by the display motion generation unit 158. In this case, the background information, sound information, emote information, or text information to be associated may be supplied by the user (for example, the user who requested the motion information) or may be automatically generated based on artificial intelligence or the like. Alternatively, the video content or animation generated with associated emote information may be registered in a menu or the like that shows a list of emotes available to the corresponding user.

[0097] Furthermore, the content generation unit 160 may perform NFT conversion processing on the generated video content or animation in response to a user request or other request. NFT conversion processing includes issuing the target item as an NFT (Non-Fungible Token). In this case, the content generation unit 160 may be minted via a smart contract on a blockchain network (not shown). Note that NFT conversion processing may also include listing the issued NFT on a marketplace or the like.

[0098] The information management unit 162 sets usage restrictions on the master motion information stored in the master motion information storage unit 144. For example, the information management unit 162 may set usage restrictions on a single master motion information based on the management information associated with that single master motion information. In this case, if the management information associated with that single master motion information indicates that it is provided for a fee, then the provision (use) of the information free of charge may be restricted. In this case, the information management unit 162 may exclude master motion information that cannot be used due to the usage restrictions from the search range (or the calculation target of the similarity calculation unit 152).

[0099] The memory processing unit 164 stores new master motion information in the master motion information storage unit 144 based on motion information for master motion information (an example of third motion information) supplied (provided) by a specific user. The specific user is optional, but may be a user who has authorized the above-described usage (the usage in which the information is extracted and used to generate avatar motion information, etc.). When storing new master motion information, the memory processing unit 164 may acquire and generate the above-described management information, etc. To encourage the provision of motion information for master motion information, incentives such as specific benefits may be given to users who provide motion information for master motion information.

[0100] The profile image generation unit 166 generates a profile image associated with the user based on the display motion information generated by the display motion generation unit 158. The profile image may be a profile image that includes the movement of the avatar associated with the user. In this case, the profile image generated by the profile image generation unit 166 may be used in SNS (Social Networking Service) or similar services. The profile image may be a still image, or it may be a relatively short video such as a short video.

[0101] The profile image generation unit 166 may function based on motion information requests from the user. For example, it may function when the motion information request includes a request for the generation of a profile image. Alternatively, it may function in response to instructions such as the reflection instruction in step S2130 described above, referring to Figure 8.

[0102] Furthermore, the profile image generation unit 166 may associate background information, sound information, emote information, or text information with the generated profile image. In this case, the background information, sound information, emote information, or text information to be associated may be supplied by the user (for example, the user who requested the motion information) or may be automatically generated based on artificial intelligence or the like. The emote information may include information that represents what emotion the user expressed or an animation of facial expressions.

[0103] Furthermore, some or all of the functions of the profile image generation unit 166 may be implemented by the content generation unit 160 described above.

[0104] Figure 12 shows an example of a profile screen including a profile image G12. If the profile image G12 is a video, it may be automatically played when the profile screen is displayed. In the example shown in Figure 12, the profile image G12 forms part of the profile screen, but it may also form the entire screen.

[0105] The animation generation unit 168 generates animation content for automatic playback. The animation content for automatic playback may be part or all of the animation generated by the content generation unit 160 described above. In this case, part or all of the functions of the animation generation unit 168 may be implemented by the content generation unit 160 described above.

[0106] The animation content for automatic playback generated by the animation generation unit 168 may be stored in association with the user (for example, the user who made the motion information request). Alternatively, the animation content for automatic playback may be stored in the storage device of the corresponding user's terminal device 20, for example, in association with the conditions for automatic playback (playback conditions). The conditions for automatic playback (playback conditions) are arbitrary and may be set by the user. The conditions for automatic playback may be used in replies to chats, etc. That is, the animation content for automatic playback may be played as a reply in a chat, etc. For example, in the example shown in Figure 13A, when user B's terminal device 20 receives a question A from another user (for example, an invitation to go somewhere together), multiple stamps STM1 to STM4 are displayed as candidates. In this case, when stamps STM1 to STM4 are selected, the corresponding animation content for automatic playback may be played. In this case, user B may select one of stamps STM1 to STM4 themselves, but in automatic mode, one of stamps STM1 to STM4 may be automatically selected according to the conditions for automatic playback.

[0107] The animation playback processing unit 170 plays an animation content for automatic playback when the corresponding conditions for automatic playback are met. The animation content for automatic playback and the conditions for automatic playback are as described above.

[0108] Furthermore, the animation playback processing unit 170 may also function in the metaverse space. For example, when the conditions for automatic playback are met in the metaverse space, the avatar may be moved with actions corresponding to the animation content for automatic playback. For example, the various emote information (and associated video content, etc.) in the emote list described above may be activated when the user selects (specifies) one of the emotes, thus fulfilling the conditions for automatic playback. When an emote is activated, the corresponding avatar performs a movement in the virtual space that represents that emote. This movement that represents the emote may be video content or animation generated by associating it with the emote information described above. By registering the desired emote information in the emote list in this way, it is possible to easily reflect the desired emote from among a variety of emotes on one's avatar. For example, in the example shown in Figure 13B, user B's terminal device 20 calls up the emote list, and several emote buttons EM1 to EM4 are displayed as candidates. The multiple emote buttons EM1 to EM4 may be pre-registered content (video content or animation generated by associating emote information). In this case, when each emote button EM1 to EM4 is selected, it may be accompanied by the playback of the corresponding auto-play animation content. In this case, user B may select one of the emote buttons EM1 to EM4 themselves, but in automatic mode, for example, user B's emotions (feelings) may be automatically detected from biometric information (e.g., heart rate information) or by image recognition, and one of EM1 to EM4 may be automatically selected. Note that in the example shown in Figure 13B, emote buttons EM1 to EM4 are labeled with characters representing the corresponding emote, but instead of or in addition to this, a digest version of the corresponding content may be played. Furthermore, in the example shown in Figure 13B, four emote buttons EM1 to EM4 are illustrated, but the number of emote buttons is arbitrary, and a number of emote buttons corresponding to the number of registered emote buttons may be provided. Also, in the example shown in Figure 13B, a terminal device 20 in the form of a smartphone is illustrated, but the same applies when other forms of terminal devices 20, such as a head-mounted display, are used. For example, in the case of a terminal device 20 in the form of a head-mounted display, the buttons corresponding to emote buttons EM1 to EM4 may be implemented by a user interface that can be called up in the virtual space.

[0109] Although each embodiment has been described in detail above, the invention is not limited to any particular embodiment, and various modifications and changes are possible within the scope described in the claims. Furthermore, it is possible to combine all or more of the components of the embodiments described above.

[0110] For example, in the embodiment described above, one input motion information and one master motion information represent the movement of a person or object, and represent the movement of one person or object, but are not limited to this. For example, one input motion information and one master motion information may represent the movements of two or more people. In this case, avatar motion information relating to the coordinated movements of multiple avatars A1 to A4, as shown in Figure 14, can be obtained based on the extracted master motion information. In this case, master motion information relating to movements involving contact or coordination between multiple avatars can be obtained, such as multiple avatars holding hands, or one or more avatars lifting one avatar.

[0111] Furthermore, in the embodiment described above, each master motion information may be managed in the master motion information storage unit 144 in a hierarchical structure as schematically shown in Figure 15. In this case, by determining the similarity to a given input motion information from a higher level, it becomes easier to search for master motion information similar to that input motion information. [Explanation of symbols]

[0112] 1… Search support system 3…Network 10…Server device 20…Terminal device 140...User information storage unit 142...Avatar Information Memory Unit 144...Master motion information storage unit 150... Input motion information acquisition unit (an example of a motion acquisition unit) 152…Similarity calculation unit 1521...Motion Similarity Evaluation Unit 1522...Other Similarity Evaluation Units 154…Extraction Processing Unit 156…Output Processing Unit 158...Display motion generation unit (an example of a second motion information generation unit) 160...Content Generation Department 162…Information Management Department 164...Memory Processing Unit 166... ​​Profile Image Generation Unit 168...Animation generation unit 170...Animation playback processing unit 190... Information storage unit for evaluation 192...User information storage unit.

Claims

1. A storage unit that stores master motion information representing the time series of movements of a person or part or the whole of an object, A motion acquisition unit that acquires first motion information relating to the movements of part or all of the user, A calculation unit calculates a value for a predetermined parameter relating to the similarity between the first motion information acquired by the motion acquisition unit and the master motion information stored in the storage unit. An extraction processing unit extracts one or more master motion information from among a plurality of master motion information stored in the storage unit, based on the value of the predetermined parameter calculated by the calculation unit, corresponding to the first motion information acquired by the motion acquisition unit. If the management information attached to the master movement information stored in the memory unit includes information indicating that it is provided for a fee, the information management unit sets usage restrictions that restrict the free use of the master movement information. A second motion information generation unit displays one or more master motion information extracted by the extraction processing unit in a manner visible to the user, and generates second motion information by reflecting the master motion information instructed by the user from among the one or more master motion information in the movement of the avatar associated with the user. A content generation unit generates animation content for automatic playback based on the second motion information generated by the second motion information generation unit, The system includes an animation playback processing unit that plays the animation content when the playback conditions related to the user are met, The content generation unit registers the animation content to which emote information is associated in the emote list, and associates the animation content with at least one of the multiple emote buttons. The animation playback processing unit determines that the playback condition is met when one of the multiple emote buttons is selected, and plays the animation content that corresponds to the selected emote button, which is registered in the emote list in advance.

2. The information processing system according to claim 1, wherein the calculation unit calculates the value of the predetermined parameter based on an outline version or part of the master motion information.

3. The information processing system according to claim 1, wherein the second motion information includes at least one of the following: one or more pieces of information identical to the master motion information extracted by the extraction processing unit, and other pieces of information generated based on the one or more pieces of master motion information extracted by the extraction processing unit.

4. The information processing system according to claim 1, further comprising a profile image generation unit that generates a profile image associated with the user, which includes the movement of an avatar associated with the user, based on the second motion information.

5. The information processing system according to claim 1, wherein the content generation unit further generates the animation content by associating background information, sound information, or text information with the second motion information.

6. The information processing system according to claim 1, further comprising a storage processing unit that stores new master motion information in the storage unit based on third motion information supplied by a specific user.

7. The motion acquisition unit acquires the first motion information in correspondence with the user's request, The information processing system according to claim 1, further comprising an output processing unit that outputs the extraction results of the extraction processing unit to the requester relating to the aforementioned request.

8. The information processing system according to claim 7, wherein the output processing unit outputs the second motion information if the request satisfies conditions relating to copyright or public order and morals.

9. At least a portion of the master motion information stored in the memory unit is such that the master's acoustic information is associated with the time series. The motion acquisition unit acquires, in addition to the first motion information, user-side acoustic information associated with the first motion information. The information processing system according to claim 1, wherein the calculation unit calculates the value of a predetermined parameter relating to the similarity between the user's acoustic information and the master's acoustic information stored in the storage unit when the user's acoustic information is acquired by the motion acquisition unit.

10. The information processing system according to claim 1, wherein the aforementioned time series of movement includes at least one of the following: a time series of movements relating to a person's facial expression, and a time series of movements of a jointed part of a person or object.

11. The aforementioned time series of movement includes the time series of movement of joints in a person or object. The first motion information relates to the movement of at least two or more body parts connected via the user's joints, The information processing system according to claim 1, wherein the calculation unit calculates the value of the predetermined parameter based on at least one of the joint angle, joint position, and inter-joint length.

12. The information processing system according to claim 1, wherein the extraction processing unit extracts one or more master motion information from among a plurality of master motion information stored in the storage unit, prioritizing the degree of similarity to the first motion information acquired by the motion acquisition unit.

13. The information processing system according to claim 1, wherein at least a portion of the master motion information stored in the memory unit is associated with information that can identify the hardware configuration used to generate the time series of motion.

14. The information processing system according to claim 1, wherein at least a portion of the master motion information stored in the memory unit is associated with information representing the originality of the motion or the originality of the information itself.

15. The information processing system according to claim 1, wherein the management information further includes whether processing is possible or whether deformation is necessary when using the master motion information.

16. First movement information is obtained regarding the movements of some or all of the users. The system calculates the value of a predetermined parameter relating to the similarity between the master motion information stored in a storage unit that stores master motion information representing the time series of movements of part or all of a person or object, and the acquired first motion information. Based on the calculated value of the predetermined parameter, one or more master motion information corresponding to the first motion information is extracted from among the multiple master motion information stored in the storage unit. If the management information attached to the master motion information stored in the memory unit includes information indicating that it is provided for a fee, then a usage restriction is set to restrict the free use of the master motion information. The extracted master motion information is displayed to the user in a visible manner, and a second motion information is generated in which the master motion information instructed by the user from among the one or more master motion information is reflected in the movement of the avatar associated with the user. Based on the generated second motion information, animation content for automatic playback is generated. When the playback conditions related to the user are met, the animation content is played, In generating the aforementioned animation content, the animation content to which emote information is associated is registered in the emote list, and the animation content is associated with at least one of the multiple emote buttons. A computer-based information processing method for playing the animation content, which includes determining that the playback condition is met when one of the multiple emote buttons is selected, and playing the animation content that is pre-registered in the emote list and corresponds to the selected emote button.

17. First movement information is obtained regarding the movements of some or all of the users. The system calculates the value of a predetermined parameter relating to the similarity between the master motion information stored in a storage unit that stores master motion information representing the time series of movements of part or all of a person or object, and the acquired first motion information. Based on the calculated value of the predetermined parameter, one or more master motion information corresponding to the first motion information is extracted from among the multiple master motion information stored in the storage unit. If the management information attached to the master motion information stored in the memory unit includes information indicating that it is provided for a fee, then a usage restriction is set to restrict the free use of the master motion information. The extracted master motion information is displayed to the user in a visible manner, and a second motion information is generated in which the master motion information instructed by the user from among the one or more master motion information is reflected in the movement of the avatar associated with the user. Based on the generated second motion information, animation content for automatic playback is generated. When the playback conditions related to the user are met, the animation content is played, In generating the aforementioned animation content, the animation content to which emote information is associated is registered in the emote list, and the animation content is associated with at least one of the multiple emote buttons. A program that, in playing the animation content, determines that the playback condition is met when one of the multiple emote buttons is selected, and causes the computer to execute a process to play the animation content that is registered in advance in the emote list and corresponds to the selected emote button.

Images