Computer program, server device, terminal device, and method

Abstract

To provide a computer program, a server device, a terminal device, and a method with which it is possible to properly express an avatar.SOLUTION: The flow processed by a system, by being executed by one or more processors, detects information pertaining to a prescribed behavior by a distributor, an expression of the distributor, a countenance of the distributor, a voice of the distributor, or one or more of a prescribed state occurring to the distributor, determines whether or not the detected information satisfies a prescribed condition, generates information for adopting the expression or pause of an avatar pertaining to the distributor and having been generated based on the expression or pause pertaining to the distributor as a prescribed expression or pause corresponding to the prescribed condition, when it is determined that the prescribed condition is satisfied, and displays an avatar to whom the prescribed expression or pause is applied.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The technology disclosed in the present application relates to a computer program, a server device, a terminal device, and a method. Regarding the law. [Background technology]

[0002] In recent years, technology using avatars played by streamers has been developing. Through the distribution service, viewers can watch the avatars played by the broadcasters. There are.

[0003] In addition, in relation to services using avatars, we will develop a system that allows users to express the facial expressions and movements of avatar objects. "Custom Cast" is a service that uses technology to control things based on the actions of people. A service called "Smartphone Live" is known (Non-Patent Document 1). A large number of facial expressions and movements are prepared for each of the multiple flick directions on the smartphone screen. You can assign a facial expression or action from the following works in advance, and then select the desired facial expression or action. The performer flicks the smartphone screen in the direction of the The displayed avatar object can be made to express its facial expressions or actions. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] "Custom Cast", [online], Custom Cast Inc., [Retrieved December 10, 2019], Internet (URL: https: / / customcast.jp / ) Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the technology disclosed in Non-Patent Document 1, the speaker uses the smartphone while speaking. The performer must flick the smartphone screen, and the performer must perform the flick operation. It can be difficult.

[0006] The above-mentioned Non-Patent Document 1 is incorporated herein by reference in its entirety.

[0007] Some embodiments disclosed in this application provide a more appropriate representation of an avatar. A computer program, a server device, a terminal device, and a method are provided. [Means for solving the problem]

[0008] According to one aspect, a computer program is executed by one or more processors. By this, a predetermined action by the distributor, the facial expression of the distributor, the complexion of the distributor, and (iii) detecting information relating to one or more of the voice of the distributor, or a predetermined situation occurring to the distributor. determining whether the detected information satisfies a predetermined condition; If it is determined that the facial expression or pose of the distributor is The facial expression or pose of the avatar related to the distributor is changed to a predetermined facial expression or pose corresponding to the predetermined condition. and generating information for the avatar to which the predetermined facial expression or pose is applied. and displaying the one or more processors.

[0009] Furthermore, the computer program according to one aspect is executed by one or more processors. By this, a predetermined action by the distributor, the facial expression of the distributor, the complexion of the distributor, Information relating to one or more of the voice of the broadcaster or a predetermined situation occurring to the broadcaster detecting information, determining whether the detected information satisfies a predetermined condition, and determining whether the predetermined condition is satisfied; If it is determined that the above condition is satisfied, the facial expression or pose of the broadcaster is generated based on the facial expression or pose of the broadcaster. The facial expression or pose of the avatar related to the distributor is changed to a predetermined facial expression corresponding to the predetermined condition. or pose, and the information for making the predetermined expression or pose is generated, and transmitting the one or more processors.

[0010] A server device according to one aspect includes one or more processors, executes computer-readable instructions to Movement, facial expression of the broadcaster, complexion of the broadcaster, voice of the broadcaster, or and detecting information relating to one or more of the predetermined situations that occur due to the detection of the information. If it is determined that the predetermined condition is satisfied, The facial expression or pose of the avatar of the distributor that was generated based on the facial expression or pose of the follower generates information for making the pose into a predetermined expression or pose corresponding to the predetermined condition. , and transmits information for achieving the predetermined facial expression or pose.

[0011] Also, a method according to one aspect includes one or more computer-readable instructions executing the computer-readable instructions. A method executed by a plurality of processors, comprising: a predetermined action by a distributor; the facial expression of the broadcaster, the complexion of the broadcaster, the voice of the broadcaster, or a predetermined situation occurring to the broadcaster. and detecting information relating to one or more of the above conditions, and determining whether the detected information satisfies a predetermined condition. If it is determined that the predetermined condition is satisfied, the facial expression or The facial expression or pose of the avatar related to the distributor that was generated based on the pose is Generate information for a predetermined facial expression or pose corresponding to a predetermined condition, and Or, the avatar is displayed with the pose applied.

[0012] Also, a method according to one aspect includes one or more computer-readable instructions executing the computer-readable instructions. A method executed by a plurality of processors, comprising: a predetermined action by a distributor; the facial expression of the broadcaster, the complexion of the broadcaster, the voice of the broadcaster, or a predetermined situation occurring to the broadcaster. and detecting information relating to one or more of the above conditions, and determining whether the detected information satisfies a predetermined condition. If it is determined that the predetermined condition is satisfied, the facial expression or The facial expression or pose of the avatar related to the distributor that was generated based on the pose is Generate information for a predetermined facial expression or pose corresponding to a predetermined condition, and Or, it transmits information for pausing.

[0013] In addition, a viewer terminal according to one aspect includes one or more processors, executes computer-readable instructions to Movement, facial expression of the broadcaster, complexion of the broadcaster, voice of the broadcaster, or and detecting information relating to one or more of the predetermined situations that occur due to the detection of the information. If it is determined that the predetermined condition is satisfied, The facial expression or pose of the avatar of the distributor that was generated based on the facial expression or pose of the follower generates information for making the pose into a predetermined expression or pose corresponding to the predetermined condition. and displaying the avatar with the predetermined facial expression or pose applied.

[0014] In addition, a distributor terminal according to one aspect includes one or more processors, executes computer-readable instructions to Movement, facial expression of the broadcaster, complexion of the broadcaster, voice of the broadcaster, or and detecting information relating to one or more of the predetermined situations that occur due to the detection of the information. If it is determined that the predetermined condition is satisfied, The facial expression or pose of the avatar of the distributor that was generated based on the facial expression or pose of the follower generates information for making the pose into a predetermined expression or pose corresponding to the predetermined condition. , and transmits information for achieving the predetermined facial expression or pose. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of a system according to an embodiment. [Figure 2] FIG. 2 is a block diagram schematically illustrating an example of a hardware configuration of the server device 20 (terminal device 30) shown in FIG. [Figure 3] FIG. 3 is a block diagram illustrating an example of the functions of a system according to an embodiment. [Figure 4] FIG. 4 shows an example of data held by a system according to an embodiment. [Figure 5] FIG. 5 shows an example of data held by a system according to an embodiment. [Figure 6] FIG. 6 shows an example of data held by a system according to an embodiment. [Figure 7]FIG. 7 is an example of a flow of processing performed by a system according to an embodiment. [Figure 8] FIG. 8 is an example of a flow of processing performed by a system according to an embodiment. [Figure 9] FIG. 9 is an example of a flow of processing performed by a system according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0016] Various embodiments of the present invention will now be described with reference to the accompanying drawings. It should be noted that components may be omitted from other drawings for clarity of illustration. Furthermore, although the attached drawings disclose one embodiment of the present invention, Please note that the figures are not necessarily drawn to scale. In this application document, the term "application" refers to software or programs. These may be called programs, which are instructions to a computer to produce a result. It is sufficient that the combination is such that the above can be achieved.

[0017] 1. System example FIG. 1 is a block diagram illustrating an example of a system configuration according to an embodiment. In this way, the system 1 includes one or more server devices 20 connected to the communication network 10 and a communication and one or more terminal devices 30 connected to the communication network 10. As an example of the server device 20, three server devices 20A to 20C are illustrated, and a terminal device 3 As an example of the server device 20, three terminal devices 30A to 30C are shown. In addition, one or more server devices 20 other than these may be connected to the communication network 10, and terminals One or more terminal devices 30 other than these may be connected to the communication network 10. In this application, the term "system" is used to include both servers and terminal devices. In some cases, the term is used to refer to only the server or only the terminal device. The system may be configured as a server only, a terminal device only, or both a server and a terminal device. may be.

[0018] The system may be an information processing device on a cloud. It constitutes a virtual information processing device and is logically configured as one information processing device. Also, the owner and administrator of the system may be different persons.

[0019] The communication network 10 may be a mobile phone network, a wireless LAN, a fixed telephone network, the Internet, an intranet, or the like. The communication may be a standard, such as a standard ethernet, a standard ethernet, and / or a combination thereof. The communication network 10 is not limited to the above. It can be trusted.

[0020] The server device 20 executes a specific application that has been installed. , a predetermined action by the broadcaster, the facial expression of the broadcaster, the complexion of the broadcaster, the voice of the broadcaster, or detects information relating to one or more of the predetermined situations occurring to the distributor, A determination is made as to whether or not the detected information satisfies a predetermined condition, and if it is determined that the predetermined condition is satisfied, When the above setting is made, the facial expression or pose of the broadcaster that has been generated based on the facial expression or pose of the broadcaster is The facial expression or pose of the avatar is set to a predetermined facial expression or pose corresponding to the predetermined condition. It may be possible to perform operations such as generating information for the purpose of

[0021] Alternatively, the terminal device 30 may be configured to use a web browser or a specific application installed on the terminal device 30. By executing the application, predetermined actions by the broadcaster, facial expressions of the broadcaster, One of the following: the complexion of the follower, the voice of the broadcaster, or a predetermined situation occurring to the broadcaster. Or, detect information related to a plurality of items and determine whether the detected information satisfies a predetermined condition. If it is determined that the predetermined condition is satisfied, a facial expression or a pose related to the distributor is displayed. The facial expression or pose of the avatar related to the distributor that has been generated based on the predetermined conditions is It can perform operations such as generating information to create a predetermined facial expression or pose corresponding to the stomach.

[0022] The terminal device 30 may be any terminal device capable of performing such operations. Smartphones, tablets, mobile phones (feature phones) and / or personal computers The device may be, but is not limited to, a computer, etc.

[0023] 2. Hardware configuration of each device Next, an example of the hardware configuration of the server device 20 will be described.

[0024] 2-1. Hardware configuration of server device 20 An example of the hardware configuration of the server device 20 will be described with reference to FIG. 1 is a block diagram showing an example of the hardware configuration of the server device 20 (terminal device 30) shown in FIG. (Note that in FIG. 2, the reference numerals in parentheses indicate the terminal devices 3, as will be described later.) 0.)

[0025] As shown in FIG. 2, the server device 20 mainly includes a calculation device 21, a main memory device 22, and an input device. The server device 20 may further include an output interface device 23. 24 and auxiliary output device 26. These devices are connected to each other via a data bus and and / or may be connected by a control bus.

[0026] The arithmetic unit 21 performs calculations using instructions and data stored in the main memory unit 22. The calculation result is stored in the main memory device 22. Furthermore, the calculation device 21 Through the input / output interface device 23, an input device 24, an auxiliary storage device 25 and an output device 26, etc. The server device 20 may include one or more computing devices 21. The computing device 21 may include one or more central processing units (CPUs), microprocessors, and / or may include a graphics processing unit (GPU).

[0027] The main memory 22 has a storage function and is connected to the input device 24, the auxiliary memory 25, and the communication network 10. etc. (server device 20 etc.) via the input / output interface device 23 and The main memory 22 stores data and the results of calculations by the calculation unit 21. RAM (Random Access Memory), ROM (Read Only Memory) and / or Flash This may include, but is not limited to, a memory.

[0028] The auxiliary storage device 25 is a storage device. The computing device 21 may store instructions and data (computer programs) that configure the above. These instructions and data (computer programs) are controlled by It may be loaded into the main memory 22 via the output interface device 23. 25 may be a magnetic disk device and / or an optical disk device, a file server, etc. Not limited to these.

[0029] The input device 24 is a device for inputting data from the outside, and includes a touch panel, buttons, keys, etc. It may be a board, a mouse and / or a sensor, etc.

[0030] The output device 26 may include a display device, a touch panel, and / or a printer device. The input device 24 and the output device 26 may be integrated. It may also be a modified version.

[0031] In such a hardware configuration, the arithmetic unit 21 performs the operations stored in the auxiliary storage unit 25. The instructions and data (computer programs) that make up a specific application are sequentially Next, the instruction and data are loaded into the main memory 22, and the loaded instruction and data are processed. Controlling the output device 26 via the interface device 23, or alternatively, the input / output interface and other devices (for example, the server device 20 and other terminal devices) via the device device 23 and the communication network 10. It may be possible to send and receive various information between the device 30 and the like.

[0032] The server device 20 has such a configuration and executes a specific application that has been installed. By performing the above, the following predetermined actions by the broadcaster, the facial expression of the broadcaster, The complexion of the distributor, the voice of the distributor, or a predetermined situation occurring to the distributor. and determining whether the detected information satisfies a predetermined condition. If it is determined that the predetermined condition is satisfied, a facial expression or pose related to the distributor is displayed. The facial expression or pose of the avatar related to the distributor that has been generated based on the predetermined condition is and generating information for a predetermined facial expression or pose corresponding to the In addition, such operations may be performed by the user. The user inputs the document of the present application using the input device 24 or the input device 34 of the terminal device 30 described later. The system of the disclosed embodiment may be operated by giving instructions. When the program is executed on the computing device 21, it is a service that serves as a system used by the user. The user may display the image on the output device 26 of the server device 20 or in a system that the user utilizes. The configuration may be such that the information is displayed on the output device 36 of the terminal device 30 as the information.

[0033] 2-2. Hardware configuration of terminal device 30 An example of the hardware configuration of the terminal device 30 will be described with reference to FIG. The hardware configuration of the device 30 is, for example, the same as that of each of the above-mentioned server devices 20. Therefore, the same configuration as that of each terminal device 30 can be used. Reference numbers for the elements are shown in parentheses in FIG.

[0034] As shown in FIG. 2, each terminal device 30 mainly comprises a calculation device 31, a main memory device 32, and an input device. an output interface device 33, an input device 34, an auxiliary storage device 35, and an output device 36; These devices are connected to each other by a data bus and / or a control bus. It is being done.

[0035] Arithmetic unit 31, main memory unit 32, input / output interface unit 33, input unit 34, auxiliary The storage device 35 and the output device 36 are included in each of the above-mentioned server devices 20. A calculation unit 21, a main memory unit 22, an input / output interface unit 23, an input unit 24, an auxiliary memory The device 25 and the output device 26 may be substantially the same or may be different. The capacity and capabilities of the computing devices and memory devices may vary. Position 36 is described in more detail below.

[0036] In such a hardware configuration, the arithmetic unit 31 performs the operations stored in the auxiliary storage unit 35. The instructions and data (computer programs) that make up a specific application are sequentially Next, the instruction and data are loaded into the main memory 32, and the loaded instruction and data are processed. Controlling the output device 36 via the interface device 33, or alternatively, the input / output interface between the device 33 and other devices (for example, each server device 20, etc.) via the communication network 10. It is possible to send and receive various information.

[0037] The terminal device 30 has such a configuration, and the terminal device 30 receives a predetermined action by the distributor, a facial expression of the distributor, and the The complexion of the distributor, the voice of the distributor, or a predetermined situation occurring to the distributor. Detecting one or more pieces of information and determining whether the detected information satisfies a predetermined condition If it is determined that the predetermined condition is satisfied, The facial expression or pose of the avatar related to the distributor that has been generated based on the predetermined conditions is changed. and generating information for making the facial expression or pose correspond to the predetermined facial expression or pose (which will be described in detail later). In addition, the installed software may be able to perform some or all of the following actions: By executing the web browser, the web page is received from the server device 20 and displayed. , the same operation can be performed. In addition, such an operation can be performed by the user using the input device 34. The system may be operated by giving instructions to the system of the invention disclosed in this application document. When the program is executed on the computing device 31, it is used as a system for the user. The configuration may be such that the information is displayed on the output device 36 of the terminal device 30 as the information.

[0038] In this application, the terminal device is referred to as two types of terminal device: a "distributor terminal" and a "viewer terminal." These may be names given to the roles of the terminals, and However, it may be a viewer's terminal, a distributor's terminal, or a viewer's terminal. The broadcaster terminal and the viewer terminal may be different. The broadcaster terminal is a terminal that allows broadcasters who appear as subjects in the broadcasted content, such as avatars, to On the other hand, a viewer terminal is a terminal that is mainly used to view what is being distributed. The terminal may be used by a viewer who is a person who is viewing the content. The act of gifting, which offers a gift to the body, makes the gift appear in what is delivered. Viewers may have some influence over what is broadcast, such as Participants may exert some influence on what is distributed in the form of voice, comments, etc. The content may be a video, a movie, or even an animation. Often, some of these will feature subjects such as avatars played by the streamer. The above description is merely an example of application of one embodiment of the present invention, and the present invention may be applied without such a premise. The embodiments of the present invention may also be applied to a virtual space in which a subject appears. In this application, animation may be virtual reality or augmented reality. That's fine.

[0039] As mentioned above, the viewer terminal can input voice, comments, and give gifts. For example, the viewer terminal may be configured as an input device 34 that allows a user to input comments. It has a keyboard, touch panel, etc. that can acquire comments as text information. The viewer terminal may also have a function to generate certain comments and gifts by the computing device 34. These acquired comments and gifts may also be transmitted to the input / output interface. These comments and gifts may be transmitted to the communication network 10 via the chair device 23. may be used to generate what is delivered.

[0040] In addition to the comments entered by the viewer terminal, the comments related to the present invention may also be entered by other means. Such comments may be automatically generated by the system. The video may be transmitted to the viewer terminal and displayed at the viewer terminal.

[0041] The terminal may also include a microphone capable of capturing sound as an input device 34. The voice is picked up by a microphone as an input device 34 and may be treated as sound or as an input. After being acquired by a microphone as the device 34, the text is converted by speech recognition by the computing device 31. The viewer terminal may be configured as a comment store by the computing device 31. These captured sounds or comments may also be transmitted to the input / output interface. The information may be transmitted to the communication network 10 via the chair device 23.

[0042] On the other hand, the distributor terminal has a motion sensor capable of detecting human movement as the input device 34. The motion sensor is attached to the broadcaster's body and can detect the movement of various parts of the body. The body parts may have various functions, such as the head, face, upper arm, forearm, hand, thigh, lower leg, The various parts of the body can be mentioned, such as the legs, trunk, etc., but are not limited to some or all of these. The sensor can be attached to a position where it can detect movement. The motion sensor may be capable of tracking facial expressions and poses and generating information. , an acceleration sensor, a gyro sensor, a geomagnetic sensor, etc. Information may also be transmitted to the communication network 10 via the input / output interface device 23. .

[0043] The distributor terminal may also include a camera capable of capturing video as the input device 34. The camera may capture the streamer's head, face, eyes, body part, whole body, etc. When capturing a head or face, it is possible to detect the direction of the face, and when capturing a face, it is possible to detect facial expressions. It is possible to detect the gaze of the body when capturing an image of the eyes. When capturing a partial or full body image, the motion of the body is detected together with or instead of the motion sensor. Based on this imaging information, it may be possible to know the face direction, line of sight, facial expression, posture, etc. The information acquired about the body movements is also transmitted via the input / output interface device 23 to the It may be transmitted to the communication network 10.

[0044] The output device 36 in the viewer terminal and the distributor terminal is a display device and a sound generating device. The display device may be any device capable of displaying information, such as a liquid crystal display (LCD). Displays, plasma displays, organic EL displays, micro LED displays These include various devices such as head-mounted displays and 3D displays. The sound generating device may be a speaker, a headphone, an earphone, etc. The information output by the device 36 may be information generated by the computing device 31, It may be information received from the communication network 10. The distributor terminal may It may be a studio device equipped with a dedicated sensor to acquire information on the performers. This has the advantage of being able to obtain more accurate information. May include performers.

[0045] 3. Functions of each device Next, an example of the functions of the server device 20 and the terminal device 30 will be described with reference to FIG. FIG. 3 is a block diagram showing an example of the functions of the system shown in FIG. is.

[0046] 3.1. Acquisition part The acquisition unit acquires information that can be used to determine whether or not a predetermined condition, which will be described later, is applicable. It has a function.

[0047] The information acquired by the acquisition unit includes a predetermined action by the distributor, a facial expression of the distributor, One or more of the following: facial color, the voice of the broadcaster, or a predetermined situation occurring to the broadcaster. It may include numerical information (sometimes referred to as predetermined operations, etc. in the present application). In addition, the information acquired by the acquisition unit is stored in an information processing device to which the system according to the present invention is connected. Therefore, the information may be detected information or information based on detected information. The information may be information according to the detector that detects it, and the detected information may be used as it is. The acquired information may be acquired by the acquisition unit, or the specific measurement information detected may be set to a predetermined threshold or The acquisition unit may acquire information corresponding to the range. ,motion data, information on the amount of alcohol consumed by the breathalyzer, The temperature information of the multi-sensor may be used. The corresponding information includes specific facial expressions and specific movement information about the motion data. , information on the amount of alcohol consumed in relation to the amount of alcohol, etc., according to the predetermined conditions of the present invention. The information acquired by the acquisition unit may be processed by the condition application unit as follows: The specific contents will be explained below. Note that Fig. 4 shows some of the information acquired by these acquisition units. An example is shown.

[0048] Here, various methods for detection may be used. For example, the detected information may be By image processing using images captured by one or more imaging devices that capture images of believers The information obtained by such image processing may include, for example, the motion Blend shapes based on motion data may also be detected. It detects sounds from a microphone that captures voices and detects the operation of sound-generating devices operated by the streamer. It may also be sound detection by detecting the movement of a sensor attached to the streamer's hand or arm. It is possible to use a sensor to detect hand or arm movements, or to use a sensor that wraps around the throat. , detecting throat movements, and detecting eating and drinking behaviors. The system according to the present application may detect the drinking behavior of the broadcaster by connecting with these detectors. The information about the streamer's actions may be detected by connecting the avatar. This may include performers who can create.

[0049] FIG. 5 shows examples of detection means and examples of detected information.

[0050] The information processing device from which the acquisition unit acquires information is an information processing device having the function of the acquisition unit. The term "acquire" also refers to the act of acquiring information from another device. It may be a higher concept of receiving information from an information processing device, or receiving information from a storage device. It may be a higher concept of reading information from another program via a storage device. It may be a higher concept of receiving information, or a higher concept of what an information processing device detects. It can be a concept.

[0051] 3.2. Condition application part The condition application unit determines whether the detected information satisfies a predetermined condition, and If it is determined that the conditions are met, a video image generated based on the facial expression or pose of the broadcaster is generated. The facial expression or pose of the avatar of the distributor that has been set is changed to a predetermined expression or pose that corresponds to the predetermined condition. It has the function of generating information for facial expressions or poses.

[0052] Here, the image of the distributor that has been generated based on the facial expression or pose of the distributor is displayed. The facial expression or pose of the avatar is set to a predetermined facial expression or pose corresponding to the predetermined condition. The generation of information for the purpose of the distribution of the information means that, when the predetermined conditions are met, The facial expression of the avatar related to the distributor, which has been generated based on the facial expression of the distributor, is changed to the predetermined condition. It is also possible to generate only information for making a corresponding predetermined facial expression, or to generate a pose related to the distributor. The pose of the avatar related to the distributor that has been generated based on the Alternatively, information on the facial expression and the like of the distributor may be generated. The facial expression and pose of the avatar related to the distributor, which have been generated based on the pose, are Information for creating a predetermined facial expression and pose corresponding to the predetermined conditions may be generated. In the present application, a predetermined facial expression and / or pose is specified in response to a predetermined condition. Regarding the expression, there are cases where it is called a predetermined expression, etc., and the expression and / or pose is also called an expression, etc. This sometimes happens.

[0053] The predetermined facial expression is based on a predetermined facial parameter or the facial expression of the broadcaster. The facial expression parameters are corrected based on a predetermined rule. The facial expression may be a facial expression displayed by a predetermined shape or movement setting. It may be based on the information, the parameters of a predetermined pose, or the pose of the broadcaster. The pose parameters are corrected based on a predetermined rule. The pose may be displayed by a predetermined shape or movement setting. Here, the predetermined facial expression parameters may be the parameters of each part of the face. and indicating a plurality of parameters that affect the movement of the By setting one or more parameters, a single facial expression may be displayed. The parameters can be any parameters that affect the movement of each part of the face. It can be about a specific area or about the whole face (or part of the face). The shape of the whole face (or part of the face) may be different for each parameter. However, by preparing multiple objects with different shapes, and combining some or all of them, It is a technique for expressing facial expressions. An example of the latter is blendshape. In this technique, a plurality of different facial shapes are prepared in advance, and the plurality of shapes are In this technology, a single facial expression is generated by synthesizing the facial expressions of normal faces. In addition to the elements, by preparing predetermined shapes in advance, you can create special expressions. For example, if a shape showing the vertical lines of a face is prepared in advance, the effect of the vertical lines of the face can be If you prepare elements for the effect of the cross in advance, you can In this case, since the parameters are set in advance, it is possible to Therefore, it is necessary to select appropriate parameters to represent facial expressions, etc. This has the advantage that it can always reproduce easy-to-understand facial expressions. In other words, the predetermined facial expression may be set to be different depending on the avatar. For Target B, the specified facial expression, etc. that is applied when the same specified conditions are met is , may be different for Avatar A and Avatar B. In this case, It may be set for each avatar, or may be set according to the type of avatar.

[0054] On the other hand, predetermined facial expressions, etc. are based on the facial expressions of the broadcaster, etc., and parameters such as facial expressions are set according to predetermined rules. In the case where the parameters of the facial expression etc. are corrected based on the For example, the streamer may express surprise in response to a gift. In this case, the avatar's facial expression changes to one or more expressions in response to the detection of the broadcaster's facial expression. The facial expression is set to a surprise expression according to the parameter, but the expression is set to a predetermined expression according to a predetermined condition. By increasing or decreasing the values of a plurality of parameters related to surprise as emotion by a predetermined value, This example shows a case where the expression of surprise is emphasized more by using the above-mentioned technique. While the technique is absolute in that one or more parameters are determined, In other words, it uses parameters relative to the information obtained from the current streamer's facial expression. Therefore, the same avatar played by the same broadcaster may be used under the same predetermined conditions. Even if the above conditions are applied, the distributor’s own facial expressions at the time of the broadcast will not be used. Parameters such as facial expressions may be the same or may differ, so The facial expression etc. according to the parameters may become the predetermined facial expression etc. It is possible to provide variations such as:

[0055] In addition, the facial expression displayed by the setting of a predetermined shape or movement may be a mask. This refers to a device that uses a predetermined shape itself as a predetermined expression, such as It is also possible.

[0056] In the above, we have explained about predetermined facial expressions, but similarly, Corresponding techniques may also be used for the size.

[0057] The predetermined facial expression may be an facial expression associated with a predetermined action. The viewer can easily understand the reason why a certain facial expression is expressed by associating it with a certain action. There is an advantage to being able to do this.

[0058] In addition, a predetermined facial expression affects the parameters of all parts that make up the facial expression. For example, a predetermined facial expression may be set to a parameter for the mouth. The parameters for parts other than the mouth are affected by the streamer's facial expressions. The parameter is based on information corresponding to the facial expression of the broadcaster from the broadcaster terminal. The predetermined facial expression may be a parameter. The predetermined facial expression may include the appearance of the avatar. For example, the predetermined facial expression may include an effect in which the avatar is burnt to a crisp. The parameters for the eye and mouth opening are numbers (e.g., 0% to 100%). It may also include parameters indicating the left and right positions of both ends of the mouth and the up and down positions of the eyebrows. The parameter may include a parameter indicating the upper or lower position of the

[0059] The predetermined pose may be a variety of poses. A pose may only involve a part of the avatar's body. For example, a tilted head pose can be used. There are poses with only the head, poses with only the hands, including those with enlarged or red, swollen hands, It can be a pose of just the upper body, like a fist pump, or a pose of holding a gun or the whole body. It can also be a full pose like a jump.

[0060] The predetermined conditions include predetermined movements of the avatar played by the broadcaster, facial expressions of the avatar, the facial expression of the avatar, the voice of the avatar, or a predetermined situation occurring regarding the avatar. Information relating to one or more of these may be used.

[0061] The predetermined action of the distributor is a predetermined action that is perceived by the distributor with the five senses. The predetermined action of the distributor may include an action of looking at a predetermined object. For example, Streamers may not interact with other streamers or certain objects in the streamer's location, such as on a stage or in a room. When a user is surprised at seeing a predetermined object, a surprised facial expression may be set as the predetermined facial expression.

[0062] In addition, the predetermined object is not limited to an object that brings surprise to the streamer, but also includes other emotions and sensations. It can be something that brings about fear, for example, something that you like, such as steak. When people see an object, they feel a certain emotion, such as "I like it," or "I dislike it," such as natto. It may be an object that sometimes evokes certain emotions or sensations. Regarding emotions, etc., predetermined expressions, etc., which represent the facial expressions that people generally make when looking at such objects are set. The above-mentioned things that bring about feelings of fear, things that bring about feelings of love, things that bring about feelings of dislike, For things that bring about feelings of disgust, there are corresponding expressions of horror, smiles, and disgust. It could be emotion, etc.

[0063] Also, if the predetermined target is another broadcaster, the action of looking at the predetermined target will be the same as the view of the first broadcaster. This may include looking into the eyes of other broadcasters, in which case such other broadcasters may also If you look into the eyes of a streamer, your eyes will meet with the other streamer's. In this case, the avatar may be set to a specific facial expression. In this case, a predetermined facial expression may be set for only one of the avatars, or both avatars may have the same facial expression. For example, a predetermined facial expression may be set for the target. The avatar's face may be red or excited. When a predetermined facial expression is set for each avatar, such set facial expression is The facial expressions may be the same or different. The reason for the different predetermined facial expressions is that the predetermined facial expressions depend on the type of avatar to which the user belongs. It may be a predetermined facial expression according to the avatar's preferences, characteristics, etc., or it may be a certain facial expression. It may also be a predetermined facial expression selected arbitrarily.

[0064] In this case, the broadcaster is photographed by an imaging device whose field of view is the broadcaster and its surroundings. In the captured image, the distance between the broadcaster and other broadcasters is within a predetermined distance. When the broadcaster sees another broadcaster, a predetermined facial expression may be set. When a streamer looks at another streamer, the streamer's gaze is reflected in the image as an image processing. By using a function such as ycast, the direction of the streamer's gaze is calculated, and the streamer's gaze is compared to other streamers. If the broadcaster crosses the screen, it may be determined that the broadcaster is viewing the other broadcaster. Similarly, if the distance between the broadcaster and the object falls below a certain distance, or if the broadcaster sees such an object, In this case, a predetermined facial expression or the like may be set. The predetermined target may be determined in advance, and the touching action may include: As described above, the broadcaster is photographed by an imaging device whose imaging field of view is the broadcaster and its surroundings. In the captured image, the distance between the broadcaster and a predetermined object is less than a predetermined distance. When the distance between the streamer and a specified object becomes 0, a specified facial expression will be displayed. good.

[0065] In this regard, depending on the position of the imaging device capturing the image of the broadcaster, the broadcaster may come into contact with a specific object. The timing and the contact between the broadcaster's outline and the outline of the specified object in the image are different In particular, the latter case involves the image being displayed even though the distributor himself has not come into contact with the specified object. In the image, the outline of the broadcaster and the outline of a specific object may overlap. In addition to or instead of the judgment using image processing, a sensor worn by the broadcaster can be used to For example, a distributor may wear gloves equipped with sensors, and the gloves may be used in conjunction with a predetermined The device may be configured to detect contact with an object by a sensor inside the glove. The sensors installed in the device detect situations where the device comes into contact with other streamers or other objects. The sensor attached to the body of the broadcaster may be a marker-type tracker. In addition, an object may be equipped with one or more trackers, The relationship between the trackers on the camera and the trackers on the streamer's body determines The relationship between the believer and the object, such as contact, may be determined.

[0066] The predetermined action of the broadcaster may include a handshake action. The finger movements were detected by the attached gloves equipped with sensors, and the finger movements of the handshake were detected. In this case, it may be determined that they are shaking hands.

[0067] The predetermined action of the broadcaster may include an eating and drinking action of a predetermined target. A sensor is placed around the throat of the broadcaster to detect throat movements. It may be possible to detect whether a follower is eating or drinking. The prescribed actions include the streamer's avatar moving its mouth with food in it. For example, the facial expression may be one in which the person is chewing something. The facial expression may be responsive to mouth movements, which may correspond to the throat movements described above. Also, in addition to or instead of the above techniques, A predetermined action may be set based on information in an image of the believer. The method determines whether a person is eating a predetermined food based on information in the image, and For example, when a streamer eats ramen, When eating curry, meat, fish, vegetables, etc. Place the corresponding food items (ramen, curry, meat, fish, vegetables, etc.) While producing it as an item or animation in the imaginary space, For example, when eating meat, the person may eat meat from the side with the bone. You can eat it by biting into it, or if it's curry, you can eat it with a spoon. The system may generate items that represent specific objects in the virtual space. If the facial expression is a "Bell" expression, a corresponding food item such as those mentioned above may be generated and displayed.

[0068] In the above, items that are displayed in the virtual space as a part of the presentation correspond to eating and drinking actions. The food and drink items that are listed above are the same as those listed above. For example, items that correspond to what the streamer perceives in real space. For example, the sound of a musical instrument may be heard. If you select a specific item, the item may be a musical instrument. When you make contact with a follower, the corresponding tub or ball item will appear on the screen. It may be displayed as a dramatic effect.

[0069] In addition, the items displayed as such effects may be used to display the avatar of the streamer with a predetermined facial expression, etc. The avatar may be displayed at the timing set in the "Avatar" field. If the specified facial expression, etc. set by the user returns to the facial expression, etc. of the avatar corresponding to the original broadcaster, Items displayed as a special effect may disappear from the screen. The system responds to the avatar of the distributor while it is set with a predetermined facial expression, etc. The system may be displayed.

[0070] In addition, the items displayed as part of such effects are related to the specific actions of the streamer. The position of the item may be determined and moved. For example, in response to a predetermined action by the broadcaster, The position and movement of the corresponding item are controlled by the volume of the sound, the speed of the action, the distance of the action, etc. More specifically, for example, if the sound is strong, the position of the item may be changed. If the sound is weak, the item may be located far away. The faster the action is performed, the faster the food items will decrease. When a ball comes flying, a ball item corresponding to the ball is displayed in the virtual space. The ball in the virtual space is displayed, and the speed and size of the ball that is flying in the real space are displayed. In this case, the speed and size of the ball in real space may be displayed. The speed of the ball in the virtual space is determined by the speed of the ball in the real space. The speed and the ball may be associated with each other, or the speed and the ball may be associated with each other.

[0071] In addition, when a broadcaster drinks alcohol as a predetermined action, an action related to the broadcaster is displayed as a predetermined facial expression, etc. You can make Butter blush and express the effects of alcohol. The alcohol detected by the breathalyzer may be determined by a breathalyzer. For example, if the amount of alcohol in the drink exceeds a first predetermined amount, the avatar's facial color may change. If the amount detected is within the range of 100, the avatar's face will be colored pink and the amount of alcohol detected will be If a second predetermined range amount is detected that is greater than the first predetermined range amount, the avatar The face color is set to red, and the amount of alcohol is set to a third predetermined range that is greater than the amount within the second predetermined range. If an amount within the range is detected, a dark red color may be set.

[0072] The predetermined action of the broadcaster may include smelling or a predetermined gesture of smelling. The gesture may include a nose movement. For example, a nose sniffing movement. When the person is making gestures that indicate that they can smell something, such as twitching or trembling, The predetermined gesture for smelling may include a gesture of closing one's nostrils or breathing. The actions of stopping inhalation or moving the air in front of the nose with the hands, etc., that cause the average person to sense the smell The predetermined smelling gesture may include a gesture of nose and face movement. The motion may be only a finger movement, only a hand movement, or both. It may be a facial expression or the like.

[0073] The broadcaster's smell or smell-like gesture is based on motion data and is blurred. The nose information may be acquired based on the shape of the nose. You can also obtain information by doing so.

[0074] In addition, the scent of the broadcaster can be used as a predetermined facial expression of the avatar corresponding to a predetermined action of the broadcaster. In response to the action of smelling or stinking, the avatar of the distributor may smell or make a facial expression that suggests a smell. Similarly, the avatar's smell or odor expression as a predetermined expression may be used to identify the smell or odor of an ordinary person. It may be a predetermined smell or smelling action that gives the impression of smelling something. The smell or smell-like expressions of the avatar are of the same type as the smell or smell-like expressions of the broadcaster. It may be a facial expression, or different facial expressions. In this case, the smell or smelly facial expression by the avatar may be used to By increasing the range of movement and clarifying the speed of movement, It is acceptable to make it clear that the reason is emotional, etc.

[0075] The predetermined action of the broadcaster may be a predetermined gesture regarding listening to / listening to a sound. The predetermined gestures for listening / listening to sounds may include movements of the ears only. It may be a movement of only the head, or a movement of only the hands, or some or all of these. It can also be a movement. For example, ear movement can be moving the ear. For example, head movement can be moving the ear. One example is pointing your ear in a certain direction. The certain direction is where the streamer feels they can hear the sound. The hand movement may be a movement of bringing the hand close to a predetermined range of the ear. The hand gesture may include a gesture of turning the palm toward the ear, and a specific It can be shaped to collect only sounds coming from one direction. It can also be done with one hand only, or with both hands. It can also be a hand movement. A form of ear that focuses only on sounds coming from a specific direction. When you place your palm on the ear, change your body position so that your ear is closer to the specific direction. The streamer's actions are similar to those of ordinary people who hear the sound. This may include gestures that are recognized as actions to listen / hear. , facial expression, etc. In response to the facial expression, etc. of the distributor, As for the predetermined facial expressions, the facial expressions of an ordinary person trying to listen to a sound, as mentioned above, The facial expression may be one that recognizes the user as a person. The facial expressions, etc. of those who are listening to / trying to listen to the sound made by the broadcaster are the same as those of those who are listening to / trying to listen to the sound made by the broadcaster. Alternatively, the facial expressions may be the same or different. In this case, the facial expressions of the avatar will be those of the streamer listening to the sound, Try to listen to the sound by increasing the range of movement and clarifying the speed of the movement. It is acceptable to make it clear that the facial expression shows that the person is trying to

[0076] Furthermore, the predetermined situation occurring to the distributor may include the occurrence of a predetermined sound. The predetermined sound may be a human voice, a natural sound, or a sound produced by a musical instrument. It may be a sound, an artificial sound, or a dramatic sound, and there is no restriction on the type of sound. There is no need to limit the number of people who can make a sound. The sounds of nature may be the sounds of wind, water, fire, etc. The sounds of instruments can be anything from the sound of a typhoon to the sound of a flood. The artificial sounds may be, for example, the sounds of various instruments such as piano, flute, etc. There are various sound effects, such as a "ping pong" sound to indicate a correct answer and a "boo" sound to indicate an incorrect answer. The dramatic sound may be, for example, a sound that is generally used when a ghost appears. It may be a sound that is recognizable as a ghost, such as a whoosh or slosh sound that ordinary people can recognize as a ghost's appearance. .

[0077] The predetermined sound may be detected by various methods. For example, the predetermined sound may be detected in the broadcaster's room, studio, etc. The system may detect sounds that the streamer can hear with a microphone installed in the studio. The sound generated by the operator's instructions or the streamer's own input is used to Alternatively, information may be detected from a sound generating device that can emit a sound.

[0078] In response to the generation of such a predetermined sound, the avatar of the distributor may display a predetermined facial expression, such as: For example, if a strange voice or a scream is detected, As a predetermined facial expression, the avatar may be set to make a strange noise or scream. is an ordinary person screaming or making strange noises, with their mouth open and their eyes and eyebrows moving up and down. You can set it to a recognizable facial expression. Also, a sound effect such as a ping-pong sound will be detected to indicate the correct answer. When the user is prompted, the avatar may have a predetermined facial expression such as an expression of inspiration. As a part of the virtual space production, to show that the avatar has had an idea, the avatar's head An exclamation mark may be displayed around a predetermined area. In response to the sound of a ghost appearing, the avatar responds with a predetermined facial expression. If the sound of a musical instrument is detected, the robot may show a frightened expression. The avatar may also have a singing facial expression. In this case, the avatar's mouth moves in time with the sound of the instrument. For example, the mouth of an avatar may move in response to changes in the sound of a musical instrument. Depending on the amount of the broadcast, a predetermined facial expression or the like of the avatar of the broadcaster may be set. When a sound with a volume greater than the threshold is detected, the avatar of the broadcaster will have a predetermined facial expression, such as an ear A pause may be set to block the sound. , the distance between the ear and the hand is shorter than a predetermined distance as a predetermined facial expression of the avatar of the distributor, etc. The distance between the ear and the hand may be set to touch the ear, and a pose with the hand placed on the ear may be set. Depending on the quality of the sound, a predetermined facial expression or the like of the avatar of the broadcaster may be set. When a hoarse voice set as a specific sound quality is detected, the broadcaster's avatar will display a specific facial expression, etc. A stern expression may be set as the predetermined tone quality. When this is detected, an intoxicated expression is set as the predetermined expression, etc., of the avatar related to the broadcaster. It is okay to do so.

[0079] The predetermined situation occurring to the distributor may be a predetermined object coming into contact with the distributor. The predetermined object may be a variety of objects, such as a ball, a bat, or a bucket. However, if an object actually comes into contact with the distributor, for safety reasons, In addition, it may not be the real thing, but an imitation of the real thing. In response, the avatar of the distributor may have a corresponding facial expression, etc. For example, the facial expression may be surprised or painful due to contact. As a part of the imaginary space, to show that the avatar is in pain, the avatar's head A star mark or other symbol generally indicating pain may be displayed around a predetermined area from the affected area.

[0080] The face color of the broadcaster may include a change in face color. The change in facial color may be determined based on image processing of an image of the person. The temperature of the face of the streamer may be measured using a temperature sensor. Also, changes in facial color may be detected by other This information may be used to set a predetermined facial expression, etc. The degree of drinking may be detected using the information obtained by detecting the message. The complexion changes from normal skin color to pink, red, etc., and the amount of alcohol detected A predetermined facial expression or the like of the avatar may be set corresponding to the degree of the amount of the The predetermined facial expression of such an avatar may be, for example, the expression of someone drunk with alcohol, as described above. In addition, the expression of a person drunk with alcohol may be expressed by moving the avatar's legs or Body movements may include staggering, and the skin may change from normal to pale or white. By using a change in facial color and a predetermined item such as a ghost character, a predetermined item related to the avatar is displayed. You can set a scared expression as an expression, etc.

[0081] Furthermore, the voice of the distributor is such that the tone quality of the voice of the distributor changes by more than a predetermined range, or , the volume of the voice of the broadcaster changes by more than a predetermined amount. The amount may be determined using information detected from a microphone. If there is a change in the facial expression, a predetermined facial expression or the like may be set according to the change. For example, the quality and / or volume of the broadcaster's voice may be detected in combination with other information. Information based on an image obtained by an imaging device that captures an image of the believer, and a predetermined avatar More specifically, for example, when the volume of the voice of the broadcaster is lower than a predetermined volume, The distance between the broadcaster's hand and mouth is set to a predetermined value. If it is determined that the avatar is within the predetermined facial expression, etc. of the avatar related to the distributor, may cause the character's face to turn red and eyes to close, in which case a megaphone may also be displayed, and / or Alternatively, an illustration of thunder may be displayed to indicate the volume of the voice. The illustration may be sized according to the volume of the voice. If the voice is detected as lower than the specified sound quality and the distance between the streamer's eyebrows is longer than the specified length, If it is determined that the video is too short, the broadcaster's avatar's facial expression, etc. will be displayed as a predetermined expression of the broadcaster's avatar. You can put a mark on your face to show anger. Here, if the distance between your eyebrows is shorter than a certain length, , so to speak, a state of wrinkled brows. Also, if the streamer is "transforming" When a specific keyword is detected and the streamer is judged to have taken a specific pose, When the avatar is updated, a predetermined transformation animation will be displayed as the predetermined facial expression of the avatar of the streamer. For example, the streamer may move one hand in a circular motion in front of the body while simultaneously For example, if you say "Transform", the transformation animation of Kamen Rider will be activated. The facial expression may be displayed as a predetermined facial expression of an avatar related to the distributor.

[0082] The detected information may include information detected as a result of input by a third party. For example, if the stream is being distributed in a studio, the content may be entered by a third party other than the director. For example, if the director answers a question correctly, The ping-pong sound in this case may be detected in addition to or instead of detecting the sound itself. The information may be information detected by a third party who inputs a ping-pong sound. ,Information other than sound, for example, when a performer comes into contact with other people or things, including other performers, A third party may enter information indicating contact in the event of food contact, or the director may Regarding eating and drinking actions, a third party may input corresponding information, or the smell or A third party may input information about the occurrence of an odor. The information that can be perceived by the senses may be information that detects input by the distributor himself or a third party.

[0083] In addition, there are cases where a plurality of predetermined conditions are satisfied. For example, When a third broadcaster contacts a broadcaster while viewing another broadcaster at the same time, etc. In such a case, it is necessary to view other broadcasters and the distance from the third broadcaster is within a predetermined distance. If the predetermined facial expressions, etc. corresponding to each condition are different, multiple predetermined conditions are For example, if a first predetermined condition is satisfied within a predetermined time, If the condition is satisfied and the second predetermined condition is satisfied, In the second predetermined condition, when a predetermined condition set with a high priority is satisfied, A corresponding predetermined expression or pose may be applied, where a priority is assigned to each predetermined condition and the associated The rule may be assigned and stored when multiple predetermined conditions are met. For example, if the predetermined conditions are A1, A2, A3, A4, etc. In this case, the predetermined conditions can be compared with each other and the predetermined condition to be prioritized can be selected. If multiple predetermined conditions are satisfied by the function using such priority, Even in cases where multiple predetermined conditions are met almost simultaneously, This has the advantage that the priority can be expressed as a probability. For example, the probability may be expressed as follows: When the determination is made by a pseudorandom number function or the like, a first predetermined condition and a second predetermined condition are satisfied. If the above condition is satisfied and the pseudorandom number function value is equal to or less than a predetermined value, the first A predetermined facial expression or the like corresponding to a predetermined condition is applied, and if it exceeds a predetermined value, a second predetermined A predetermined facial expression or the like corresponding to the condition may be applied. This will increase the number of opportunities for viewers to watch, which may attract more interest from viewers.

[0084] In addition, when determining whether multiple conditions can be satisfied, the judgment is made after a predetermined time has elapsed. Therefore, if the predetermined time is set to a certain time or more, the predetermined condition will be satisfied and the predetermined There may be a time lag before the facial expressions are even produced. When the above multiple conditions are met, the first condition that is met is Here, the term "preceding in time" refers to the timing of the network or information Even if the predetermined conditions are met substantially at the same time as a result of processing, the processing unit The satisfaction of the predetermined condition may be determined first. In this case, the predetermined time may be elapsed before the determination is made. This has the advantage that a predetermined facial expression or the like corresponding to a predetermined condition that has been satisfied first can be set without first having to select the condition. In such a case, a predetermined facial expression or the like corresponding to the predetermined condition that is subsequently satisfied may be set. If it is set, it is necessary to set a predetermined table corresponding to the predetermined condition that was satisfied first. Although emotions are overwritten, certain expressions corresponding to certain conditions that are later met are also viewed. On the other hand, if you do not set it, the viewer will not be able to see the predetermined facial expression etc. The listener can fully enjoy the sound. If both of the above conditions are satisfied, the timing when the first condition is satisfied is A predetermined facial expression may be set according to the timing when the second predetermined condition is satisfied. For example, a first timing when a first predetermined condition is satisfied and a second timing when a second predetermined condition is satisfied are set as follows: If the time between the second timing and the first timing is shorter than a predetermined time, the first predetermined condition is satisfied. and only a first predetermined facial expression, etc., corresponding to a second predetermined condition is produced. In this case, the viewer may select the first predetermined facial expression or the like as follows: The advantage is that it can be enjoyed without being changed midway. If the time between the second timing is longer than a predetermined time, the second predetermined condition is satisfied. At the timing of the addition, a predetermined facial expression or the like corresponding to the second predetermined condition may be set. In this case, the viewer can enjoy both the first predetermined facial expression and the second predetermined facial expression. This has the advantage of being able to

[0085] The above-described method for determining the time between the first timing and the second timing is also applicable. The predetermined time may be set depending on a set of predetermined conditions being met. For example, When a predetermined condition is met, a first predetermined expression is produced, and when a second predetermined condition is met, When a predetermined condition is met, a second predetermined expression is produced, and when a third predetermined condition is met, a third predetermined expression is produced. Furthermore, a first predetermined condition is satisfied at a first timing, The second predetermined condition is satisfied at a second timing, and the third predetermined condition is satisfied at a third timing. At this time, the time between the first timing and the second timing is If the time is shorter than the 12th predetermined time, the second predetermined facial expression etc. is not produced, and the 12th predetermined time In the above cases, the second predetermined facial expression is produced. When the time between the timings is shorter than the thirteenth predetermined time, the third predetermined facial expression, etc. If the thirteenth predetermined time is not reached, the third predetermined facial expression is displayed. In this case, the 12th predetermined time and the 13th predetermined time may be the same or different. If they are the same, the second predetermined condition is satisfied after the first predetermined condition is satisfied. Even if the third predetermined condition is met, the duration of the first predetermined facial expression, etc. will not change. On the other hand, if they are different, after the first predetermined condition is satisfied, the second predetermined condition is satisfied. When the above condition is satisfied and when the third predetermined condition is satisfied, the first predetermined facial expression, etc. In this case, the variety of performances will increase. This may be able to attract more viewers' interest.

[0086] It should be noted that in the above, a priority is associated with or satisfies a predetermined condition. We have explained examples of rules for specific conditions, but in addition, priority can be set based on the items within the specific conditions. , events, situations, etc.

[0087] An object, event, or situation related to a specified condition (sometimes referred to as an object, etc. in this application document) The priority of the above is determined based on the size of the object, the location where the object appears, the timing of the object's appearance, It may be generated using the following:

[0088] For example, the displayed size of an object may be larger than the size of a first object. If the size of the displayed item is large, the second item will be larger than the first item. In this case, the larger the image, the more likely the viewer will be interested in it. This is because it attracts attention and it is believed that giving it priority will be in line with viewers' wishes. On the other hand, the opposite may be true. This has the advantage of creating a sense of surprise.

[0089] In addition to the size of the item, if the item has a price, that may be used. If consumption points are set, they may be consumption points. For example, the first item, etc. If the price of the second item is higher than the price of the first item, In this case, the higher the price, the more viewers it will attract. This is because it attracts a lot of attention, and it is believed that giving it priority will be in line with the wishes of viewers. On the other hand, you can also set it the other way around. If you set it the other way around, it has the advantage of being unexpected. Similarly, for example, the points consumed for the second item are more valuable than the points consumed for the first item. If the consumption points of the second item are higher, the second item will be given a higher Priority may be set, in which case the higher the price, the more viewers will pay attention to it. Therefore, it is believed that giving priority to this will be in line with the wishes of viewers. Conversely, setting it to "0" has the advantage of creating an element of surprise.

[0090] The location of an object in the virtual space may be, for example, an object that appears in the center of the virtual space. The priority may be set higher than that of objects that appear at the left or right edge of the virtual space. If the priority of an object that appears in a conspicuous place in the image is increased, the viewer will Since the viewer can see the objects that appear in the location, the viewer can easily understand the meaning of a given facial expression. It has the advantage of being easier to understand.

[0091] Furthermore, the priority of an object may be static information that is set in advance for the object. Alternatively, it may be dynamic information that is set only when the product is provided. For example, the information may be set according to the position where the object or the like appears. The priority may be set according to the relationship between the position where the user appears on the screen and the position of the avatar. In this case, distance itself may play the role of priority, or distance may be the only thing that sets priority. For example, the distance between the avatar and the first object may be used as information on the distance between the avatar and the first object. is the first distance, and the distance between the avatar and the second object is the second distance. In this case, if the first distance is longer than the second distance, the first item is given higher priority than the second item. In addition, a first distance between the center of the screen and the position of the first object may be set. If the distance between the center of the screen and the position of the second object is shorter than the second distance, the first object is closer to the center of the screen. The first item may be given a higher priority than the second item because it appears in the center. When priorities related to facial expressions, etc. are dynamically set, it is possible to more appropriately and dynamically display predetermined facial expressions, etc. The advantage is that you can choose the gift that will be set.

[0092] In addition, when a predetermined condition is met, the predetermined facial expression, etc. is applied. For example, the predetermined condition may be a type to which the avatar belongs. using a type to which the avatar belongs, the type being one of a plurality of types to which the avatar belongs; This may be the case when certain information is acquired, and the first avatar type has a predetermined facial expression. However, the second type of avatar may not have a predetermined facial expression. For example, a certain type of avatar may be set to have a weak personality, which may lead to a lack of interaction with other streamers. When the distance is within the first distance range, a surprised expression may be easily produced. The avatar is set to have a cheerful personality, and the predetermined facial expressions include a smile. The conditions for the avatar may be set to be looser than those for other avatars. The target is given personality traits such as timid, cheerful, strong-willed, and competitive. The satisfaction of a predetermined condition may be determined using a characteristic called personality. It may be set according to the type to which the avatar belongs, or it may be set separately from the type to which the avatar belongs. It may be set for each individual avatar. The type may be the personality set when the avatar is set, or it may be the personality set when the avatar meets certain conditions. This may be determined based on the parts and clothing equipped at the time of determination.

[0093] The predetermined condition may include a probability. In other words, the predetermined condition may be determined by using a pseudo-random number function or the like. When it is determined that the function value satisfies a predetermined condition, a predetermined facial expression or the like is generated. If such a probability is included, a predetermined facial expression or the like is not always generated, so It is possible to continuously attract the interest of the listener. When using a rare facial expression, viewers can see a precious facial expression. This can attract the interest of viewers.

[0094] The condition application unit applies the facial expression, etc., specified by the satisfaction of a predetermined condition to the distributor. It may be the facial expression of the avatar played by the broadcaster. More specifically, it may be the fulfillment of a predetermined condition. The information indicating the facial expression, etc., identified by the feet is stored as an ID indicating the identified facial expression, etc., and the The facial expression may be one or more parameters indicating a facial expression, etc., and the identified facial expression, etc. may be In order to make the facial expression of the avatar, the information indicating the specified facial expression, etc. is used to You can set the expression of the butter. Such ID and parameter information is used in the condition application section. Once identified or generated, such information is then transferred to a server or terminal for the generation of the video or animation. The condition application unit may be configured to apply a condition to the user's device associated with each predetermined condition. The information indicating the specified facial expression or the like may be stored, and when each of the predetermined conditions is satisfied, In this case, the information indicating the corresponding specified facial expression or the like can be specified. A predetermined facial expression, etc., that is applied when a predetermined condition is met is displayed on the avatar for a predetermined period of time. After that, you can return to your original avatar behavior. In other words, the avatar will not behave as if it were performed by the broadcaster. After a certain period of time, the avatar's facial expression and pose will be the same as the broadcaster's. Here, based on the information of the broadcaster's facial expression or pose, The avatar's movements are controlled based on the followers' facial expressions and poses detected by optical and inertial sensors. may be defined.

[0095] A first information processing device in which a condition application unit is executed, and a setting device for setting information indicating the facial expression of the avatar, etc. When the second information processing device that performs the process including the predetermined setting is the same as the first information processing device that performs the process including the predetermined setting, the condition application unit By using the facial expression identified by satisfying the above condition, information indicating the facial expression of the avatar is set. It may be processed as follows.

[0096] A first information processing device in which the condition application unit is executed and information indicating the facial expression of the avatar are also included. If the first information processing device is different from the second information processing device that performs processing including setting of information, a condition application unit applies a predetermined condition The first information processing device transmits information indicating the facial expression or the like identified by satisfying the above to the second information processing device. and in the second information processing device, using the information indicating the specified facial expression, etc., Processing may be performed to set information such as the facial expression of the avatar.

[0097] Here, if the first information processing device is a server and the second information processing device is a server, The server of the first information processing device transmits the specified facial expression to the server of the second information processing device. Further, the first information processing device may be a server, and the second information processing device may be a server. When the processing device is a viewer terminal, the server of the first information processing device sends the Information indicating the identified facial expression or the like may be transmitted to the viewer terminal. When the first information processing device is a server and the second information processing device is a distributor terminal, The server of the second information processing device transmits information indicating the specified facial expression, etc. to the distributor terminal of the second information processing device. may be transmitted.

[0098] Similarly, when the first information processing device is a viewer terminal and the second information processing device is a server, In this case, the specified information is transmitted from the viewer terminal of the first information processing device to the server of the second information processing device. In addition, when the first information processing device is a viewer terminal, When the second information processing device is a viewer terminal, the viewer terminal of the first information processing device Information indicating the identified facial expression or the like may be transmitted to the viewer terminal of the second information processing device. In addition, when the first information processing device is a viewer terminal and the second information processing device is a distributor terminal, In this case, the viewer terminal of the first information processing device transmits the special information to the distributor terminal of the second information processing device. Information indicating the determined facial expression, etc. may be transmitted.

[0099] Similarly, when the first information processing device is a distributor terminal and the second information processing device is a server, In this case, the specified information is transmitted from the distributor terminal of the first information processing device to the server of the second information processing device. In addition, when the first information processing device is a broadcaster terminal, When the second information processing device is a distributor terminal, the distributor terminal of the first information processing device sends the Information indicating the identified facial expression or the like may be transmitted to the distributor terminal of the second information processing device. In addition, when the first information processing device is a distributor terminal and the second information processing device is a viewer terminal, In this case, the special information is transmitted from the distributor terminal of the first information processing device to the viewer terminal of the second information processing device. Information indicating the determined facial expression, etc. may be transmitted.

[0100] In the above, the condition application unit is executed in a single information processing device. The condition application unit may be distributed among a plurality of information processing devices.

[0101] The condition application unit also uses the timing at which the predetermined facial expression or the like is set to determine whether the predetermined facial expression or the like is being used. The timing may be set, for example, as described above, depending on the timing of the one broadcaster and the other broadcaster. The timing when the gaze of the person meets with the avatar or character, and when the avatar becomes an object. In this case, the timing of contact may be included in the information showing the predetermined facial expression, etc. The information including the information may be transmitted to an information processing device and displayed thereon. The timing discrepancy may include a time lag in communication or processing.

[0102] On the other hand, the timing when information indicating a predetermined facial expression is set after a predetermined condition is satisfied. For example, after a predetermined condition is set, For example, the avatar may change to a certain facial expression after a certain number of seconds. When a surprising situation occurs, the robot will intentionally delay the timing by two seconds, and then set a predetermined surprised expression. It may also be something that specifies the

[0103] Figure 6 shows an example of the relationship between predetermined conditions and predetermined facial expressions. By using this as a predetermined condition, a corresponding predetermined facial expression or the like can be identified. .

[0104] As described above, when a predetermined condition is satisfied, the function of the condition application unit is to apply the corresponding This allows for a variety of facial expressions and poses for avatars in virtual space. It also has the advantage of allowing the performer to have a real-life experience. In response to the above, it is possible to set facial expressions etc. for the avatar in the virtual space. The information for the facial expression or pose is information for operating the avatar from the broadcaster terminal. In this case, the actor may generate the situation without acquiring it. Without the burden of inputting changes in the avatar's facial expressions, etc., and without any information from the performer, the avatar's The advantage is that facial expressions can be changed.

[0105] 4. Embodiment 4-1.Embodiment 1 In the first embodiment, the condition application unit operates in the viewer terminal. The unit may operate in the viewer terminal. This example will be explained with reference to FIG. In the viewer terminal, a condition application unit operates, and information based on the viewer terminal is applied under a predetermined condition. By including this as a condition, it is possible to change the avatar's appearance for each viewer device. It's okay.

[0106] Step 1 The acquisition unit acquires information on predetermined actions and the like (step 001). The information may be acquired at the distributor's terminal or at the server. These may be acquired by the viewer terminal on which the condition application unit operates via a server or the like. Good too.

[0107] Step 2 Next, the condition application unit determines whether a predetermined condition is met using at least information on the predetermined action, etc. If it is determined that the predetermined condition is satisfied, the facial expression or the like corresponding to the satisfied predetermined condition is identified. (Step 002). For this purpose, the condition application unit applies a predetermined condition to at least some of the data. The information may include information indicating a predetermined facial expression or the like when the condition is satisfied.

[0108] Step 3 The identified facial expressions, etc. are transferred to the avatar used to fulfill the specified conditions on the viewer's device. The viewer can set the avatar's facial expression etc. in response to a predetermined action etc. This has the advantage of allowing you to watch the character's reactions as their facial expressions change.

[0109] Step 4 On each viewer's device, a predetermined facial expression of the avatar is displayed for a predetermined period of time, and then the original facial expression is restored. The facial expression of the avatar is returned to the facial expression of the broadcaster (step 004).

[0110] 4-2.Embodiment 2 In the second embodiment, the condition application unit operates in the server. In this case, the acquisition unit In this example, the animation may be executed on the viewer terminal. The following explanation will be given with reference to FIG. 8, assuming that the .

[0111] Step 1 The acquisition unit acquires information such as a predetermined action (step 001).

[0112] Step 2 Next, the condition application unit determines whether a predetermined condition is met using at least information on the predetermined action, etc. If it is determined that the predetermined condition is satisfied, the facial expression or the like corresponding to the satisfied predetermined condition is identified. (Step 002).

[0113] Step 3 The server transmits the information indicating the specified facial expression, etc. to each user who is watching the broadcast of the avatar. The data is transmitted to the viewer terminal (step 003).

[0114] Step 4 Each viewer terminal uses the facial expression information obtained from the server to display the avatar. Then, the information is set (step 004). An animation may be generated using the avatar's facial expressions, etc.

[0115] Step 5 On each viewer's device, the avatar's facial expression information is acquired from the server. The facial expression, etc. is displayed for a predetermined period of time, and then the original avatar is changed based on the facial expression, etc. of the broadcaster. The facial expression is set to the correct expression (step 005). The generated facial expressions are displayed for a predetermined period in each animation, and After that, the facial expressions obtained by tracking based on the facial expressions of the broadcaster related to the avatar are It may be used as an avatar's facial expression, etc.

[0116] 4-3.Embodiment 3 In the third embodiment, the condition application unit operates in the server. In this case, the acquisition unit In this example, the server may operate on the avatar's facial expression. The following description will be given with reference to FIG. 9, on the assumption that a video including the above will be generated.

[0117] Step 1 The acquisition unit acquires information such as a predetermined action (step 001).

[0118] Step 2 Next, the condition application unit determines whether a predetermined condition is met using at least information on the predetermined action, etc. If it is determined that the predetermined condition is satisfied, the facial expression or the like corresponding to the satisfied predetermined condition is identified. (Step 002).

[0119] Step 3 The server uses the information indicating the specified facial expression, etc. to Generate a video (step 003). Here, a video of the avatar distribution is generated. If the server and the server that executes the condition application unit are the same server, The server that performs the process may generate the video for the avatar distribution. The server that generates the video for distribution and the server that executes the condition application unit are different. In the case of a server, the server that executes the condition application unit transmits the condition to the server that generates the video. The information indicating the specified facial expression, etc. is transmitted to the server that generates the video. Information indicating the specified facial expression, etc. may be used. Similarly, when creating a video, you may provide a video together with or in place of the video. In this case, animation may be generated along with or instead of the video. Animation may be used.

[0120] Step 4 An avatar generated using a specified facial expression based on information indicating the specified facial expression, etc. The video is transmitted to each viewer terminal (step 004).

[0121] Step 5 At each viewer terminal, the video using the information indicating facial expressions, etc. obtained from the server is The avatar will be displayed for a certain period of time, and then the original avatar will be displayed based on the facial expressions of the original broadcaster. The image of the avatar created is displayed (step 005).

[0122] 4-4. Various embodiments The computer program according to the first aspect is "executed by one or more processors" By this, a predetermined action by the distributor, the facial expression of the distributor, the complexion of the distributor, Information relating to one or more of the voice of the broadcaster or a predetermined situation occurring to the broadcaster detecting information, determining whether the detected information satisfies a predetermined condition, and determining whether the predetermined condition is satisfied; If it is determined that the above condition is satisfied, the facial expression or pose of the broadcaster is generated based on the facial expression or pose of the broadcaster. The facial expression or pose of the avatar related to the distributor is changed to a predetermined facial expression corresponding to the predetermined condition. and generating information for applying the predetermined facial expression or pose to the image. "causing the one or more processors to function so as to display butter."

[0123] The computer program according to the second aspect is the computer program according to the first aspect, The predetermined action is a predetermined action that is perceived by the distributor through the five senses.

[0124] A computer program according to a third aspect of the present invention is a computer program according to the first aspect or the second aspect of the present invention. In this case, "the specified action of the distributor includes the action of looking at a specified object."

[0125] A computer program according to a fourth aspect of the present invention is a computer program according to any one of the first to third aspects of the present invention. In the above example, "the specified action of the distributor includes the action of touching a specified object." .

[0126] A computer program according to a fifth aspect of the present invention is a computer program according to any one of the first to fourth aspects of the present invention. In the method, "the specified action of the distributor includes the eating and drinking action of the specified target."

[0127] A computer program according to a sixth aspect of the present invention is a computer program according to any one of the first to fifth aspects of the present invention. In this case, "the item corresponding to the predetermined object is generated in the virtual space." It is something that ``happens.''

[0128] A computer program according to a seventh aspect of the present invention is a computer program according to any one of the first to sixth aspects of the present invention. In the above, "the specified actions of the distributor include smelling or a specified gesture of smelling." is.

[0129] A computer program according to an eighth aspect of the present invention is a computer program according to any one of the first to seventh aspects of the present invention. In the above, "the predetermined action of the distributor is a predetermined gesture for listening to / listening to sound." This includes "the

[0130] A computer program according to a ninth aspect of the present invention is a computer program according to any one of the first to eighth aspects of the present invention. "The detected information includes information detected due to input by a third party." "It is something like that.

[0131] A computer program according to a tenth aspect of the present invention is a computer program according to any one of the first to ninth aspects of the present invention. In the aspect, "the detected information is captured by one or more imaging devices that capture the image of the broadcaster." "The term 'information' includes information detected by image processing using images captured by the camera."

[0132] A computer program according to an eleventh aspect of the present invention is a computer program for implementing any one of the first to tenth aspects of the present invention. In the embodiment, "the predetermined situation occurring to the distributor includes the occurrence of a predetermined sound." "It is something like that.

[0133] A computer program according to a twelfth aspect of the present invention is a computer program that includes any one of the first to eleventh aspects described above. In the embodiment, "the predetermined situation occurring to the distributor is when a predetermined object comes into contact with the distributor." This includes "the act of

[0134] A computer program according to a thirteenth aspect of the present invention is a computer program for executing any one of the first to twelfth aspects of the present invention. In this aspect, "the facial color of the distributor includes a change in facial color."

[0135] A computer program according to a fourteenth aspect of the present invention is a computer program for executing any one of the first to thirteenth aspects of the present invention. In the above aspect, "the voice of the broadcaster is such that the quality of the voice of the broadcaster does not change by more than a predetermined range." and / or the volume of the broadcaster's voice changes by more than a predetermined amount.

[0136] A computer program according to a fifteenth aspect of the present invention is a computer program for executing any one of the first to fourteenth aspects of the present invention. In the embodiment, "within a predetermined time, a first predetermined condition is satisfied and a second predetermined condition is satisfied" If the condition is satisfied, the first predetermined condition or the second predetermined condition, When a predetermined condition is met, a corresponding predetermined expression or pose is applied. It is something that ``does.''

[0137] A computer program according to a sixteenth aspect of the present invention is a computer program for executing any one of the first to fifteenth aspects of the present invention. In this embodiment, "the information for making the predetermined facial expression or pose is transmitted from the broadcaster terminal in advance." The avatar is generated without obtaining information to operate it.

[0138] A computer program according to a seventeenth aspect is a computer program, When executed by one or more processors, a predetermined action by a distributor, the distributor The facial expression of the follower, the complexion of the broadcaster, the voice of the broadcaster, or a predetermined event occurring to the broadcaster and detecting whether the detected information satisfies a predetermined condition. If it is determined that the predetermined condition is satisfied, The facial expression or pose of the avatar related to the distributor that was generated based on the facial expression or pose, generating information for making a predetermined expression or pose corresponding to the predetermined condition, and transmitting information for determining a facial expression or pose. It is something that "makes things work."

[0139] The computer program according to the eighteenth aspect is the computer program according to the seventeenth aspect, The predetermined action of the person is a predetermined action that is perceived by the distributor with the five senses. do.

[0140] The server device according to the 19th aspect "includes one or more processors, The server executes the computer-readable instructions to the distributor's movements, the distributor's facial expression, the distributor's complexion, the distributor's voice, or and detecting information relating to one or more of the predetermined situations that occur as a result of the detection of the information. If it is determined that the predetermined condition is satisfied, The facial expression of the avatar related to the distributor that was generated based on the facial expression or pose related to the distributor. or generating information for making the pose into a predetermined expression or pose corresponding to the predetermined condition. and transmits information for achieving the predetermined facial expression or pose.

[0141] The server device according to the 20th aspect of the present invention is a server device according to the 19th aspect of the present invention, The action is a predetermined action that is perceived by the distributor through the five senses.

[0142] The method according to the 21st aspect is a method for performing computer-readable instructions on one or more A method executed by a plurality of processors, comprising: a predetermined action by a distributor; the facial expression of the person, the complexion of the distributor, the voice of the distributor, or a predetermined event occurring to the distributor and detecting information relating to one or more of the above conditions, and determining whether the detected information satisfies a predetermined condition. If it is determined that the predetermined condition is satisfied, or a pose of the avatar of the distributor that has been generated based on the previous generating information for making a predetermined facial expression or pose corresponding to the predetermined conditions, and displaying the avatar with the image or pose applied.

[0143] The method according to the 22nd aspect is the same as the method according to the 21st aspect, in which "the predetermined action of the distributor is It is a predetermined action that is perceived by the distributor through the five senses.

[0144] The method according to the 23rd aspect is a method for executing computer-readable instructions. A method executed by a plurality of processors, comprising: a predetermined action by a distributor; the facial expression of the person, the complexion of the distributor, the voice of the distributor, or a predetermined event occurring to the distributor and detecting information relating to one or more of the above conditions, and determining whether the detected information satisfies a predetermined condition. If it is determined that the predetermined condition is satisfied, or a pose of the avatar of the distributor that has been generated based on the previous generating information for making a predetermined facial expression or pose corresponding to the predetermined conditions, "The purpose of the present invention is to transmit information for the purpose of impersonating or posing."

[0145] The method according to the 24th aspect is the same as the method according to the 23rd aspect, except that "the predetermined action of the distributor is It is a predetermined action that is perceived by the distributor through the five senses.

[0146] The viewer terminal device according to the 25th aspect "has one or more processors, The processor executes the computer readable instructions to A predetermined action, the facial expression of the broadcaster, the complexion of the broadcaster, the voice of the broadcaster, or and detecting information relating to one or more of the predetermined situations occurring with respect to the detected information. determines whether or not a predetermined condition is satisfied, and if it is determined that the predetermined condition is satisfied, The avatar of the distributor that was generated based on the facial expression or pose of the distributor Information for changing a facial expression or pose to a predetermined facial expression or pose corresponding to the predetermined condition and displaying the avatar with the predetermined facial expression or pose applied. .

[0147] The distributor terminal device according to the 26th aspect "includes one or more processors, The processor executes the computer readable instructions to A predetermined action, the facial expression of the broadcaster, the complexion of the broadcaster, the voice of the broadcaster, or and detecting information relating to one or more of the predetermined situations occurring with respect to the detected information. determines whether or not a predetermined condition is satisfied, and if it is determined that the predetermined condition is satisfied, The avatar of the distributor that was generated based on the facial expression or pose of the distributor Information for changing a facial expression or pose to a predetermined facial expression or pose corresponding to the predetermined condition and "generate and transmit information for achieving the predetermined expression or pose."

[0148] Furthermore, the systems and information processes described in this application may be implemented in software, hardware, or The present invention may also be implemented by a combination of these systems and information processing. Some or all of the processes and procedures are implemented as computer programs and used on various computers. These computer programs may be stored in a storage medium. These programs may also be stored in a non-transitory or temporary storage medium.

[0149] The items described in the present application document are not limited to those described in the present application document. It can be applied to various examples within the scope of various technical ideas having various technical advantages and configurations. It goes without saying that. [Explanation of symbols]

[0150] 1 System 10. Communication Network 20(20A~20C) Server equipment 30(30A~30C) Terminal equipment 21(31) Arithmetic device 22(32) Main memory 23(33) Input / Output Interface 24(34) Input Device 25(35) Auxiliary storage device 26(36) Output Device 41 Acquisition Department 42 Condition application part

Claims

1. Computer, a first means for acquiring a motion in real space of an actor corresponding to an avatar moving in virtual space; 、 a second determination as to whether or not the movement of the actor in real space satisfies one or more conditions; Two methods, (1) If none of the one or more conditions is satisfied, the avatar in the virtual space the movement of the performer in real space acquired by the first means, (2) If one or more of the above conditions are met, The avatar's action is set as an action associated with the certain condition, and after a predetermined period of time has elapsed, a third means for making the motion of the performer correspond to the motion of the performer in the real space; Program.

2. The apparatus according to claim 1 , wherein the avatar's action is a facial expression or a pose of the avatar. Program.

3. 10. The method according to claim 1, wherein an action is associated with each of the one or more conditions in advance. The program described in

4. the one or more conditions include at least a first condition and a second condition; The action associated with the first condition and the action associated with the second condition are different.

4. The program according to claim 1, wherein

5. The movement of the actor in the real space is detected by a motion sensor, and the first means The program according to any one of claims 1 to 4, wherein the program is acquired.

6. The behavior associated with the certain condition is the current state of the actor acquired by the first means.

6. The program according to claim 1, wherein the action is different from the action corresponding to the action in the real space. Hmm.

7. One of the one or more conditions is that the broadcaster smells or makes a predetermined gesture that causes a smell.

7. The program according to claim 1, wherein:

8. One of the one or more conditions is to perform a predetermined action on a predetermined object existing in real space. This is what we have done. The action associated with this condition is to move an item corresponding to the predetermined object in the virtual space.

7. The program according to claim 1, wherein the program is for generating a program.

9. a first means for acquiring a motion in real space of an actor corresponding to an avatar moving in virtual space; 、 a second determination as to whether or not the movement of the actor in real space satisfies one or more conditions; Two methods, (1) If none of the one or more conditions is satisfied, the avatar in the virtual space the movement of the performer in real space acquired by the first means, (2) If one or more of the above conditions are met, The avatar's action is set as an action associated with the certain condition, and after a predetermined period of time has elapsed, and a third means for making the motion of the performer correspond to the motion in real space. Place.

10. The first means acquires the movements of an actor in real space corresponding to an avatar moving in virtual space. and A second means determines whether the movement of the actor in the real space satisfies one or more conditions. determining whether A third means (1) when none of the one or more conditions is satisfied, The movement of the avatar is compared with the movement of the actor in real space acquired by the first means. (2) If one of the above conditions is satisfied, the The avatar's movement in the imaginary space is set as a movement associated with the certain condition, and the avatar moves at a predetermined time. and after a period of time has elapsed, making the motion correspond to the movement of the actor in real space. Information processing methods.

Images