Information processing apparatus

Abstract

To provide a system for allowing a user to charge a desired distributer by using health care information of the user without damaging sense of immersion to a VR distribution content.SOLUTION: An information processing apparatus according to the present invention can carry out charging processing without damaging sense of immersion of a viewer in distribution on a VR (Virtual Realty). Specifically, the apparatus detects motion of charging of a user during distribution viewing and charges a charging amount responding to satisfaction calculated from the user health care information for a distributor in response to the user movement.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, and more particularly to an information processing apparatus that enables a billing process to be performed without impairing the immersion feeling of a viewer in VR (virtual space) distribution.

Background Art

[0002] In recent years, it has become possible to stream and distribute content photographed using a camera in VR (virtual space) so that a user can experience it without actually being on the spot.

[0003] Also, in streaming distribution, a viewer can post a tip to a distributor by making a charge to the distribution site, and a two-way viewing experience in which the distributor responds to it is possible.

[0004] Furthermore, Patent Document 1 proposes a system in which a character of a distributor acts and throws an item for charging by an action in VR streaming distribution.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] However, in the method described in Patent Document 1 above, when charging by throwing an item, a user interface operation for selecting an item and setting a charge amount is required, which impairs the immersion feeling during live distribution viewing.

[0007] Therefore, an object of the present invention is to provide a system that allows a user to charge a desired distributor without impairing the immersion of VR delivery content by using the user's healthcare information.

Means for Solving the Problems

[0008] To achieve the above object, the present invention provides an apparatus capable of viewing online delivery, comprising means for acquiring the user's healthcare information, means for determining the user's satisfaction based on the user's healthcare information, means for determining the amount to be charged according to the user's satisfaction, means for detecting the user's movement, means for performing a charging process, and means for registering the user's movement at the time of charging. The movement of the user to be charged is detected, and the charging process of the amount to be charged is performed according to the user's movement.

Effects of the Invention

[0009] According to the information processing apparatus of the present invention, a user who is a viewer of the delivery can charge a desired distributor in the delivery content without impairing the immersion.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Modes for Carrying Out the Invention

[0011] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

Example

[0012] <Configuration of Each Device> FIG. 1 is a block diagram showing the configuration of the entire system in this embodiment. The system in this embodiment is composed of a VR device A100, a controller B100, a distribution server C100, and the Internet D100. In this embodiment, only one VR device is described, but the VR devices communicating with the distribution server C100 may have a configuration of two or more.

[0013] First, the VR device A100 will be described.

[0014] The VR device A100 may be equipped with a function to realize AR (augmented reality).

[0015] The control unit A101 is, for example, one or more processors such as a CPU (Central Processing Unit), an MPU (Micro Processor Unit), or a GPU (Graphics Processing Unit), and controls the entire VR device A100.

[0016] The camera unit A102 is a camera that captures the real space and starts capturing under the control of the control unit A101.

[0017] The gaze detection unit A103 is a processing unit that detects the gaze and blinks of the user during distribution viewing under the control of the control unit A101. The gaze detection unit A103 detects, for example, by a method called the corneal reflection method. The corneal reflection method is a method of detecting the direction and position of the gaze from the positional relationship between the reflected light (especially the reflected light reflected by the cornea) of the infrared light reflected by the user's eyeball and the pupil of the eyeball. Note that various methods for specifying the direction and position of the gaze can be used, not limited to the corneal reflection method. Also, the user's gaze may be detected for the operation of the VR device A100.

[0018] The operation unit A104 is a processing unit that receives user operations and notifies the input information to the control unit A101. The operation unit A104 detects user operations, for example, by buttons provided on the VR device A100. It may also be realized by acquiring input information from the operation unit B104 of the controller B100 via the communication unit A108 described later.

[0019] The display unit A105 is a processing unit for notifying various messages such as the operation state of the device to the user and for displaying VR content. The display unit A105 is composed of, for example, a liquid crystal panel, LEDs, etc.

[0020] The ROM (Read Only Memory) A106 is a rewritable non-volatile memory and is a processing unit that holds programs such as the OS and programs supplied from external devices. The ROM A106 includes a program that communicates with the controller B100 using the communication unit A108 according to predetermined conditions to acquire the user's healthcare information. It also includes a program for controlling the billing process display based on the healthcare information acquired from the controller B100.

[0021] The RAM (Random Access Memory) A107 is a volatile memory and is a processing unit that temporarily stores programs, captured image data, communication data, etc.

[0022] The communication unit A108 is a communication unit for realizing wireless communication. The communication unit A108 is composed of an antenna for wireless communication and a communication controller for processing wireless signals. It realizes short-range wireless communication for directly communicating with the controller B100, such as Bluetooth, and wireless communication according to system standards such as 5G for connecting to the Internet D100. The control unit A101 controls the communication unit A108 to realize communication with the distribution server C100 via the Internet D100. Also, the control unit A101 controls the communication unit A108 to realize direct communication with the controller B100. Communication with the Internet D100 may be wireless communication or wired communication.

[0023] The motion detection unit A109 is a module that detects parts such as hands and arms from the input image and detects how the user is moving. The control unit A101 transfers the input image from the camera unit A102 to the motion detection unit A109 to detect the user's movement and the action of inserting coins.

[0024] The internal bus A110 is a processing unit that communicably connects each block of A101~A109.

[0025] The above is the description of the VR device A100.

[0026] Next, the controller B100 will be described.

[0027] The control unit B101 is, for example, an MPU and is a processing unit that controls the entire controller B100.

[0028] The communication unit B102 is a communication unit for realizing proximity wireless communication for directly communicating with the VR device A100 such as Bluetooth. The communication unit B102 communicates only with the communication unit A108 of the VR device A100, and the controller B100 may be controlled by receiving the control information of the VR device A100.

[0029] The healthcare information acquisition unit B103 is, for example, an optical heart rate sensor and is a processing unit that measures healthcare information such as the heart rate of the user wearing it.

[0030] The operation unit B104 is a processing unit that receives the user's operation and notifies the input information to the control unit B101. The operation unit B104 is, for example, a cross key or a button.

[0031] The internal bus B105 is a processing unit that communicably connects each block of B101~B104.

[0032] The above is the description of the controller B100.

[0033] Next, the distribution server C100 will be described.

[0034] Each server device may be realized by a single computer device, or each function may be realized by being distributed to a plurality of computer devices as necessary. When it is configured by a plurality of computer devices, they are connected by a LAN (Local Area Network) or the like so as to be able to communicate with each other.

[0035] The control unit C101 is a processing unit that controls the entire distribution server C100, which is, for example, a CPU (Central Processing Unit).

[0036] The ROM C102 is a rewritable non-volatile memory and is a processing unit that holds programs such as an OS (Operation System) and programs supplied from external devices.

[0037] The RAM C103 is a volatile memory and is a processing unit that temporarily stores programs and data.

[0038] The communication unit C104 is a communication unit for realizing wired communication. The communication unit C104 is composed of hardware for wired communication and a communication controller for processing wired signals, and realizes wired communication according to the IEEE802.3 series standard.

[0039] The storage unit C105 is composed of an HDD (hard disk drive) fixedly installed in the distribution server C100, an SSD (solid state drive) composed of a flash memory, a hybrid drive using both a hard disk and a flash memory, a memory card, or the like. The storage unit C105 holds distributor information such as distributor account information and viewer information such as viewer account information. The viewer's account information includes payment information and information regarding the payment method.

[0040] The internal bus C106 is a processing unit that communicably connects each block of C101 to C105.

[0041] The above is the description of the distribution server C100.

[0042] FIG. 2 is a display example of the coin - throwing items to be displayed during the coin - throwing preparation operation described later. It is the display when the user wears the VR device and looks at their hands.

[0043] In FIG. 2(a), during the coin - throwing preparation operation, an item 202 to be charged is displayed on the user's right hand 201. The charging amount of the item 202 is displayed at 203 and is ￥1000.

[0044] FIG. 2(b) is an example of the charging items and amounts displayed during the coin - throwing preparation operation. An item 205 and a charging amount 204 of ￥500 - ￥1500 are displayed. The amount range of the charging amount is changed according to the magnitude of the user's charging movement when the user throws coins, which will be described later.

[0045] <Operation of the VR device A100> Regarding the flow related to the settings performed by the user before watching the distribution, it will be described with reference to FIGS. 3 and 4. In the following description, the viewer corresponds to the user.

[0046] In S301, the control unit A101 communicates with the distribution server C100 via the communication unit A108 and acquires the distributor information. The distributor information includes the distributor's account information and the streaming information necessary for watching the distribution content.

[0047] In S302, the control unit A101 acquires the viewer information. The viewer information is connected to the Internet D100 via the communication unit A108 and acquired from the distribution server C100. Also, the viewer information held in the VR device A100 may be acquired. The viewer information is, for example, the viewer's account information such as payment information.

[0048] In S303, the control unit A101 communicates with the controller B100 via the communication unit A108 and transmits a control signal for acquiring the average value of the viewer's heart rate. The controller B100 receives the control information from the VR device A100 through the communication unit B102. The control B101 acquires the viewer's healthcare information from the healthcare information acquisition unit B103 based on the control information from the VR device A100. The control unit B101 measures the viewer's heart rate a plurality of times from the healthcare information acquisition unit B103 within a predetermined time and calculates the average value. The average value is, for example, the average value with respect to the heart rates acquired a predetermined number of times within a predetermined time. The control unit B101 transmits the average value information of the viewer's heart rate to the VR device A100 via the communication unit B102. The control unit A101 receives the average value information of the viewer's heart rate via the communication unit A108 and stores it in the RAM A107. Further, the average value of the heart rate may be calculated by the control unit A101 by receiving the viewer's heart rate information from the controller B100 at any time in the communication unit A108 within a predetermined time.

[0049] In S304, the control unit A101 stores, in the RAM A107, as the threshold value of the viewer's satisfaction, a value obtained by multiplying the average value of the heart rate saved in S303 by a predetermined coefficient.

[0050] In this embodiment, it is a flow regarding the setting performed immediately before the viewer views the distribution, but the processes of S303 and S304 may be such that they can be set at an arbitrary timing in advance by an initial setting or the operation unit A104.

[0051] Next, the operation registration processing flow when the user inserts coins will be described.

[0052] In S401, the control unit A101 transfers the input image from the camera unit A102 to the motion detection unit A109. In the motion detection unit A109, registration of the coin-throwing preparation operation is performed based on the detected motion of the user's arm. The coin-throwing preparation operation is the motion of the arm that takes out the wallet from the user's bosom. Or it may be the motion of the arm that pulls the dominant arm backward. The control unit A101 stores and registers the coin-throwing preparation operation in the ROM A106.

[0053] In S402, as a continuation of the coin-throwing preparation operation registered in S401, the operation from the user's coin-throwing preparation operation to the completion of coin-throwing is registered. After the motion of taking out the wallet from the user's bosom in S401, the motion of the arm that throws the disc is registered as the coin-throwing operation. Or after the motion of pulling the dominant arm backward, the motion of the arm that throws an object is registered as the coin-throwing operation. The control unit A101 stores and registers the coin-throwing operation in the ROM A106.

[0054] The above is the explanation of the flow regarding the settings performed by the user, who is the viewer, before watching the distribution.

[0055] The operation of the VR device A100 while the user is watching the distribution will be described with reference to FIG. 5.

[0056] In S501, the control unit A101 communicates with the controller B100 via the communication unit A108 and transmits a control signal for acquiring the health care information of the viewer. The controller B100 receives the control information from the VR device A100 through the communication unit B102. The control B101 acquires the health care information of the viewer from the health care information acquisition unit B103 based on the control information from the control unit A101 and transmits it to the VR device A100 via B102. The health care information is, for example, the average value of the heart rate of the viewer (user) over an arbitrary period.

[0057] In S502, the control unit A101 calculates the satisfaction level of the viewer based on the health care information. The satisfaction level is, for example, a value obtained by multiplying the heart rate by an arbitrary coefficient or a value obtained by combining and weighting other information related to the viewer. The other information related to the viewer is, for example, the number of blinks. The weighting method uses, for example, a weighted moving average that decreases the weight of past health care information in order to emphasize the most recent health care information.

[0058] In S503, the control unit A101 transfers the input video from the camera unit A102 to the motion detection unit A109. The control unit A101 compares the motion of the user detected by the motion detection unit A109 with the motion of the coin - throwing preparation operation pre - registered in the ROM A105 and determines whether the coin - throwing preparation operation has started. If the control unit A101 determines that the coin - throwing preparation operation has started, it proceeds to the process of S504; otherwise, it returns to the process of S501.

[0059] In S504, the control unit A101 sets the charging amount corresponding to the satisfaction level of the viewer calculated in S502. The setting of the charging amount is determined, for example, using the maximum value of the satisfaction level of the viewer. When the satisfaction level is the maximum value, it is set to ￥4000, and when the satisfaction level is the threshold value, it is set to ￥500.

[0060] In S505, the control unit A101 detects the motion of the user from the image input from the camera unit A102 using the motion detection unit A109 and determines whether a motion different from the coin - throwing motion registered in the ROM A106 has been made. It determines whether the coin - throwing preparation motion of the user detected in S503 has been canceled. After performing the coin - throwing preparation motion, the control unit A101 determines whether a motion different from the coin - throwing motion after the coin - throwing preparation motion registered in the ROM A106 has been made. For example, if the coin - throwing preparation motion is performed and then the coin - throwing preparation motion is performed again, it is determined that the operation has been canceled. If it is determined that the coin - throwing preparation motion has been canceled, it returns to the process of S501; otherwise, it proceeds to the process of S506.

[0061] In S506, the control unit A101 detects the movement of the user's hand and arm from the image input from the camera unit A102 by the motion detection unit A109, compares it with the movement of the hand and arm in the coin - throwing operation registered in the ROM A106, and determines whether the user's coin - throwing operation is completed. If the control unit A101 determines that the coin - throwing operation is completed, it proceeds to the process of S507; otherwise, it returns to the process of S501.

[0062] In S507, the control unit A101 detects the viewer's gaze information via the gaze detection unit A103. The gaze information includes the viewer's gaze position and the number of blinks of the viewer. The control unit A101 detects the viewer's gaze position via the gaze detection unit A103. Further, the control unit A101 detects the direction of the user's hand and arm in the coin - throwing operation detected in S506 by the motion detection unit A109. The control unit A101 detects the distributor based on the direction of the user's hand and arm in the coin - throwing operation and the viewer's gaze position, and sets the distributor to whom the user has thrown the coin as the charging target. If there is one distributor, the distributor is set as the charging target. The account information of the charging target is stored in the RAM A107.

[0063] In S508, it is determined the magnitude of the operation when the user throws a coin in S506. The control unit A101 detects the position of the fingertip of the user's hand detected by the motion detection unit A109 when the user throws a coin in S506, approximates the trajectory of the detected fingertip to an arc in the control unit A101, and determines the radius at that time as the magnitude of the operation. By comparing with the magnitude of the movement of the user's hand and arm in the coin - throwing operation registered in the ROM A106 in advance, it is determined whether the operation is large, small, or equivalent. As shown in Fig. 2(b), when there is a range in the coin - throwing amount, the amount is set according to the magnitude of the user's operation. When the operation is large, the charging amount is set to ￥1500, and when the operation is small, the charging amount is set to ￥500. As shown in Fig. 2(a), when the coin - throwing amount is fixed, the amount is not changed. Whether to fix the coin - throwing amount or give it a range can be set in advance by the operation unit A104.

[0064] In S509, the control unit A101 executes a charging process. In the charging process, the control unit A101 connects to the Internet D100 via the communication unit A108 and transmits information for executing the charging process to the distribution server C100. The information for executing the charging process is, for example, the account information of the viewer who is to be charged and the account information of the charging destination. The control unit C101 receives the information for executing the charging process via the communication unit C104 and executes the charging process. When the execution of the charging process is completed, the control unit C101 transmits, via the communication unit C104, a message indicating that the execution of the charging process has been completed to the VR device A100. The control unit A101 receives, via the communication unit A108, a message indicating that the charging process on the distribution server C100 has been completed, and completes the charging process in the VR device A100.

[0065] The history of the charges made by the user is stored by the control unit A101 in the ROM A106. The history includes the account information of the distributor, the charging date and time, and the amount charged.

[0066] The above is a detailed description of the operation of the VR device A100 while the user is viewing the distribution.

[0067] Next, the operation of the VR device A100 when there is an upper limit on the amount charged will be described with reference to FIGS. 6 and 7.

[0068] In S601, the control unit A101 sets, through the operation unit A104, an upper limit on the total amount that can be charged in a day. The set amount is set in the ROM A106 by the operation unit A101.

[0069] In S602, the amount to be charged corresponding to the satisfaction level of the viewer in S504 is confirmed.

[0070] In S603, the control unit A101 refers to the daily charging history stored in the ROM A106.

[0071] In S604, the control unit A101 determines whether the total sum obtained by adding the total amount of the charging history for the day and the charging amount calculated in S602 exceeds the upper limit of the total amount chargeable for the day. If the control unit A101 determines that it exceeds the upper limit, it proceeds to the process of S605, and if it determines that it does not exceed, it proceeds to S606.

[0072] In S605, the control unit A101 displays on the display unit A105 that the total charge for the day has been exceeded. FIG. 7(a) is an example of item display when the charge for the day is exceeded. An item 702 to be charged to the right hand of the user 701 is displayed. Since the charge amount is ¥0, a transparent charging item 702 and a charge amount 703 ¥0 are displayed. The user is notified as shown in 704 that the daily charging limit has been exceeded.

[0073] In S606, the control unit A101 compares the charging amount in the charging history with the charging amount calculated in S602 and determines whether the calculated charging amount is larger. When the calculated charging amount is ¥2000 or more with respect to the average amount of the charging amount in the charging history, it is determined that the charging amount is larger. If the calculated charging amount is larger, it proceeds to the process of S607, and if it is not larger, it proceeds to S608.

[0074] In S607, the control unit A101 displays on the display unit A105 that it is a high - value charge. FIG. 7(b) is an example of item display when it is a high - value charge. An item 705 to be charged to the right hand of the user 701 is displayed. Since the charge amount is ¥4000, a charge amount 706 ¥4000 is displayed. The user is notified as shown in 707 that it is a high - value charge.

[0075] In S608, it is the charging item display process in S504.

[0076] As described above, the VR device A100 of the present embodiment determines the charging amount according to the satisfaction of the viewer (user) calculated from the health care information of the viewer (user) during distribution, and performs charging on the charging target by the operation of the viewer (user). Thereby, the viewer (user) can charge the desired distributor in the distribution content without impairing the sense of immersion.

Explanation of symbols

[0077] A100 VR device A101 Control unit A102 Camera unit A103 Gaze detection unit A104 Operation unit A105 Display unit A106 ROM A107 RAM A108 Communication unit A109 Motion detection unit A110 Internal bus B100 Controller B101 Control unit B102 Communication unit B103 Health care information acquisition unit B104 Operation unit B105 Internal bus C100 Distribution server C101 Control unit C102 ROM C103 RAM C104 Communication unit C105 Storage unit C106 Internal bus D100 Internet 201 Distributor's hand 202 Coin item 203 Charging amount 204 Charging amount 205 Coin item 701 Distributor's hand 702 Coin item 703 Charging amount 704 User notification 705 Coin item 706 Billing amount 707 User notification

Claims

1. An apparatus capable of viewing online distribution, comprising: means for acquiring user's healthcare information; means for determining user's satisfaction based on the user's healthcare information; means for determining a charging amount according to the user's satisfaction; means for detecting the user's movement; means for performing a charging process; means for registering the user's movement at the time of charging, and characterized in that: An information processing apparatus that detects the movement of a user to be charged and performs a charging process for the charging amount according to the user's movement.

2. The information processing apparatus according to claim 1, wherein the healthcare information includes at least a pulse / heart rate.

3. The movement of the user at the time of charging is composed of two steps: the movement of the coin-throwing preparation operation for preparing to throw coins and the movement of throwing coins. When a movement different from the movement of the user to be charged registered in advance after detecting the coin-throwing preparation operation is detected, the charging process is not performed. The information processing apparatus according to claim 1, characterized in that.

4. The information processing apparatus according to claim 1, characterized in that when detecting the movement of the user to be charged, the charging amount determined by the means for determining the charging amount is displayed.

5. The information processing apparatus according to claim 4, further comprising means for detecting the magnitude of the user's movement, and changing the charging amount according to the magnitude of the user's movement with respect to the movement registered by the means for registering the user's movement at the time of charging.

6. The information processing apparatus according to claim 1, further comprising means for setting an upper limit of the charging amount, and characterized in that the charging amount is set in advance.

7. The information processing apparatus according to claim 4, characterized in that when the charging history or the calculated amount is large, the user is notified at the time of the coin-throwing preparation operation.

8. The information processing apparatus according to claim 1, further comprising means for detecting the direction in which the user throws coins, and determining the charging target according to the direction in which the user throws coins when there are a plurality of target distributors.

Images