In-vehicle performance device, in-vehicle performance system, in-vehicle performance method and storage medium

Abstract

An in-vehicle performance device performs a game performed by passengers in a vehicle, wherein the device comprises: a motion detection unit for detecting the motion of the passenger that is unrelated to the driving of the vehicle; a display unit for displaying an image that can be visually confirmed by the passenger; and a display control unit for displaying a response image corresponding to the motion of the passenger on the display unit based on the motion of the passenger detected by the motion detection unit, and outputting the result of the game based on prescribed rules.

Description

Technical Field

[0001] The invention relates to an in-vehicle performance device, an in-vehicle performance system, an in-vehicle performance method, a storage medium and an indication and measuring device.

[0002] This application claims priority based on Japanese Patent Application No. 2017-119025 filed in Japan on June 16, 2017, the contents of which are incorporated herein by reference. Background Art

[0003] Conventionally, there is known a technique for calculating a score of a driving skill of a driver of a vehicle (for example, Patent Document 1).

[0004] Prior Art Literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2015-90676 Summary of the invention

[0007] Problems to be solved by the invention

[0008] The prior art does not provide entertainment to the passengers of the vehicle. The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to provide entertainment to the passengers of the vehicle during the movement of the vehicle.

[0009] Means for solving problems

[0010] The in-vehicle performance device, in-vehicle performance system, in-vehicle performance method and storage medium of the present invention adopt the following structure.

[0011] (1) One embodiment of the present invention is an in-vehicle performance device that performs a game performed by passengers in a performance vehicle, wherein the in-vehicle performance device comprises: a motion detection unit 30 that detects the motion of the passenger that is not related to the driving of the vehicle; a display unit 60 that displays an image that the passenger can visually confirm; and a display control unit 18 that displays a response image corresponding to the passenger's motion on the display unit based on the motion of the passenger detected by the motion detection unit, and outputs the result of the game based on prescribed rules.

[0012] (2) In the in-vehicle performance device according to the aspect (1), the display control unit may display the response image on the display unit by overlapping it with an object in the surrounding environment of the vehicle.

[0013] (3) According to the in-vehicle performance device of the scheme (2), the passenger's action may be to indicate the action of an object in the surrounding environment, the action detection unit detects the direction indicated by the passenger, and the display control unit displays the response image on the display unit in a manner overlapping with a specific object in the indicated direction based on the direction detected by the action detection unit.

[0014] (4) The in-vehicle performance device according to the scheme of (2) or (3) may be that the in-vehicle performance device also includes a score calculation unit 17, which calculates the number of times the action detection unit detects an action indicating a specific object as a valid score, and the display control unit displays an image representing the score calculated by the score calculation unit on the display unit at a specified time.

[0015] (5) According to the in-vehicle performance device of the scheme (4), the specific object may be the specific road sign leading to the destination, and the in-vehicle performance device further comprises: a destination information acquisition unit 14, which acquires destination information indicating the destination; a camera unit 40, which photographs the surrounding environment and generates a surrounding environment image; and an extraction unit 16, which extracts the specific road sign leading to the destination indicated by the destination information from the surrounding environment image, and the score calculation unit calculates the number of times the action detection unit detects the passenger's action of indicating the position corresponding to the road sign extracted by the extraction unit as a valid score.

[0016] (6) According to the in-vehicle performance device of the scheme of (5), the specific object may be another vehicle traveling near the vehicle, the extraction unit extracts the other vehicle in the surrounding environment image, and the score calculation unit calculates the number of times the action detection unit detects the passenger's action of indicating the position corresponding to the other vehicle extracted by the extraction unit as a valid score.

[0017] (7) According to the in-vehicle performance device of the scheme of (5) or (6), the specific object may be a sign of a store existing around the vehicle, the extraction unit extracts the sign in the surrounding environment image, and the score calculation unit calculates the number of times the action detection unit detects the passenger's action of indicating the position corresponding to the sign extracted by the extraction unit as a valid score.

[0018] (8) In the in-vehicle performance device according to any one of (4) to (7), the display control unit may cause the display unit to display an image representing the total of the scores calculated by the score calculation unit.

[0019] (9) is an in-vehicle performance system, comprising a plurality of in-vehicle performance devices of any one of (4) to (8), wherein the in-vehicle performance device comprises: a sending unit 80, which sends information indicating the score calculated by the score calculation unit to other in-vehicle performance devices; and a receiving unit 80, which receives score information indicating the score calculated by the other in-vehicle performance devices, and the display control unit causes an image indicating the score calculated by the score calculation unit and an image indicating the score of the score information received by the receiving unit to be displayed on the display unit.

[0020] (10) According to the in-vehicle performance system of the scheme (9), the display control unit may compare the image of the score representing the score information received from the in-vehicle performance device mounted on a vehicle belonging to the same team as the device and the image of the score representing the score information received from the in-vehicle performance device mounted on a vehicle belonging to another team, and display them on the display unit.

[0021] (11) In the in-vehicle performance system according to the scheme of (9) or (10), the display control unit may display an image urging movement based on information indicating the movement of a vehicle equipped with the device and information indicating the movement of a vehicle equipped with other in-vehicle performance devices.

[0022] (12) An in-vehicle performance method, which causes a computer equipped with a display unit and performing a game played by passengers in a performance vehicle to perform the following processing: detecting the passenger's movements that are unrelated to the driving of the vehicle; displaying an image that the passenger can visually confirm; and based on the detected passenger's movements, displaying a response image corresponding to the passenger's movements, and outputting the result of the game based on specified rules.

[0023] (13) A storage medium storing a program that causes a computer having a display unit and performing a game played by passengers on a vehicle to perform the following processing: detecting the passenger's movements that are not related to the driving of the vehicle; displaying an image that the passenger can visually confirm; and based on the detected passenger's movements, displaying a response image corresponding to the passenger's movements, and outputting the result of the game based on specified rules.

[0024] (14) One aspect of the present invention is a pointing measurement device, the pointing measurement device comprising: a motion detection unit that detects a motion of a passenger of a vehicle; a sight line detection unit that detects a viewpoint position of the passenger; a coordinate acquisition unit that acquires a three-dimensional point group in a real space indicating a direction of the pointing motion of the passenger based on the pointing motion of the passenger detected by the motion detection unit and the viewpoint position of the passenger detected by the sight line detection unit; an object information acquisition unit that acquires information indicating an object corresponding to the three-dimensional point group acquired by the coordinate acquisition unit from an interface that supplies information indicating an object existing in the real space indicated by the three-dimensional point group; a service provider identification unit that identifies a service provider corresponding to the object indicated by the information acquired by the object information acquisition unit based on service provider information indicating the service provider corresponding to the object; and a history information generation unit that generates history information indicating that the service provider identified by the service provider identification unit is in a corresponding relationship with the object indicated by the information acquired by the object information acquisition unit.

[0025] (15) In the indication measuring device according to the aspect (14), in the history information, a time when the indication action was performed and information indicating the passenger are further associated with each other.

[0026] (16) An indication measuring device according to the scheme of (14) or (15), wherein the indication measuring device further comprises: a camera unit that captures the surrounding environment and generates a surrounding environment image; and an attribute determination unit that determines the attributes of the surrounding environment based on the surrounding environment image captured by the camera unit, wherein the coordinate acquisition unit acquires a three-dimensional point group in the actual space of the indication direction of the passenger's indication action based on the attributes of the surrounding environment determined by the attribute determination unit.

[0027] (17) The instruction measuring device according to any one of (14) to (16), further comprising a notification unit for notifying the service provider indicated by the history information that the instruction action has been performed.

[0028] (18) According to the indication measuring device of (17), the notification unit notifies the passenger of the historical information.

[0029] (19) An indication measuring device according to any one of the schemes (14) to (18), wherein the indication measuring device also includes a camera unit that captures the surrounding environment and generates an image of the surrounding environment, and the coordinate acquisition unit obtains information representing the three-dimensional point group based on the surrounding environment image captured by the camera unit from an interface that provides a three-dimensional point group based on the surrounding environment image.

[0030] Effects of the Invention

[0031] According to the above-mentioned aspects (1) to (19), it is possible to provide entertainment to the passengers of the vehicle during the movement of the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a diagram showing an overview of the in-vehicle performance device according to the first embodiment.

[0033] Figure 2 1 is a functional configuration diagram showing an example of the configuration of the in-vehicle performance device according to the first embodiment.

[0034] Figure 3 FIG. 1 is a diagram showing an example of a response image indicating detection of a shooting action.

[0035] Figure 4 FIG. 1 is a diagram showing an example of a response image indicating that a shot has been taken.

[0036] Figure 5 : is a figure which shows an example of the response image of the score which shows the score information.

[0037] Figure 6 This is a flowchart showing an example of the operation of the in-vehicle performance device according to the first embodiment.

[0038] Figure 7 1 is a functional configuration diagram showing an example of the configuration of the in-vehicle performance device according to the second embodiment.

[0039] Figure 8 This is a diagram showing an example of a response image displayed by another in-vehicle performance device according to the second embodiment.

[0040] Fig. 9 This is a diagram showing an example of the result of a shooting game under the territory area rule based on the travel route.

[0041] Fig.10 This is a diagram showing an example of winning and losing in a shooting game under the territory area rule.

[0042] Fig.11 1 is a functional configuration diagram showing an example of the configuration of the in-vehicle performance device according to the third embodiment.

[0043] Fig.12 This is a diagram showing an example of the content of history information.

[0044] Fig.13 This is a diagram showing an example of the content of the service provider information. DETAILED DESCRIPTION

[0045] [First embodiment]

[0046] Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. Figure 1 1 is a diagram showing an overview of an in-vehicle performance device 1 according to a first embodiment. The in-vehicle performance device 1 is a device that provides entertainment to passengers (hereinafter referred to as passengers PS) of a vehicle (hereinafter referred to as vehicle V) that moves through the vehicle V. The in-vehicle performance device 1, for example, performs a shooting game in which objects existing in the surrounding environment of the vehicle V are used as shooting targets while the vehicle V is moving. In one example of the present embodiment, the vehicle V travels by automatic driving. The in-vehicle performance device 1 is arranged in the vehicle V. In addition, the vehicle V is provided with a sound detection unit 20, a motion detection unit 30, a camera unit 40, an input unit 50, and a display 60.

[0047] The sound detection unit 20 detects the sound made by the passenger PS. The sound detection unit 20 is, for example, a microphone. The motion detection unit 30 detects the motion of the passenger PS. The motion detection unit 30 is, for example, a motion sensor. The camera unit 40 captures the surrounding environment of the vehicle V. The input unit 50 includes an input device and receives input operations of the passenger PS. The input device includes a device for inputting text information such as a keyboard, a pointing device such as a mouse or a touch panel, a button, a dial, a joystick, a contact sensor, a touch pad, etc. The display 60 displays various information based on the control of the in-vehicle performance device 1. In one example of the present embodiment, the display 60 is a transmissive display and is configured to contact the front window of the vehicle V. The passenger PS can visually confirm the surrounding environment of the vehicle V and the image displayed on the display 60 through the front window. It should be noted that the vehicle V may not have a front window. In this case, the display 60 displays the surrounding environment image of the vehicle V captured by the camera unit 40 and the response image in an overlapping manner. It should be noted that the display 60 may also be a HUD (Head-Up Display).

[0048] The in-vehicle performance device 1 displays a response image corresponding to the action of the passenger PS on the display 60 based on the detection results detected by the sound detection unit 20 and the action detection unit 30 and the information input to the input unit 50. For example, the in-vehicle performance device 1 presents a response image based on the action of the passenger PS shooting at an object existing in the surrounding environment of the vehicle V (hereinafter referred to as a shooting action) and calculates a score accompanying the shooting action. The specific structure of the in-vehicle performance device 1 is described below.

[0049] [About the functional structure of the in-car performance device]

[0050] Figure 2 1 is a functional configuration diagram showing an example of the configuration of the in-vehicle performance device 1 according to the first embodiment.

[0051] The in-vehicle performance device 1 includes a control unit 10, a sound detection unit 20, a motion detection unit 30, a camera unit 40, an input unit 50, a display 60, and a storage unit 500. The storage unit 500 stores information representing a response image (hereinafter referred to as response image information 500-1).

[0052] The control unit 10 implements the sound information acquisition unit 11, the motion information acquisition unit 12, the image information acquisition unit 13, the destination information acquisition unit 14, the game management unit 15, the extraction unit 16, the score calculation unit 17 and the display control unit 18 as its functional units by executing the program stored in the storage unit 500 through a processor such as a CPU (Central Processing Unit). These functional units can be implemented by hardware such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or by the cooperation of software and hardware.

[0053] The sound information acquisition unit 11 acquires sound information indicating the sound of the passenger PS detected by the sound detection unit 20. The motion information acquisition unit 12 acquires motion information indicating the motion of the passenger PS detected by the motion detection unit 30. In one example of the present embodiment, the passenger PS makes a sound imitating a shooting sound and performs a motion of pointing to an object (shooting target) existing in the surrounding environment of the vehicle V with a finger as a shooting motion. Therefore, the sound imitating the shooting sound is included in the sound information. Figure 1 The "Bang" shown indicates the content of the sound of the passenger PS imitating the shooting sound. In addition, the action information includes the instruction action of instructing the shooting target. The sound information acquisition unit 11 outputs the acquired sound information to the display control unit 18 and the score calculation unit 17. In addition, the action information acquisition unit 12 outputs the acquired action information to the display control unit 18 and the score calculation unit 17.

[0054] The camera unit 40 captures the surrounding environment in which the vehicle V is traveling, and the image information acquisition unit 13 acquires information indicating the generated surrounding environment image (hereinafter referred to as image information). The image information acquisition unit 13 outputs the acquired image information to the extraction unit 16. The destination information acquisition unit 14 acquires information indicating the destination indicated by the operation input to the input unit 50 (hereinafter referred to as destination information). The vehicle V, for example, travels by automatic driving toward the destination indicated by the destination information acquired by the destination information acquisition unit 14. In addition, the destination information acquisition unit 14 outputs the destination information to the display control unit 18.

[0055] The game management unit 15 manages the shooting game performed by the in-vehicle performance device 1. The game management unit 15 manages the start and end of the shooting game, for example. For example, when the operation input to the input unit 50 indicates the start of the shooting game, the game management unit 15 executes a process to start the shooting game. In addition, when the operation input to the input unit 50 indicates the end of the shooting game, the game management unit 15 executes a process to end the shooting game. The process to start the shooting game is a process that allows the extraction unit 16, the score calculation unit 17, and the display control unit 18 to perform various processes (hereinafter referred to as an enable process). In addition, the process to end the shooting game is a process that does not allow the extraction unit 16, the score calculation unit 17, and the display control unit 18 to perform various processes (hereinafter referred to as a disable process). The extraction unit 16, the score calculation unit 17, and the display control unit 18 perform various processes when the game management unit 15 performs the enable process, and do not perform various processes when the game management unit 15 performs the disable process.

[0056] It should be noted that, in the above, although the game management unit 15 obtains information indicating the start and end of the shooting game based on the operation input to the input unit 50, it is not limited to this. The start and end of the shooting game can also be indicated by the voice of the passenger PS. In this case, the game management unit 15 can also manage the start and end of the shooting game based on the detection result of the sound detection unit 20. In addition, the game management unit 15 can also be configured to determine the warmth of the in-vehicle environment of the vehicle V based on the detection results of the sound detection unit 20 and the action detection unit 30. The game management unit 15 can also automatically perform the enabling process when the warmth of the in-vehicle environment of the vehicle V is low. In this case, the in-vehicle performance device 1 can perform the shooting game when the in-vehicle environment is not warm, making the in-vehicle warm. In addition, the game management unit 15 can also automatically perform the disabling process when the warmth of the in-vehicle environment of the vehicle V is low. In this case, the in-vehicle performance device 1 can end the shooting game when the in-vehicle environment is not warm due to the performed shooting game.

[0057] The extraction unit 16 receives input of image information from the image information acquisition unit 13. The extraction unit 16 performs image recognition processing on the surrounding environment image represented by the image information, and extracts objects that are shooting targets of the passenger PS. The extraction unit 16 extracts objects such as the front vehicle, the rear vehicle, the parallel-running vehicle, and the oncoming vehicle traveling near the vehicle V, and the signs of stores and road signs existing around the vehicle V as shooting targets of the passenger PS. In addition, the extraction unit 16 matches the position of the actual object existing around the vehicle V with the position of the extracted object (shooting target) based on the surrounding environment image, and outputs information indicating the position of the shooting target to the score calculation unit 17.

[0058] Specifically, the extraction unit 16 performs image recognition processing on the surrounding environment image to identify the positions of various objects captured in the surrounding environment image on the surrounding environment image. In addition, the extraction unit 16 matches the position on the front window when the surrounding environment image is displayed in real size on the front window with the position of the extracted object (shooting target) on the surrounding environment image. The extraction unit 16 outputs the position of the front window as the imaginary position information indicating the imaginary position of the shooting target to the score calculation unit 17 and the display control unit 18.

[0059] The score calculation unit 17 calculates the score with respect to the action of the passenger PS based on the sound information input from the sound information acquisition unit 11, the action information acquired from the action information acquisition unit 12, and the virtual position information input from the extraction unit 16. For example, when the sound information indicates a sound imitating a shooting sound and the indicated direction of the indicated action indicated by the action information is the direction of the position of the shooting object indicated by the virtual position information, the score calculation unit 17 calculates a high score for the action of the passenger PS. On the other hand, when the sound information indicates a sound imitating a shooting sound and the indicated direction of the indicated action indicated by the action information is not the direction of the position of the shooting object indicated by the virtual position information, the score calculation unit 17 calculates a low score for the action of the passenger PS. The score calculation unit 17 outputs score information indicating the calculated score to the display control unit 18.

[0060] The display control unit 18 displays a response image on the display 60 based on the sound information input from the sound information acquisition unit 11, the motion information input from the motion information acquisition unit 12, the destination information input from the destination information acquisition unit 14, the virtual position information input from the extraction unit 16, and the score information input from the score calculation unit 17. A specific example of the response image displayed on the display 60 by the display control unit 18 will be described below.

[0061] [Example of a response image showing the detection of a shooting action]

[0062] Figure 3 1A is a diagram showing an example of a response image GP1A indicating that a shooting action has been detected. When the sound information indicates a sound imitating a shooting sound and the action information indicates an instruction action, the display control unit 18 displays a response image (the response image GP1A shown in the figure) indicating that a shooting action has been detected on the display 60. Figure 3 As shown, the response image GP1A is, for example, an image indicating that the shooting has hit the target. The display control unit 18 displays the response image GP1A to inform the passenger PS that the shooting action has been detected.

[0063] [Example of a response image showing that a shot has been taken]

[0064] Figure 41 is a diagram showing an example of a response image GP1B indicating that a shot has been taken. The display control unit 18 displays a response image GP1B indicating that a shot has been taken (response images GP1B-1 to GP1B-4 shown in the figure) on the display 60. Figure 4 As shown, the response image GP1B is, for example, an image in which ink is applied to a predetermined area. The display control unit 18 prompts the response image GP1B in the indicated direction indicated by the action information, indicating to the passenger PS that a shooting action has been performed. In addition, the display control unit 18 can display the response image GP1B in a manner overlapping with the shooting object. Specifically, the display control unit 18 can display the response image (the illustrated response image GP1B-1) in a manner overlapping with an oncoming vehicle. In addition, the display control unit 18 can also display the response image (the illustrated response image GP1B-2) in a manner overlapping with a road sign. In addition, the display control unit 18 can also display the response image (the illustrated response image GP1B-3) in a manner overlapping with a store sign existing around the vehicle V. In addition, when there is no shooting object in the direction indicated by the indicated action, the display control unit 18 can also display the response image (the illustrated response image GP1B-4) on the road surface in that direction.

[0065] It should be noted that the display control unit 18 may also be configured to enlarge or reduce the predetermined area represented by the response image GP1B and display it according to the approach or distance of the shooting target as the vehicle V moves. Specifically, the display control unit 18 may also be configured to enlarge or reduce the predetermined area represented by the response image GP1B so that it represents a predetermined area.

[0066] [Example of response image showing score]

[0067] Figure 5 1 is a diagram showing an example of a response image GP1C indicating a score of the score information. The display control unit 18 displays a response image (the response image GP1C shown in the figure) indicating a score indicated by the score information and calculated based on the action of the passenger PS on the display 60. Figure 5 As shown, the response image GP1C is, for example, an image that expresses the score using numerical values. The display control unit 18 presents the response image GP1C to indicate the evaluation of the shooting action of the passenger PS to the passenger PS. It should be noted that the response image GP1C may also be an image that expresses the score using grades.

[0068] Specifically, the response image GP1C may present an image indicating the level of the passenger PS (eg, “beginner,” “mature,” “expert,” “celebrity,” “master,” etc.) according to the score.

[0069] [About the operation of the in-car performance device]

[0070] Figure 6 1 is a flowchart showing an example of the operation of the in-vehicle performance device 1 of the first embodiment. The sound information acquisition unit 11 acquires sound information from the sound detection unit 20 all the time or at a predetermined time interval (step S110). The motion information acquisition unit 12 acquires motion information from the motion detection unit 30 all the time or at a predetermined time interval (step S120). The image information acquisition unit 13 acquires image information from the camera unit 40 all the time or at a predetermined time interval (step S130). The destination information acquisition unit 14 acquires the destination information indicated by the operation input to the input unit 50 (step S140). The passenger PS inputs the destination information using the input unit 50, for example, when starting the travel of the vehicle V.

[0071] The game management unit 15 manages the start and end of the game (step S145). When the operation input to the input unit 50 indicates the start of the shooting game, the game management unit 15 advances the process to step S150. In addition, when the operation input to the input unit 50 does not indicate the start of the shooting game or indicates the end of the shooting game that has already started, the game management unit 15 ends the process.

[0072] The extraction unit 16 performs image processing on the surrounding environment image represented by the image information acquired by the action information acquisition unit 12, extracts the shooting target, and generates the virtual position information (step S150). For example, the extraction unit 16 extracts the shooting target and generates the virtual position information every time the image information acquisition unit 13 acquires the image information. The score calculation unit 17 determines whether the passenger PS has performed a shooting action based on the sound information acquired by the sound information acquisition unit 11 and the action information acquired by the action information acquisition unit 12 (step S160). When the score calculation unit 17 determines that the passenger PS has performed a shooting action, it calculates the score for the action of the passenger PS based on the action information, the image information acquired by the image information acquisition unit 13, and the virtual position information generated by the extraction unit 16 (step S170). In addition, when the score calculation unit 17 determines that the passenger PS has not performed a shooting action, it does not calculate the score for the action of the passenger PS, and the process proceeds to step S180. The display control unit 18 displays the response image on the display 60 based on the sound information, the action information, the destination information, and the virtual position information (step S180).

[0073] It should be noted that, in the above description, although the shooting action is described as an action of emitting a sound imitating a shooting sound and indicating a shooting target existing in the surrounding environment of the vehicle V with a finger, the present invention is not limited to this. The shooting action may be, for example, either an action of emitting a sound imitating a shooting sound or an action of indicating a shooting target existing in the surrounding environment of the vehicle V. In this case, the in-vehicle performance device 1 may not include a detection unit (sound detection unit 20 or action detection unit 30) for detecting information not used for the shooting action.

[0074] [Summary of the First Embodiment]

[0075] As described above, the in-vehicle performance device 1 of the present embodiment includes the motion detection unit 30 for detecting the motion of the passenger PS of the vehicle V, and the display control unit 18 for controlling the display of the response image (in this example, the response image GP1A-1) corresponding to the motion of the passenger PS based on the motion of the passenger PS detected by the motion detection unit 30. Therefore, the in-vehicle performance device 1 of the present embodiment can provide entertainment to the passenger PS moving through the vehicle V.

[0076] In addition, the display control unit 18 of the in-vehicle performance device 1 of the present embodiment displays a response image (in this example, the response images GP1B-1 to GP1B-3) superimposed on an object (shooting target) in the surrounding environment of the vehicle V. In addition, in the in-vehicle performance device 1 of the present embodiment, the shooting target is a motion indicating an object in the surrounding environment, the motion detection unit 30 detects the direction indicated by the passenger PS (in this example, the motion information), and the display control unit 18 displays a response image (in this example, the response images GP1B-1 to GP1B-4) based on the motion information detected by the motion detection unit 30. In addition, the in-vehicle performance device 1 of the present embodiment further includes a score calculation unit 17 that calculates a score based on the motion information and the virtual position information, and the display control unit 18 further displays a response image (in this example, the response image GP1C) indicating the score calculated by the score calculation unit 17. As a result, the in-vehicle performance device 1 of the present embodiment can display the response image on the display 60 through a more entertaining performance, thereby increasing the enthusiasm of the passenger PS for the shooting action.

[0077] [About the rules and scoring of shooting games]

[0078] It should be noted that, in the above description, although the structure in which the score calculation unit 17 calculates the score based on the direction of the instruction action indicated by the action information and the position of the shooting target is described, it is not limited to this. The score calculation unit 17 can also be configured to calculate the score in accordance with the rules of the shooting game performed by the in-vehicle performance device 1. The rules of the shooting game are, for example, a rule that sets the road sign in the direction of the destination indicated by the destination information as the shooting target, a rule that sets the vehicle of a certain model as the shooting target, a rule that sets the sign of a store as the shooting target, etc. In the case of this rule, a high score is calculated for the shooting action of the shooting target that complies with the rules among the shooting targets extracted by the instruction extraction unit 16.

[0079] In addition, the score calculation unit 17 may also be configured to calculate the score based on the area of ​​the displayed response image GP1B (an image in which ink is applied to a predetermined area). In the following description, the rule for calculating the score based on the area of ​​the displayed response image GP1B is referred to as the territory area rule. In the territory area rule, the score calculation unit 17 calculates the score based on the area of ​​the ink represented by the response image GP1B. Thus, the in-vehicle performance device 1 of the present embodiment can display the response image on the display 60 through a more entertaining performance, thereby increasing the enthusiasm of the passenger PS for the shooting action.

[0080] It should be noted that, in the above description, although the extraction unit 16, the score calculation unit 17, and the display control unit 18 do not perform various processes when the game management unit 15 performs the disabling process, the present invention is not limited to this. For example, it is also possible to configure that when the game management unit 15 performs the disabling process, only the display control unit 18 does not perform various processes. In this case, various functional units perform processes, but various images are not displayed on the display 60.

[0081] [Second Embodiment]

[0082] Hereinafter, a second embodiment of the present invention will be described with reference to the accompanying drawings. In the second embodiment, an in-vehicle performance device 2 that displays an image according to the position of a vehicle V will be described. In addition, an in-vehicle performance device 2 that displays an image displayed by an in-vehicle performance device 2 mounted on another vehicle V on a display 60 of the device will be described. It should be noted that the same reference numerals are used for the same structures as those in the above-mentioned embodiment and the description thereof will be omitted.

[0083] Figure 7 1 is a functional block diagram showing an example of the structure of the in-vehicle performance device 2 according to the second embodiment. The in-vehicle performance device 2 includes a control unit 10A, a sound detection unit 20, a motion detection unit 30, a camera unit 40, an input unit 50, a display 60, a position detection unit 70, a communication unit 80, and a storage unit 500.

[0084] The position detection unit 70 detects the position of the vehicle V. The position detection unit 70 detects the position of the vehicle V by, for example, a method using a global navigation satellite system (GNSS) such as GPS (Global Positioning System) or a method using a regional satellite positioning system (RNSS) such as quasi-zenith satellites (QZS). The position detection unit 70 outputs position information indicating the position of the vehicle V to the control unit 10A.

[0085] The communication unit 80 communicates with a server (not shown) that collects information used in the processing of the in-vehicle performance device 2 through wireless communication. Wireless communication is, for example, short-range wireless communication based on Wi-Fi (registered trademark), wireless communication via a mobile communication network such as LTE (LongTerm Evolution), etc. It should be noted that direct communication can also be performed between vehicles V. In this case, the vehicles V communicate using an AdHoc network such as DSRC (Dedicated Short Range Communications).

[0086] The control unit 10A executes the program stored in the storage unit 500, and implements the sound information acquisition unit 11, the action information acquisition unit 12, the image information acquisition unit 13, the destination information acquisition unit 14, the game management unit 15, the extraction unit 16, the score calculation unit 17, the display control unit 18 and the position information acquisition unit 19 as its functional units.

[0087] The position information acquisition unit 19 acquires the position information from the position detection unit 70. The position information acquisition unit 19 acquires the position information all the time or at predetermined time intervals. The position information acquisition unit 19 outputs the acquired position information to the display control unit 18 and the communication unit 80. In addition, the display control unit 18 of this embodiment outputs the response image displayed on the display 60 to the communication unit 80 all the time or at predetermined time intervals.

[0088] The communication unit 80 establishes a correspondence between the position information input from the position information acquisition unit 19 and the response image input from the display control unit 18, and sends the correspondence to the server. In the following description, the information in which the position information and the response image are corresponded is recorded as position image information. In addition, the communication unit 80 receives position image information sent to the server by other in-vehicle performance devices 2. Specifically, the communication unit 80 receives position image information corresponding to the position information based on the position information input from the position information acquisition unit 19. In other words, the communication unit 80 receives position image information corresponding to the position information representing the current position of the vehicle V from the server. The communication unit 80 outputs the received position image information to the display control unit 18.

[0089] The display control unit 18 of this embodiment receives input of position image information from the communication unit 80. The display control unit 18 displays a response image of the position image information corresponding to the position information that matches the position information input from the position information acquisition unit 19 on the display 60. That is, the display control unit 18 displays a response image that was displayed in the past by the other in-vehicle performance device 2 at the current position of the vehicle V on the display 60.

[0090] [Example of display of response image displayed by other in-vehicle performance device]

[0091] Figure 8 This is a diagram showing an example of the display of response images displayed by other in-vehicle performance devices 2 of the second embodiment. The display control unit 18 displays response images (response images GP1B-1 to GP1B-3 shown in the figure) based on the processing of this device and response images (response images GP2-1 to GP2-3 and response images GP3-1 to GP3-2 shown in the figure) that were displayed by another in-vehicle performance device 2 in the past at the current position. The response images GP2-1 to GP2-3 shown in the figure are response images that were displayed by another in-vehicle performance device 2 in the past at the current position of the vehicle V. In addition, the response images GP3-1 to GP3-2 shown in the figure are response images that were displayed by yet another in-vehicle performance device 2 in the past at the current position of the vehicle V.

[0092] [Summary of the Second Embodiment]

[0093] As described above, the position detection unit 70 of the in-vehicle performance device 2 of the present embodiment detects the position of the vehicle V, the communication unit 80 transmits and receives the position image information, and the display control unit 18 displays the response image (response images GP1B-1 to GP1B-3 shown in the figure) based on the processing of the present device and the response image (response images GP2-1 to GP2-3 and response images GP3-1 to GP3-2 shown in the figure) displayed by other in-vehicle performance devices 2 in the past at the current position of the vehicle V. Thus, the in-vehicle performance device 2 of the present embodiment can display the response image of someone other than the passenger PS playing the shooting game on the display 60, thereby increasing the competitive spirit of the passenger PS and the enthusiasm for the shooting action.

[0094] [Confrontation under the territory area rule]

[0095] It should be noted that the shooting game performed by the in-car performance device 2 can also be configured as a duel between the passenger PS and others under the territory area rule based on the position image information. Specifically, the score calculation unit 17 calculates the score based on the response image ( Figure 8 The score calculation unit 17 calculates the score based on the areas of the response images GP2-1 to GP2-3 and the response images GP3-1 to GP3-2 shown in FIG. 1 and FIG. 2 . In addition, the score calculation unit 17 calculates the score based on the areas of the response images ( Figure 8 The score is calculated based on the area of ​​the response images GP1B-1 to GP1B-3 shown.

[0096] The in-vehicle performance device 2 determines the passenger PS of the in-vehicle performance device 2 (vehicle V) who has displayed a response image with a high score as a winner.

[0097] [Team battle under the territory area rule]

[0098] In addition, it is also possible to configure a shooting game under the territory area rule by team battles. For example, it is also possible to configure a shooting game under the territory area rule by multiple teams (red team, blue team, yellow team, etc.). In this case, the score calculation unit 17 calculates the sum of the area of ​​the response image of the color consistent with the ink color of the response image of the device in the response image of the position image information and the area of ​​the response image displayed based on the processing of the device as the score of the team. In addition, the in-car performance device 2 determines the team with the higher score as the winner.

[0099] [Territory area rules based on driving paths]

[0100] In addition, although the above description is about the case where a shooting game with a territory area rule is performed based on the area of ​​the response image corresponding to the shooting action of the passenger PS, it is not limited to this. The in-vehicle performance device 2 can also perform a shooting game with a territory area rule based on the area of ​​the path traveled by the vehicle V based on the position information obtained by the position information acquisition unit 19. In this case, the display control unit 18 calculates the area by multiplying the path traveled by the vehicle V equipped with this device by a prescribed value indicating the width of the path, and calculates the score corresponding to the area. In addition, the communication unit 80 receives the travel path information indicating the path traveled by the vehicle equipped with other in-vehicle performance devices 2. The score calculation unit 17 calculates the area by multiplying the path traveled by other vehicles indicated by the received travel path information by a prescribed value indicating the width of the path, and calculates the score corresponding to the area. The in-vehicle performance device 2 determines the passenger PS of the vehicle V with the high score as the winner.

[0101] [Display of territory area rules based on driving routes]

[0102] Furthermore, the display control unit 18 may be configured to display, on the display 60 , an image indicating the area of ​​the route along which the vehicle V travels and an image indicating the area of ​​the route indicated by the received travel route information. Fig. 9 : is a diagram showing an example of the result of a shooting game under the territory area rule based on the driving path. In this case, the display control unit 18 generates an image (image P1 shown in the figure) indicating the path traveled by the vehicle V based on the position information acquisition unit 19. In addition, the communication unit 80 receives the driving path information indicating the path traveled by the vehicle V equipped with other in-vehicle performance devices 2. The display control unit 18 generates images (images P2 and P3 shown in the figure) indicating the driving path information received by the communication unit 80. The display control unit 18 overlaps the generated images P1 to P3 with the image of the map representing the specified range (image P5 shown in the figure), and displays them on the display 60. As a result, the in-vehicle performance device 2 of the present embodiment can display the path of movement of others other than the passenger PS on the display 60, thereby increasing the competitiveness of the passenger PS and the enthusiasm based on the movement of the vehicle V.

[0103] It should be noted that, when the passenger PS moves the vehicle V for a long time within a prescribed range in order to win the territory area rule, traffic congestion may occur within the range. In this case, the automatic driving function of the vehicle V may also control the driving of the vehicle V so that the vehicle moves to the destination indicated by the destination information or moves in a manner that avoids staying in the range for a long time.

[0104] [Image showing victory and defeat in a shooting game under the territory area rule]

[0105] Furthermore, when the shooting game based on the territory area rule of the travel route is played in a team battle, the display control unit 18 may display an image indicating the victory or defeat of the game on the display 60 . Fig.10 : is a diagram showing an example of winning or losing in a shooting game under a territory area rule. The display control unit 18 calculates the area of ​​the own team and the area of ​​other teams based on the position information and the driving path information. In addition, the display control unit 18 generates an image (image P6 shown in the figure) representing the ratio of the area of ​​the own team to the area of ​​other teams in the range (Japan as shown in the figure) where the shooting game based on the territory area rule based on the driving path is being played, and displays it on the display 60. It should be noted that the display control unit 18 can also be configured to generate image P6 in a specified range (for example, the range of Kanto Koshinetsu) in the range where the shooting game based on the territory area rule based on the driving path is being played, and display the image P6. As a result, the in-vehicle performance device 2 of the present embodiment can enhance the competitiveness of the passenger PS and can enhance the enthusiasm based on the movement of the vehicle V.

[0106] [Relationship between shooting games and reality]

[0107] In addition, the in-vehicle performance device 2 can also be configured to provide services, content, etc. to the passenger PS based on the score calculated by the score calculation unit 17 and the result of the territory area rule. For example, when the passenger PS performs a shooting action on the signboard of a store existing around the vehicle V, the in-vehicle performance device 2 sends information indicating that the shooting action has been performed to the server device of the store. The server device of the store provides the passenger PS with a coupon that can be used when purchasing the goods of the store. In addition, when the vehicle V is a vehicle used by a car sharing service, the in-vehicle performance device 2 sends the score calculated by the score calculation unit 17 to the server device of the operator providing the service. The server device of the operator provides the passenger PS who has obtained a high score with a coupon that can be used in the next car sharing service. In addition, it can also be configured to provide various coupons to the passenger PS who wins the territory area rule or the passenger PS belonging to the winning team.

[0108] In addition, the in-vehicle performance device 2 may also perform a process of urging movement to a certain shooting target based on information indicating the behavior of each vehicle V (hereinafter referred to as behavior information). In this case, the behavior information is, for example, position image information, position information, and driving path information obtained by each in-vehicle performance device 2.

[0109] The in-vehicle performance device 2 obtains action information obtained from other in-vehicle performance devices 2 from a server storing action information (hereinafter referred to as an action information server). For example, the in-vehicle performance device 2 can be configured to highlight and display, based on the obtained action information, a shooting target for which many other people have performed shooting actions, a shooting target for which a store coupon is provided, and the like. Specifically, the in-vehicle performance device 2 can be configured to display an emphasized image in a manner overlapping with the position of the shooting target on the display 60. In addition, the in-vehicle performance device 2 can also be configured to display on the display 60 an image of the shooting target that was previously photographed by a camera unit mounted on another person's vehicle V. In addition, the in-vehicle performance device 2 can also be configured to display on the display 60 an image indicating a moving direction toward the position of the shooting target.

[0110] Here, the passenger PS sometimes moves the vehicle V in the direction of a shooting target that is emphasized, a shooting target that many other people have shot at, and a shooting target that is provided with store coupons. Therefore, the in-vehicle performance device 2 can urge the in-vehicle performance device 2 (vehicle V) to move toward a certain shooting target based on the action information. The in-vehicle performance device 2 detects a congested state based on the positions of other in-vehicle performance devices 2 (vehicles V) indicated by the action information. The in-vehicle performance device 2 can move the vehicle V in a way that avoids congestion by urging the shooting target to move to a position different from the crowded position (direction) in a way that does not move to the crowded position.

[0111] It should be noted that, instead of the in-vehicle performance device 2 performing the process of urging movement to a certain shooting target based on the action information, the action information server may perform the process of urging movement to a certain shooting target based on the action information. In this case, the action information server transmits an image urging movement of the in-vehicle performance device 2 (vehicle V) and information urging movement to the in-vehicle performance device 2 based on the action information obtained from each in-vehicle performance device 2. The in-vehicle performance device 2 displays the image obtained from the action information server on the display 60, for example.

[0112] In addition, an index indicating the frequency of shooting actions may be given to the shooting object based on the action information obtained from each in-vehicle performance device 2. The in-vehicle performance device 2 or the action information server calculates the index of shooting actions based on the action information obtained from each in-vehicle performance device 2. The in-vehicle performance device 2 may also be configured to emphasize the display of the response image and the shooting object based on the index.

[0113] For example, the in-vehicle performance device 2 determines that there are many in-vehicle performance devices 2 (vehicles V) around a shooting target that is assigned a high index, that is, it is crowded. In this case, the in-vehicle performance device 2 may also perform a (filtering) process of not emphasizing the shooting target or not treating the shooting target as a shooting target. In addition, when the shooting target that is assigned a high index is a shooting target to which a store coupon is provided, the in-vehicle performance device 2 (or the action information server) may also perform a process of enabling a store that is assigned a shooting target (signboard) with a low index in a series of stores of the shooting target store to have the function of providing coupons. In addition, the in-vehicle performance device 2 determines that there are not many in-vehicle performance devices 2 (vehicles V) around a shooting target that is assigned a low index, that is, it is not crowded. In this case, the in-vehicle performance device 2 may also perform a process of emphasizing the shooting target.

[0114] Through these processes, the in-vehicle performance device 2 can urge the in-vehicle performance device 2 (vehicle V) to move toward a certain shooting target based on the action information (index).

[0115] [About output methods other than images]

[0116] In addition, in the above description, although the display control unit 18 outputs the result of the shooting game by displaying various images, the present invention is not limited thereto, and the display control unit 18 may output the result of the shooting game by, for example, sound. Specifically, the vehicle V may be provided with a speaker, and the display control unit 18 may display the response image and output the shooting sound from the speaker. In addition, the display control unit 18 may be configured to display the result and victory of the shooting game represented by the images P5 and P6 on the display 60, and to output the sound notifying the result and victory of the shooting game from the speaker.

[0117] [About the method of displaying images other than on a monitor]

[0118] In addition, although the above description is made of the case where various images are displayed on the display 60, it is not limited to this. For example, in the case of a convertible or a vehicle without a front window, a head-mounted device may be used by a passenger PS in the vehicle. In this case, the head-mounted device displays the surrounding environment image of the vehicle captured by the camera unit 40 and the response image in an overlapping manner. In addition, the vehicle V may also be configured to have a display 60 on the side window in addition to the front window. In this case, the extraction unit 16 performs image recognition processing on the surrounding environment image to recognize the positions of various objects captured in the surrounding environment image on the surrounding environment image. In addition, the extraction unit 16 matches the position on the side window when the surrounding environment image is displayed on the side window in a life-size manner with the position of the extracted object (shooting target) on the surrounding environment image. The extraction unit 16 outputs the position of the front window as the imaginary position information representing the imaginary position of the shooting target to the score calculation unit 17 and the display control unit 18. It should be noted that when the side window is smaller than the front window, the target of the shooting action from the side window is preferably a shooting target with a large volume and area, that is, a shooting target that is easy to shoot in a small area (side window).

[0119] [About a method for displaying an image based on vehicle information]

[0120] In addition, the in-vehicle performance device 1 and the in-vehicle performance device 2 (hereinafter, simply referred to as "in-vehicle performance device") may also have a structure for detecting the state of the vehicle V. In this case, the in-vehicle performance device includes a vehicle state acquisition unit. The vehicle state acquisition unit acquires motion parameters such as the speed, steering angle, pitch and roll angle of the vehicle V. The display control unit 18 displays various images based on the motion parameters acquired by the vehicle state acquisition unit, for example. Specifically, the display control unit 18 performs a refresh process on the image displayed on the display 60 based on the speed of the vehicle V. More specifically, the display control unit 18 performs a refresh operation on the image displayed on the display 60 based on the speed of the vehicle V at time intervals that are sufficiently faster than the changes in the surrounding environment caused by the movement of the vehicle V.

[0121] In addition, the display control unit 18 displays an image corresponding to the traveling direction of the vehicle V on the display 60 based on the steering angle and pitch / roll angle of the vehicle V among the acquired motion parameters. Here, the vehicle V may turn (right turn, left turn) in a direction other than straight travel based on control. In this case, the passenger PS may direct his / her sight not toward the front of the vehicle V but toward the traveling direction, that is, the direction of turning back. The display control unit 18 can display an image corresponding to the sight of the passenger PS by displaying an image corresponding to the traveling direction of the vehicle V on the display 60 based on the motion parameters.

[0122] In addition, the display control unit 18 may also be configured to display various images on the display 60 based on the vibration of the vehicle V. For example, the display control unit 18 displays various images at a distance and direction opposite to the distance and direction in which the vehicle V moves due to the vibration, based on the vibration given to the vehicle V. Thus, the in-vehicle performance device 2 can suppress the image displayed on the display 60 from shaking due to the vibration. Therefore, the in-vehicle performance device 2 can suppress the passenger PS from getting motion sickness due to visually confirming the image on the display 60. It should be noted that the display control unit 18 may also be configured to display various images on the display 60 based on the relative movement distance and direction of the vibration provided to the vehicle V and the vibration provided to the passenger PS.

[0123] [Third Embodiment]

[0124] Hereinafter, a third embodiment of the present invention will be described with reference to the accompanying drawings. In the third embodiment, an in-vehicle performance device 3 is described that determines an indication object based on a three-dimensional point group in a real space in a direction indicated by an indication action of a passenger PS and generates a history of the indication action of the passenger PS. It should be noted that the same reference numerals are used for the same structures as those in the above-mentioned embodiment and the description thereof is omitted. Here, the in-vehicle performance device 3 is an example of an "indication measuring device".

[0125] Fig.11 The functional block diagram shows an example of the structure of the in-vehicle performance device 3 of the third embodiment. The in-vehicle performance device 3 (communication unit 80) of this embodiment transmits and receives information with the terminal of the service provider (hereinafter, service provider terminal device TM) and the interface device 800 via the network NW.

[0126] The interface device 800 includes a control unit 810 and a storage unit 820. The storage unit 820 stores object information 820-1 and service provider information 820-2. The object information 820-1 is information in which a correspondence is established between information indicating an object existing in a real space represented by a three-dimensional point group and information indicating a three-dimensional point group. The service provider information 820-2 is information in which a correspondence is established between information indicating an object and information indicating a service provider associated with the object.

[0127] The control unit 810 searches for the object information 820 - 1 and the service provider information 820 - 2 based on the information received from the in-vehicle performance device 3 , and transmits the information indicating the object and the information indicating the service provider to the in-vehicle performance device 3 .

[0128] The in-vehicle performance device 3 includes a sound detection unit 20, a motion detection unit 30, a camera unit 40, an input unit 50, a display 60, a position detection unit 70, a communication unit 80, and a sight detection unit 90. In addition, the in-vehicle performance device 3 includes a control unit 10B instead of (or in addition to) the control unit 10 or the control unit 10A, and includes a storage unit 500A instead of (or in addition to) the storage unit 500. The storage unit 500A stores response image information 500-1, history information 500-2, and service provider information 500-3. The details of each information will be described later.

[0129] The control unit 10B executes the program stored in the storage unit 500A, and implements the sound information acquisition unit 11, the action information acquisition unit 12, the image information acquisition unit 13, the display control unit 18, the position information acquisition unit 19, the passenger information acquisition unit 111, the line of sight information acquisition unit 112, the coordinate acquisition unit 113, the object information acquisition unit 114, the service provider determination unit 115, the history information generation unit 116, the notification unit 117 and the attribute determination unit 118 as its functional units.

[0130] The passenger PS inputs information that can identify the passenger (hereinafter referred to as a passenger ID) into the input unit 50, and the passenger information acquisition unit 111 acquires the passenger ID input into the input unit 50. The passenger PS inputs the passenger ID into the input unit 50 when boarding the vehicle V, for example.

[0131] The sight line detection unit 90 detects, for example, a position in the actual space visually recognized by the passenger PS (hereinafter referred to as a viewpoint position) captured by a camera unit (not shown) capturing the interior of the vehicle V. The sight line information acquisition unit 112 acquires information indicating the viewpoint position of the passenger PS detected by the sight line detection unit 90 .

[0132] The coordinate acquisition unit 113 acquires a three-dimensional point group in the actual space of the indicated direction of the indicating action of the passenger PS based on the surrounding environment image of the vehicle V captured by the camera unit 40, the indicating action of the passenger PS acquired by the action information acquisition unit 12, and the viewpoint position of the passenger PS acquired by the line of sight information acquisition unit 112. The coordinate acquisition unit 113 detects, for example, feature points of an object in the actual space in the indicated direction that is shown in the surrounding environment image of the vehicle V captured by the camera unit 40, and acquires a set of the detected feature points as a three-dimensional point group in the actual space in the indicated direction. It should be noted that in this case, the camera unit 40 is a stereo camera.

[0133] The object information acquisition unit 114 acquires information indicating an object existing in the actual space based on the information indicating the three-dimensional point group in the actual space of the indicated direction determined by the coordinate acquisition unit 113. For example, the object information acquisition unit 114 sends the information indicating the three-dimensional point group in the actual space of the indicated direction determined by the coordinate acquisition unit 113 to the interface device 800 via the network NW. When the control unit 810 of the interface device 800 receives the information indicating the three-dimensional point group from the in-vehicle performance device 3, it uses the information as a search keyword to search for the object information 820-1 and determines the object that has established a corresponding relationship with the three-dimensional point group indicated by the information. The control unit 810 sends the information indicating the determined object to the in-vehicle performance device 3 via the network NW. The object information acquisition unit 114 acquires the information indicating the object received from the interface device 800.

[0134] The service provider determination unit 115 determines the service provider with which the object acquired by the object information acquisition unit 114 has established a corresponding relationship. For example, the service provider determination unit 115 sends the information indicating the object acquired by the object information acquisition unit 114 to the interface device 800 via the network NW. When the control unit 810 of the interface device 800 acquires the information indicating the object from the in-vehicle performance device 3, it uses the information as a search keyword to search the service provider information 820-2 and determines the service provider with which the object indicated by the information has established a corresponding relationship. The control unit 810 sends the information indicating the determined object to the in-vehicle performance device 3 via the network NW. The service provider determination unit 115 acquires the information indicating the service provider received from the interface device 800.

[0135] The history information generating unit 116 generates history information 500 - 2 indicating the history of the instruction target instructed by the passenger PS through the instruction action.

[0136] Fig.12 500-2 is a diagram showing an example of the content of the history information 500-2. Fig.12As shown, the history information 500-2 is information in which the passenger ID, the date and time when the passenger PS performed the instruction action (hereinafter referred to as the instruction date and time), the indicated object indicated by the instruction action of the passenger PS, and the service provider corresponding to the indicated object are established in correspondence with each other for each passenger ID. The history information generating unit 116 establishes a correspondence between, for example, the indicated date and time when the instruction action of the passenger PS is obtained by the action information acquiring unit 12, the object indicated by the instruction action indicated by the information obtained by the object information acquiring unit 114, the service object corresponding to the indicated object indicated by the information obtained by the service provider determining unit 115, and the passenger ID of the passenger PS, generates the history information 500-2, and stores it in the storage unit 500A. It should be noted that the relationship between the object and the service provider can be arbitrarily set. For example, it is not necessary that the object is the property of the service provider. In addition, an object without commerciality (for example, a logo, a signal, etc. in the first embodiment) can also be defined as an object associated with the service provider.

[0137] The notification unit 117 notifies the service provider associated with the object designated by the passenger PS that the designated object has been designated by the passenger PS.

[0138] Fig.13 500-3 is a diagram showing an example of the content of the service provider information 500-3. Fig.13 As shown, the service provider information 500-3 is information indicating that a correspondence relationship is established between information of a service provider (name of the service provider in the figure) and a notification object. The notification unit 117 uses the service provider identified by the service provider identification unit 115 as a search keyword to search the service provider information 500-3, and notifies the notification object that has established a correspondence relationship with the service provider that the passenger PS has performed an instruction action, that is, the passenger PS has an interest in the object that has established a correspondence relationship with the service provider. The notification object shows, for example, the address of the service provider terminal device TM (for example, a server device).

[0139] The notification unit 117 may, for example, notify the service provider each time the passenger PS performs an instruction action, or extract each service provider with which a corresponding relationship is established in the history information 500-2 of a predetermined period, and notify the service provider. In addition, the notification unit 117 may notify the passenger PS of the history information 500-2 in addition to the service provider. Thus, the passenger PS can use the history information 500-2 as information indicating his or her own interests. In addition, the passenger PS can obtain information related to the service provider who performed the instruction action by referring to the notified history information 500-2.

[0140] [Summary of the Third Embodiment]

[0141] As described above, the in-vehicle performance device 3 of the present embodiment comprises: an action detection unit 30 for detecting the action of the passenger PS of the vehicle V; a sight line detection unit 90 for detecting the viewpoint position of the passenger PS; a sight line information acquisition unit 112 for acquiring a three-dimensional point group in the actual space of the indicated direction of the indicated action of the passenger PS based on the indicated action of the passenger PS detected by the action detection unit 30 and the viewpoint position of the passenger PS detected by the sight line detection unit 90; and an object information acquisition unit 114 for acquiring, from the interface device 800 for supplying information indicating an object existing in the actual space indicated by the three-dimensional point group, the three-dimensional point group corresponding to the three-dimensional point group obtained by the coordinate acquisition unit 113. The information of the object with which the point group has established a corresponding relationship; the service provider determination unit 115 determines the service provider with which the object represented by the information obtained by the object information acquisition unit based on the service provider information 820-2 indicating the service provider with which the object has established a corresponding relationship; and the history information generation unit 116 generates the history information 800-2 which establishes a corresponding relationship between the service provider determined by the service provider determination unit 115 and the object represented by the information obtained by the object information acquisition unit 114, and by obtaining the history of the passenger PS performing the indicating action, it is possible to collect information related to the interests and concerns of the passenger PS. In addition, according to the in-vehicle performance device 3 of this embodiment, it is possible to notify the service provider with which the object with which the indicating action has been performed that the indicating action has been performed.

[0142] [Regarding the Three-Dimensional Point Group Sent to the Interface Device 800]

[0143] It should be noted that, in the above description, although the information representing the three-dimensional point group obtained by the coordinate acquisition unit 113 is all sent to the interface device 800, the present invention is not limited to this. The coordinate acquisition unit 113 may be configured to select a three-dimensional point group to be sent to the interface device 800 from among the obtained three-dimensional point groups based on the attributes of the surrounding environment of the vehicle V, and send the three-dimensional point group.

[0144] In this case, the attribute determination unit 118 determines the attribute of the surrounding environment of the vehicle V based on the surrounding environment image of the vehicle V taken by the camera unit 40. The attribute is, for example, the property (feature) of the surrounding environment of the vehicle V, such as a highway, an area with dense shops, etc. The attribute determination unit 118 determines the attribute of the surrounding environment of the vehicle V by, for example, analyzing whether the traffic road signs of the highway or many store signs are captured in the surrounding environment image of the vehicle V taken by the camera unit 40. Here, as an example of "attribute", the property (feature) of the surrounding environment is cited, but it is not limited to this. "Attribute" can also determine multiple attributes of a part or all of the attributes of an identifiable object in the form of examples relative to coordinates, directions, etc.

[0145] It should be noted that the attribute determination unit 118 may also be configured to send the surrounding environment image of the vehicle V captured by the camera unit 40 to the interface device 800 via the network NW, and the interface device 800 determines the attribute of the surrounding environment of the vehicle V. In this case, the attribute determination unit 118 determines the attribute by transmitting the surrounding environment image of the vehicle V captured by the camera unit 40 via the network NW and acquiring information indicating the attribute of the surrounding environment of the vehicle V from the interface device 800.

[0146] In addition, the attribute determination unit 118 may be configured to determine the attribute of the surrounding environment of the vehicle V based on the position attribute information in which the position information and the attribute are associated with each other. In this case, the position attribute information may be stored in the storage unit 500A or in the storage unit 820. When the position attribute information is stored in the storage unit 820, the attribute determination unit 118 determines the attribute by transmitting information indicating the current position of the vehicle V via the network NW and acquiring information indicating the surrounding environment of the vehicle V from the interface device 800.

[0147] The coordinate acquisition unit 113 sends information representing the three-dimensional point group to the interface device 800 based on the attribute determined by the attribute determination unit 118. Here, when the vehicle V moves at high speed on the "highway", the amount of data of the three-dimensional point group obtained in the instruction action may increase. Therefore, when the attribute determined by the coordinate acquisition unit 113 is, for example, "highway", the range of the three-dimensional point group to be obtained indicated by the instruction action can be narrower than usual, thereby suppressing the amount of data sent to the interface device 800.

[0148] In addition, when the vehicle V moves in a "store-dense area", there are sometimes multiple objects (i.e., signs) indicated by the indication action. Therefore, when the determined attribute is, for example, a "store-dense area", the coordinate acquisition unit 113 can make the range of the acquired three-dimensional point group indicated by the indication action wider than usual, and obtain a lot of information representing the object from the interface device 800. As a result, the in-vehicle performance device 3 can generate information-rich history information 500-2. That is, based on the indication actions including the actions of passengers not related to driving measured by the indication measuring device as one aspect of the invention, various displays and performances can be performed for the passengers as described in the first embodiment. On the other hand, by appropriately recording and maintaining the indication actions and their history, it is possible to seek reuse of information and utilization for other purposes.

[0149] Although the embodiments of the present invention have been described in detail with reference to the drawings, the specific structure is not limited to the embodiments and can be appropriately modified within the scope of the present invention. The structures described in the above embodiments can also be combined.

Claims

1. An in-vehicle performance device that performs a game played by passengers in a vehicle, wherein: The in-car performance device comprises: a motion detection unit that detects a motion of an object indicating a surrounding environment that is not related to driving of the vehicle; An imaging unit that captures the surrounding environment and obtains image information representing the generated surrounding environment image; an extraction unit for extracting a specific object and its attribute based on the image information acquired by the imaging unit; a display unit that displays an image that can be visually recognized by the passenger; as well as a display control unit that displays a response image corresponding to the passenger's action on the display unit based on a comparison between the indicated direction of the action and the position information of the object, the attribute of the specific object extracted by the extraction unit, and the action of the passenger detected by the action detection unit, and outputs a result of a game based on a prescribed rule, The display control unit causes the response image to be displayed on the display unit by overlapping an object in the surrounding environment of the vehicle. The in-vehicle performance device further includes a score calculation unit that calculates the number of times the motion detection unit detects a motion indicating a specific object as a valid score. the display control unit causes the display unit to display an image indicating the score calculated by the score calculation unit at a predetermined timing, The in-vehicle performance device further includes a destination information acquisition unit for acquiring destination information indicating a destination. The extraction unit extracts the specific road sign leading to the destination indicated by the destination information in the surrounding environment image. The score calculation unit calculates the number of times the action detection unit detects the action of the passenger indicating the position corresponding to the road sign extracted by the extraction unit as a valid score.

2. The in-vehicle performance device according to claim 1, wherein: The display control unit causes the display unit to display the response image on the display unit so as to overlap with a specific object in the indicated direction based on the direction detected by the motion detection unit.

3. The in-vehicle performance device according to claim 1 or 2, wherein: The extracting unit extracts another vehicle traveling near the vehicle in the surrounding environment image as the specific object. The score calculation unit calculates the number of times the action detection unit detects the action of the passenger indicating the position corresponding to the other vehicle extracted by the extraction unit as a valid score.

4. The in-vehicle performance device according to claim 1 or 2, wherein: The extracting unit extracts a signboard existing around the vehicle in the surrounding environment image as the specific object. The score calculation unit calculates the number of times the action detection unit detects the action of the passenger indicating the position corresponding to the sign extracted by the extraction unit as a valid score.

5. The in-vehicle performance device according to claim 1 or 2, wherein: The display control unit causes the display unit to display an image indicating the total of the scores calculated by the score calculation unit.

6. An in-vehicle performance system, comprising a plurality of in-vehicle performance devices according to any one of claims 1 to 5, wherein: The in-car performance device comprises: a transmitting unit that transmits information indicating the score calculated by the score calculating unit to other in-vehicle performance devices; and a receiving unit for receiving score information indicating the score calculated by the other in-vehicle performance device, The display control unit causes the display unit to display an image indicating the score calculated by the score calculation unit and an image indicating the score of the score information received by the reception unit.

7. The in-vehicle performance system according to claim 6, wherein: The display control unit compares the image of the score representing the score information received from the in-vehicle performance device mounted on a vehicle belonging to the same team as this device and the image of the score representing the score information received from the in-vehicle performance device mounted on a vehicle belonging to another team, and displays them on the display unit.

8. The in-vehicle performance system according to claim 6 or 7, wherein: The display control unit displays an image urging movement based on information indicating the behavior of a vehicle equipped with the own device and information indicating the behavior of a vehicle equipped with another in-vehicle performance device.

9. A method for performing in a car, wherein: The in-vehicle performance method causes a computer equipped with a display unit and performing a game played by passengers in a vehicle to execute the following processing: Detecting motion of objects indicative of the surrounding environment that are not relevant to driving of the vehicle; photographing the surrounding environment and obtaining image information representing the generated surrounding environment image; Extract specific objects and attributes based on the acquired image information; displaying an image that can be visually confirmed by the passenger; Based on the comparison between the indicated direction of the motion and the position information of the object, the extracted attributes of the specific object, and the detected motion of the passenger, a response image corresponding to the motion of the passenger is displayed, and a result of a game based on a prescribed rule is output; causing the response image to overlap with an object in the surrounding environment of the vehicle and to be displayed on the display unit; The number of times the motion indicating the specific object is detected is calculated as a valid score; causing the display unit to display an image representing the calculated score at a predetermined timing; obtaining destination information indicating a destination; extracting the specific road sign leading to the destination indicated by the destination information in the surrounding environment image; as well as The number of times the action of the passenger pointing to the position corresponding to the extracted road sign is detected is calculated as a valid score.

10. A storage medium storing a program, wherein: The program causes a computer equipped with a display unit and performing a game played by passengers on a vehicle to execute the following processing: Detecting motion of objects indicative of the surrounding environment that are not relevant to driving of the vehicle; photographing the surrounding environment and obtaining image information representing the generated surrounding environment image; Extract specific objects and attributes based on the acquired image information; displaying an image that can be visually confirmed by the passenger; Based on the comparison between the indicated direction of the motion and the position information of the object, the extracted attributes of the specific object, and the detected motion of the passenger, a response image corresponding to the motion of the passenger is displayed, and a result of a game based on a prescribed rule is output; causing the response image to overlap with an object in the surrounding environment of the vehicle and to be displayed on the display unit; The number of times the motion indicating the specific object is detected is calculated as a valid score; causing the display unit to display an image representing the calculated score at a predetermined timing; obtaining destination information indicating a destination; extracting the specific road sign leading to the destination indicated by the destination information in the surrounding environment image; as well as The number of times the action of the passenger pointing to the position corresponding to the extracted road sign is detected is calculated as a valid score.

Images