IMAGE DISPLAY SYSTEM, IMAGE DISPLAY METHOD, AND COMPUTER PROGRAM

Abstract

The present invention improves the convenience of an image display system. This image display system is provided with: an image display device; a support device that supports the image display device and makes it possible to change the orientation of the image display device; and a control device for controlling the image display device and the support device. The control device includes a space information acquisition unit, an orientation control unit, and an image display control unit. The space information acquisition unit acquires space information including first position information indicating the position of the image display device and second position information indicating the position of an object in the vicinity of the image display device. The orientation control unit causes the support device to change the orientation of the image display device on the basis of the space information. The image display control unit causes the image display device to display an image corresponding to the orientation of the image display device.

Description

[Technical field]

[0001] The technology disclosed in this specification relates to an image display system. [Background technology]

[0002] In recent years, the development of autonomous driving technology for automobiles and other vehicles has been progressing rapidly. Autonomous driving technology is a technology that uses various sensors (cameras, radar, LIDAR, ultrasonic sensors, GPS, etc.) and high-precision 3D maps to recognize the vehicle's position and surrounding environment, and drives the vehicle autonomously to its destination.

[0003] When the technology for automatic driving of a vehicle is sufficiently improved so that the system performs all driving operations such as acceleration, steering, and braking under certain circumstances (for example, when driving on a highway) or under all circumstances, and the driver does not need to perform driving operations, the driver, as a passenger, can allocate his / her time to things other than driving operations. For example, the passenger can watch videos or play games using an image display system (infotainment system) including an image display device such as an in-vehicle display used in a car navigation system or an image display device such as a smartphone, a tablet terminal, or an HMD (Head Mounted Display) brought by the passenger (for example, see Patent Documents 1 to 3). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2017-49901 A [Patent Document 2] JP 2017-97921 A [Patent Document 3] JP 2017-81256 A Summary of the Invention [Problem to be solved by the invention]

[0005] In an image display system for passengers of an autonomous vehicle, for example, when introducing tourist attractions visible from the vehicle window, it is sometimes necessary to display an image corresponding to the direction on the image display device while indicating the direction (for example, an image showing information for introducing an object (for example, a historical building) located in the direction (for example, the name, size, construction year, etc. of the building). In such a case, even if an image (for example, an image of an arrow pointing right) is displayed on the screen of an in-vehicle display, smartphone, or tablet terminal, or a sound (for example, a sound saying "Look to your right") is output from a speaker, it is difficult to convey the accurate direction in which the passenger should look. In addition, although it is possible to convey the accurate direction by using a wearable image display device such as an HMD, wearing the wearable image display device is cumbersome, and the appearance when worn is unnatural. Thus, in an image display system including a conventional image display device, it is difficult to display an image corresponding to the direction on the image display device while indicating the accurate direction without using a wearable image display device, and there is room for improvement in terms of convenience.

[0006] Note that such a problem is not limited to image display systems mounted on autonomous vehicles, but is a common problem in general image display systems equipped with image display devices.

[0007] This specification discloses a technique that can solve the above-mentioned problems. [Means for solving the problem]

[0008] The technology disclosed in this specification can be realized, for example, in the following forms.

[0009] (1) The image display system disclosed in the present specification is an image display system comprising an image display device, a support device that supports the image display device and is capable of changing the orientation of the image display device, and a control device that controls the image display device and the support device, the control device having a spatial information acquisition unit that acquires spatial information including first position information indicating the position of the image display device and second position information indicating the position of an object around the image display device, an orientation control unit that causes the support device to change the orientation of the image display device based on the spatial information, and an image display control unit that causes the image display device to display an image corresponding to the orientation of the image display device. In this image display system, the orientation of the image display device is changed based on spatial information including the first position information indicating the position of the image display device and the second position information indicating the position of an object around the image display device, and an image corresponding to the orientation of the image display device is displayed on the image display device. Therefore, according to this image display system, it is possible to indicate an accurate direction to a user by the orientation of the image display device without using a wear-side image display device, and to display an image corresponding to the direction on the image display device, thereby improving the convenience of the image display system.

[0010] (2) In the above image display system, the control device may further include a target setting unit that sets one of the surrounding objects as a target object based on the spatial information, and the orientation control unit may be configured to cause the support device to change the orientation of the image display device so that the image display device points to the target object. According to this image display system, the direction of the user's line of sight can be guided to the target object, which is one of the surrounding objects, depending on the orientation of the image display device, thereby further improving the convenience of the image display system.

[0011] (3) In the above image display system, the control device may further include a target information acquisition unit that acquires target information, which is information related to the target object, and the image display control unit may be configured to display an image showing the target information on the image display device. According to this image display system, the direction of the user's line of sight can be guided to a target object, which is one of the objects in the vicinity, depending on the orientation of the image display device, and an image showing information related to the target object can be displayed on the image display device, thereby further improving the convenience of the image display system.

[0012] (4) In the above image display system, the target information may include information indicating at least one of the name, size, and construction year of the target object. According to this image display system, an image indicating information indicating at least one of the name, size, and construction year of the target object can be displayed on the image display device, and useful information can be presented to the user.

[0013] (5) In the above image display system, the support device may support the image display device so that the orientation of the image display device can be manually changed, and the target setting unit may set an object located in a direction indicated by the image display device as the target object when the orientation of the image display device is manually changed. According to this image display system, when a user points the image display device in the direction of a desired object, the object is set as the target object, so that the target object can be set by an intuitive operation of the user, and the convenience of the image display system can be further improved.

[0014] (6) In the above image display system, the image display device may have a camera that captures an image of a subject located in a direction pointed by the image display device, and the image display control unit may be configured to cause the image display device to display an image captured by the camera when the orientation of the image display device is manually changed. According to this image display system, the current orientation of the image display device can be fed back to the user by the image displayed on the image display device, so that the user can easily manually change the orientation of the image display device so that the image display device accurately points to a desired object, thereby further improving the convenience of the image display system.

[0015] (7) In the above image display system, the image display device may have a planar image display surface, and the direction indicated by the image display device may be perpendicular to the image display surface and point in a direction opposite to the image display surface. This image display system allows a user to more accurately grasp the direction indicated by the image display device, thereby further improving the convenience of the image display system.

[0016] (8) In the above image display system, the support device may have three or more rotation axes. According to this image display system, the orientation of the image display device can be adjusted more flexibly, thereby further improving the convenience of the image display system.

[0017] (9) In the above image display system, the image display system may be mounted on an autonomous vehicle, the support device may support the image display device on the vehicle, and the spatial information acquisition unit may acquire the spatial information obtained by a sensor installed in the vehicle. According to this image display system, by changing the orientation of the image display device using information used for autonomous driving, it is possible to display an image corresponding to the direction on the image display device while indicating an accurate direction related to the traveling of the vehicle to passengers of the autonomous vehicle and people outside the vehicle, thereby improving the convenience of the image display system.

[0018] The technology disclosed in this specification can be realized in various forms, such as an image display device, an image display system, an image display method, a computer program for realizing these methods, a non-transitory recording medium on which the computer program is recorded, etc. [Brief description of the drawings]

[0019] [Figure 1] FIG. 1 is an explanatory diagram showing a schematic configuration of an image display system 10 according to an embodiment of the present invention. [Diagram 2] 1 is a flowchart showing the contents of an image display control process according to the present embodiment. [Diagram 3] FIG. 1 is an explanatory diagram showing a schematic state of the image display device 150 and the like during image display control processing in this embodiment. [Figure 4] FIG. 1 is an explanatory diagram showing a schematic state of the image display device 150 and the like during image display control processing in this embodiment. [Diagram 5] FIG. 1 is an explanatory diagram showing a schematic state of the image display device 150 and the like during image display control processing in this embodiment. [Figure 6] FIG. 1 is an explanatory diagram showing a schematic diagram of an image display device 150 according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] A. Embodiment: A-1. Configuration of image display system 10: 1 is an explanatory diagram showing a schematic configuration of an image display system 10 according to the present embodiment. The image display system 10 according to the present embodiment is an infotainment system that is mounted on an autonomous vehicle VH and can provide infotainment such as videos, games, and tourist information to a passenger PA of the vehicle VH.

[0021] As shown in FIG. 1, the image display system 10 includes an image display device 150, a support device 160, and a control device 100. The image display device 150 includes a display such as a liquid crystal display or an organic EL display, and displays images such as still images and videos. The image display device 150 is shaped like a substantially flat plate having a substantially planar front surface FS (i.e., an image display surface) and a surface opposite to the front surface FS (hereinafter referred to as a "rear surface BS"). The image display device 150 includes a camera (rear surface camera) 152 having a lens arranged on the rear surface BS. The camera 152 captures an image of a subject located in the direction of the rear surface BS of the image display device 150 to generate an image. The image display device 150 may include other components such as a front surface camera having a lens arranged on the front surface FS and a speaker. The image display device 150 may also include a touch panel, a microphone, and the like, and function as an input device.

[0022] The support device 160 is a device that is fixed to a predetermined position (the dashboard in this embodiment) of the vehicle VH and supports the image display device 150. In this embodiment, the support device 160 is configured by a robot arm having a large number of (three or more) joints and drive motors 162 that drive each joint. The support device 160 can change the position and orientation of the image display device 150 by driving each joint with each drive motor 162 (see Figs. 3 and 4). Hereinafter, such a change in the position and orientation of the image display device 150 is referred to as an "automatic change" of the position and orientation of the image display device 150.

[0023] Moreover, the support device 160 of this embodiment is configured such that the position and orientation of the image display device 150 can be changed by the application of an external force to the image display device 150 or the support device 160 by setting the torque of each drive motor 162 to zero, and the position and orientation of the image display device 150 are fixed when the application of the force is removed. Therefore, for example, the passenger PA can grasp the image display device 150 and move or rotate the image display device 150 to change the position and orientation of the image display device 150 to a desired position and orientation (see FIG. 5). Hereinafter, such a change in the position and orientation of the image display device 150 is referred to as a "manual change" of the position and orientation of the image display device 150.

[0024] The control device 100 is a computer for controlling each component of the vehicle VH, and includes a control unit 110 and a storage unit 170. These units are connected to each other via a bus 190 so as to be able to communicate with each other.

[0025] The storage unit 170 of the control device 100 is configured with, for example, a ROM, a RAM, a hard disk drive (HDD), etc., and is used to store various programs and data, and as a work area when executing various programs and a temporary storage area for data. For example, the storage unit 170 stores an automatic driving control program 172, which is a computer program for executing automatic driving of the vehicle VH, and an image display control program 174 and a support control program 176, which are computer programs for executing an image display control process described later. These programs are provided in a state stored in a computer-readable recording medium (not shown), such as a CD-ROM, a DVD-ROM, or a USB memory, and are stored in the storage unit 170 by installing them in the control device 100.

[0026] A three-dimensional map database MD is also constructed in the storage unit 170 of the control device 100. A highly accurate three-dimensional map used for the automatic driving of the vehicle VH is stored in the three-dimensional map database MD. This three-dimensional map includes, for example, information on the positions and shapes of traffic signals, road signs, road trees, guard rails, buildings, etc., in addition to information on the positions and shapes of roads.

[0027] An infotainment database ID is also constructed in the storage unit 170 of the control device 100. The infotainment database ID stores information for providing infotainment such as videos, games, and tourist information to the passenger PA of the vehicle VH. In this embodiment, in order to provide tourist information to the passenger PA of the vehicle VH, the infotainment database ID stores information indicating the position of an object (more specifically, a tourist attraction such as a building or natural object) (hereinafter referred to as "tourist attraction position information I2") and information indicating specifications of the tourist attraction such as its appearance, name, size, and year of construction (hereinafter referred to as "specification information I3").

[0028] The control unit 110 of the control device 100 is configured with, for example, a CPU, and controls the operation of each part of the vehicle VH by executing a computer program read from the storage unit 170. For example, the control unit 110 controls the autonomous driving of the vehicle VH by reading an autonomous driving control program 172 from the storage unit 170 and executing it. At this time, the control unit 110 functions as an autonomous driving control unit 120. The autonomous driving control unit 120 includes a position / orientation detection unit 122, a surrounding situation detection unit 124, and a route setting unit 126. The functions of each of these units will be described in conjunction with the description of the autonomous driving control process described later.

[0029] Furthermore, the control unit 110 controls the image display device 150 and the support device 160 by reading and executing an image display control program 174 and a support control program 176 from the storage unit 170. At this time, the control unit 110 functions as an image display control unit 130 and a support control unit 140. The image display control unit 130 includes a spatial information acquisition unit 132, a target setting unit 134, and a target information acquisition unit 136. Furthermore, the support control unit 140 includes a direction control unit 142. The functions of each of these units will be described in conjunction with the description of the image display control process described later.

[0030] A-2.Automatic driving control processing: Next, an automatic driving control process by the automatic driving control unit 120 of the control device 100 will be described. The automatic driving control process is a process for controlling the automatic driving of the vehicle VH. The contents of the automatic driving control process in this embodiment are similar to known contents and are roughly as follows.

[0031] First, the route setting unit 126 of the automatic driving control unit 120 sets a travel route (hereinafter referred to as a "route") from the departure point to the destination in accordance with an instruction from the passenger PA. The route can be set by various methods, for example, by referring to a road network database in which roads are represented by nodes and links, and setting the route by route search based on the database.

[0032] When the route is set, the automatic driving control unit 120 controls the acceleration, steering, and braking of the vehicle VH so that the vehicle VH travels along the set route, thereby autonomously driving the vehicle VH. At this time, the position / orientation detection unit 122 of the automatic driving control unit 120 detects the position (latitude, longitude, altitude, etc.) and orientation (direction of travel, azimuth, etc.) of the vehicle VH based on signals from various sensors 102 (GPS, geomagnetic sensor, etc.) mounted on the vehicle VH, or based on the signals and a high-precision three-dimensional map stored in the three-dimensional map database MD. In addition, the surrounding situation detection unit 124 of the automatic driving control unit 120 detects the surrounding situation of the vehicle VH (positions and shapes of objects (other vehicles, pedestrians, roadside trees, lanes, etc.) surrounding the vehicle VH, etc.) based on signals from various sensors 102 (cameras, radar, LIDAR, ultrasonic sensors, GPS, etc.) mounted on the vehicle VH, or based on the signals and a high-precision three-dimensional map stored in the three-dimensional map database MD. The autonomous driving control unit 120 autonomously drives the vehicle VH to the destination along a set route based on the position and orientation of the vehicle VH detected by the position / orientation detection unit 122 and the surrounding conditions detected by the surrounding conditions detection unit 124.

[0033] A-3. Image display control processing: Next, the image display control process executed by the image display system 10 of this embodiment will be described. FIG. 2 is a flowchart showing the contents of the image display control process in this embodiment. Also, FIGS. 3 to 5 are explanatory diagrams that typically show the state of the image display device 150 and the like during the image display control process in this embodiment. The image display control process in this embodiment is a process that presents information introducing tourist attractions visible from the car window to the passenger PA of the vehicle VH traveling toward a destination by automatic driving as infotainment using the image display device 150, and is started, for example, when an instruction to start the process is given by the passenger PA. FIG. 3 shows the state of the image display device 150 at the start of the image display control process. As shown in FIG. 3, at the start of the image display control process, the image display device 150 is positioned by the support device 160 in front of the passenger PA who is seated facing forward, and is positioned and oriented so that the front surface FS faces the passenger PA directly.

[0034] When the image display control process is started, the spatial information acquisition unit 132 of the image display control unit 130 acquires the vehicle position information I1 indicating the position of the vehicle VH (S110). In this embodiment, the vehicle position information I1 includes information indicating the orientation of the vehicle VH in addition to the position of the vehicle VH. In this embodiment, the spatial information acquisition unit 132 acquires, as the vehicle position information I1, information on the position and orientation of the vehicle VH detected by the position / orientation detection unit 122 of the automatic driving control unit 120 based on signals from the various sensors 102 in the above-mentioned automatic driving control process. Since the image display system 10 including the image display device 150 is mounted on the vehicle VH, the vehicle position information I1 indicating the position and orientation of the vehicle VH can also be said to be information indicating the position and orientation of the image display device 150. The position and orientation of the image display device 150 referred to here are not the "relative" position and orientation with respect to the vehicle VH, but the "absolute" position and orientation with respect to features such as roads. For example, even if the relative position and orientation of the image display device 150 with respect to the vehicle VH is fixed, if the position and orientation of the vehicle VH change as the vehicle VH travels, the absolute position and orientation of the image display device 150 with respect to the above-mentioned features also change. The vehicle position information I1 is an example of the first position information and spatial information in the claims.

[0035] Next, the image display control unit 130, in cooperation with the support control unit 140, determines whether or not the orientation of the image display device 150 described above has been manually changed (S120). Whether or not the orientation of the image display device 150 has been manually changed can be determined, for example, by detecting with a sensor whether or not each joint of the support device 160 has moved without being driven by the drive motor 162.

[0036] When it is determined that the orientation of the image display device 150 has not been manually changed (S120: NO), the goal setting unit 134 of the image display control unit 130 determines whether or not there is an object (tourist attraction) that can be introduced in the vicinity of the vehicle VH (S130). More specifically, as described above, the tourist attraction position information I2 is stored in the infotainment database ID, and this tourist attraction position information I2 is acquired by the spatial information acquisition unit 132. The goal setting unit 134 determines whether or not there is a tourist attraction that can be seen from the vehicle window of the vehicle VH at the current position, based on the relationship between the position of the vehicle VH specified by the vehicle position information I1 and the positions of each tourist attraction specified by the acquired tourist attraction position information I2. Note that a correspondence relationship between each position on a road or a parking lot and a tourist attraction that can be seen from there may be set in advance, the correspondence relationship may be stored in the infotainment database ID, and the goal setting unit 134 may make the determination in S130 by referring to the correspondence relationship. The tourist attraction position information I2 is an example of second position information and spatial information in the claims.

[0037] In the above S130, if it is determined that there is an object (tourist attraction) that can be introduced in the vicinity of the vehicle VH (S130: YES), the target setting unit 134 sets one of such tourist attractions as the target object TA, and the target information acquisition unit 136 acquires specification information I3 of the target object TA from the infotainment database ID (S140). The specification information I3 of the target object TA is information indicating the specifications of the target object TA (appearance, name, size, year of construction, etc.). The specification information I3 of the target object TA is an example of the target information in the claims.

[0038] Next, the orientation control unit 142 of the support control unit 140 causes the support device 160 to change the orientation of the image display device 150 so that the image display device 150 points to the target object TA based on the vehicle position information I1 and the tourist attraction position information I2 of the target object TA (S150). Here, in this specification, the direction in which the image display device 150 points is a direction that is perpendicular to the front surface (image display surface) FS of the image display device 150 and faces the opposite side (back surface BS side) from the front surface FS. In other words, the state in which the image display device 150 is oriented to point to the target object TA is a state in which a virtual straight line VL that passes through an arbitrary point (for example, the center point P0 of the screen) on the front surface (image display surface) FS of the image display device 150 and is perpendicular to the front surface FS intersects with the target object TA, as shown in FIG. 4. The orientation control unit 142 specifies the position and orientation of the vehicle VH (i.e., the position and orientation of the image display device 150) based on the vehicle position information I1, and specifies the position of the target object TA based on the tourist attraction position information I2, calculates the direction from the image display device 150 toward the target object TA (the direction of the virtual straight line VL), and controls the drive motor 162 to drive each joint of the support device 160 based on the direction, thereby changing the orientation of the image display device 150. FIG. 4 shows a state in which the image display device 150 has been changed to point to the first target object TA1. In response to the change in the orientation of the image display device 150, the passenger PA moves his / her line of sight to the direction pointed by the image display device 150 (i.e., the direction in which the target object TA is located). In this embodiment, the support device 160 is configured by a robot arm having many joints, and is capable of changing the position of the image display device 150 in addition to the orientation, so that the position of the image display device 150 is also changed in addition to the orientation of the image display device 150. The position of the image display device 150 after the change is a position where the virtual straight line VL also intersects with the passenger PA (i.e., a position sandwiched between the target object TA and the passenger PA). At this position, the front surface FS of the image display device 150 is approximately perpendicular to the line of sight of the passenger PA looking at the target object TA, improving the visibility of the image displayed on the image display device 150 by the passenger PA.

[0039] Further, the image display control unit 130 causes the image display device 150 to display an image indicating the specification information I3 based on the specification information I3 of the target object TA acquired in S140 (S160). FIG. 4 shows a state in which an image indicating the specifications (appearance, name, size, etc.) of the first target object TA1 is displayed on the image display device 150 as the specification information I3 of the first target object TA1. Note that a part of the specification information I3 may be output as sound from a speaker (not shown) provided in the image display device 150. By displaying the image indicating the specification information I3 on ​​the image display device 150, the passenger PA can visually recognize the actual appearance of the first target object TA1 and the contents of the specification information I3 of the first target object TA1 displayed on the image display device 150 without moving his / her line of sight.

[0040] In addition, in the above S130, if it is determined that there is no object (tourist attraction) that can be introduced in the vicinity of the vehicle VH (S130: NO), the processes of S140 to S160 are skipped.

[0041] Thereafter, the image display control unit 130 determines whether or not the destination has been reached (S200), and if it is determined that the destination has not been reached yet (S200: NO), the above-mentioned processes from S110 onwards are repeated.

[0042] In addition, the position and orientation of the vehicle VH change from moment to moment as the vehicle VH travels. Therefore, if the relative position and orientation of the image display device 150 with respect to the vehicle VH is fixed, the absolute position and orientation of the image display device 150 with respect to features such as roads also change from moment to moment in response to the change in the position and orientation of the vehicle VH as the vehicle VH travels. Therefore, even after the target object TA is set in S140 and the image display device 150 is changed in S150 to point to the target object TA, the relative position and orientation of the image display device 150 with respect to the vehicle VH are adjusted in response to the change in the position and orientation of the vehicle VH as the vehicle VH travels so that the image display device 150 continues to point to the target object TA.

[0043] Furthermore, tourist attractions that can be seen from the window of the vehicle VH also change from moment to moment as the vehicle VH travels. Therefore, after the target object TA is set in S140, the orientation of the image display device 150 is changed in S150, and an image showing the specification information I3 of the target object TA is displayed on the image display device 150 in S160, if the target object TA cannot be seen from the window due to changes in the position and orientation of the vehicle VH as the vehicle VH travels, the processing from S130 onwards is executed again.

[0044] Furthermore, if it is determined in S130 that there are multiple objects (tourist attractions) that can be introduced around the vehicle VH, one tourist attraction may be set as the target object TA in S140, and thereafter, for example, the target object TA may be switched every time a predetermined time elapses or every time an instruction is given by the passenger PA. That is, another tourist attraction may be set as a new target object TA in S140, the orientation of the image display device 150 may be changed to point to the new target object TA in S150, and an image showing the specification information I3 of the new target object TA may be displayed on the image display device 150 in S160.

[0045] When the orientation of the image display device 150 is manually changed during the image display control process, it is determined in the above S120 that the orientation of the image display device 150 has been manually changed (S120: YES). FIG. 5 shows a state in which the passenger PA grasps the image display device 150 and changes the orientation (and position) of the image display device 150 to an orientation (and position) such that the image display device 150 points to a second target object TA2 during the image display control process. When it is determined that the orientation of the image display device 150 has been manually changed (S120: YES), the target setting unit 134 of the image display control unit 130 determines whether or not there is an object (tourist attraction) that can be introduced in the direction pointed to by the image display device 150 (S170). More specifically, as described above, the tourist attraction position information I2 is stored in the infotainment database ID, and this tourist attraction position information I2 is acquired by the spatial information acquisition unit 132. The target setting unit 134 calculates the above-mentioned virtual straight line VL (a virtual straight line that passes through a point on the front surface FS of the image display device 150 and is perpendicular to the front surface FS) from the position of the vehicle VH specified by the vehicle position information I1 and the orientation of the image display device 150 detected by the orientation control unit 142 of the support control unit 140, and judges whether or not there is a tourist attraction located on the virtual straight line VL (or a position whose distance from the virtual straight line VL is equal to or less than a predetermined threshold value). Note that it is also possible to set a correspondence relationship between each combination of the position on the road or parking lot and the position and orientation of the image display device 150, and the tourist attraction, store the correspondence relationship in the infotainment database ID, and have the target setting unit 134 make the judgment in the above S170 by referring to the correspondence relationship.

[0046] In the above S170, if it is determined that there is an object (tourist attraction) that can be introduced in the direction indicated by the image display device 150 (S170: YES), the target setting unit 134 sets one of such tourist attractions as the target object TA, and the target information acquisition unit 136 acquires specification information I3 of the target object TA from the infotainment database ID (S180). The specification information I3 of the target object TA is an image showing the specifications of the target object TA (appearance, name, size, year of construction, etc.).

[0047] Further, the image display control unit 130 causes the image display device 150 to display an image indicating the specification information I3 of the target object TA (S190). Fig. 5 shows a state in which an image indicating the specifications (appearance, name, size, etc.) of the second target object TA2 is displayed as the specification information I3 of the second target object TA2 on the image display device 150. By displaying the image indicating the specification information I3 on ​​the image display device 150, the passenger PA can visually recognize the appearance of the second target object TA2 that he or she wishes to see and the contents of the specification information I3 of the second target object TA2 displayed on the image display device 150, without moving his or her line of sight.

[0048] In this embodiment, when the orientation (and position) of the image display device 150 is manually changed, the image display control unit 130 causes the image display device 150 to display an image captured by the camera 152 included in the image display device 150. Therefore, the passenger PA can confirm an object located in the direction indicated by the image display device 150 by referring to the image displayed on the image display device 150, and can change the position and orientation of the image display device 150 so that the image display device 150 accurately points to the desired object.

[0049] In addition, if it is determined in the above S170 that there is no object (tourist attraction) that can be introduced in the vicinity of the vehicle VH (S170: NO), the processes of S180 to S190 are skipped. Alternatively, the passenger PA may be notified that there is no object (tourist attraction) that can be introduced in the vicinity of the vehicle VH, and prompted to manually change the orientation (and position) of the image display device 150 (S120) again.

[0050] The above-mentioned various processes are repeatedly executed, and when it is determined in S200 that the destination has been reached (S200: YES), the image display control unit 130 ends the image display control process.

[0051] A-4. Advantages of this embodiment: As described above, the image display system 10 of the present embodiment includes the image display device 150, the support device 160 that supports the image display device 150 and can change the orientation of the image display device 150, and the control device 100 that controls the image display device 150 and the support device 160. The control device 100 includes a spatial information acquisition unit 132, an orientation control unit 142, and an image display control unit 130. The spatial information acquisition unit 132 acquires spatial information including vehicle position information I1 indicating the position of the vehicle VH (i.e., the position of the image display device 150) and tourist attraction position information I2 indicating the positions of objects around the image display device 150. The orientation control unit 142 causes the support device 160 to change the orientation of the image display device 150 based on the spatial information. The image display control unit 130 causes the image display device 150 to display an image according to the orientation of the image display device 150.

[0052] In this way, in the image display system 10 of this embodiment, the orientation of the image display device 150 is changed based on spatial information including the vehicle position information I1 indicating the position of the image display device 150 and tourist attraction position information I2 indicating the positions of objects around the image display device 150, and an image corresponding to the orientation of the image display device 150 is displayed on the image display device 150. Therefore, according to the image display system 10 of this embodiment, it is possible to indicate an accurate direction to the passenger PA (user) by the orientation of the image display device 150 without using a mounting-side image display device, and to display an image corresponding to the direction on the image display device 150, thereby improving the convenience of the image display system 10.

[0053] Moreover, in the image display system 10 of this embodiment, the control device 100 further has a target setting unit 134 that sets one of the surrounding objects as a target object TA based on the above spatial information, and the orientation control unit 142 causes the support device 160 to change the orientation of the image display device 150 so that the image display device 150 points to the target object TA. Therefore, according to the image display system 10 of this embodiment, the direction of the passenger PA's line of sight can be guided to the target object TA, which is one of the surrounding objects, depending on the orientation of the image display device 150, and the convenience of the image display system 10 can be further improved.

[0054] Moreover, in the image display system 10 of this embodiment, the control device 100 further includes a target information acquisition unit 136 that acquires specification information I3, which is information related to the target object TA, and the image display control unit 130 causes an image indicating the specification information I3 to be displayed on the image display device 150. Therefore, according to the image display system 10 of this embodiment, the direction of the line of sight of the passenger PA can be guided to the target object TA, which is one of the surrounding objects, depending on the orientation of the image display device 150, and an image indicating the specification information I3 related to the target object TA can be displayed on the image display device 150, thereby further improving the convenience of the image display system 10.

[0055] In the image display system 10 of the present embodiment, the specification information I3 of the target object TA includes information indicating at least one of the name, size, and construction year of the target object TA. Therefore, according to the image display system 10 of the present embodiment, an image indicating information indicating at least one of the name, size, and construction year of the target object TA can be displayed on the image display device 150, and useful information can be presented to the passenger PA.

[0056] In addition, in the image display system 10 of this embodiment, the support device 160 supports the image display device 150 so that the orientation of the image display device 150 can be changed manually, and when the orientation of the image display device 150 is changed manually, the target setting unit 134 of the image display control unit 130 sets an object located in the direction pointed to by the image display device 150 after the orientation is changed as a target object TA. That is, in the image display system 10 of this embodiment, the image display device 150 functions as a 3D pointing device. Therefore, according to the image display system 10 of this embodiment, when the passenger PA points the image display device 150 in the direction of a desired object, the object is set as the target object TA, so that the target object TA can be set by an intuitive operation of the passenger PA, and the convenience of the image display system 10 can be further improved.

[0057] Moreover, in the image display system 10 of this embodiment, the image display device 150 has a camera 152 that captures an image of a subject located in a direction pointed by the image display device 150, and the image display control unit 130 causes the image display device 150 to display an image captured by the camera 152 when the orientation of the image display device 150 is manually changed. Therefore, according to the image display system 10 of this embodiment, the current orientation of the image display device 150 can be fed back to the passenger PA by the image displayed on the image display device 150, so that the passenger PA can easily manually change the orientation of the image display device 150 so that the image display device 150 accurately points to a desired object, and the convenience of the image display system 10 can be further improved.

[0058] In the image display system 10 of this embodiment, the image display device 150 has a planar front surface (image display surface) FS, and the direction indicated by the image display device 150 is perpendicular to the front surface FS and faces the opposite side (back surface BS side) to the direction in which the front surface FS faces. Therefore, according to the image display system 10 of this embodiment, the passenger PA can more accurately grasp the direction indicated by the image display device 150, and the convenience of the image display system 10 can be further improved.

[0059] Moreover, in the image display system 10 of the present embodiment, the support device 160 has three or more rotation axes. Therefore, according to the image display system 10 of the present embodiment, the orientation of the image display device 150 can be adjusted more flexibly, and the convenience of the image display system 10 can be further improved.

[0060] Moreover, the image display system 10 of this embodiment is mounted on an autonomous vehicle VH, the support device 160 supports the image display device 150 on the vehicle VH, and the spatial information acquisition unit 132 acquires the spatial information obtained by the sensor 102 installed in the vehicle VH. Therefore, according to the image display system 10 of this embodiment, by changing the orientation of the image display device 150 using information used for autonomous driving, it is possible to display an image corresponding to the direction on the image display device 150 while indicating an accurate direction related to the traveling of the vehicle VH to the passenger PA of the autonomous vehicle VH and people outside the vehicle, and the convenience of the image display system 10 can be improved.

[0061] B. Variations: The technology disclosed in this specification is not limited to the above-described embodiments, and can be modified in various forms without departing from the spirit of the invention. For example, the following modifications are also possible.

[0062] The configuration of the image display system 10 in the above embodiment is merely an example, and various modifications are possible. For example, in the above embodiment, the image display device 150 is supported on the dashboard of the vehicle VH by the support device 160, but as in the modified example shown in Fig. 6, the image display device 150 may be supported on the ceiling or floor of the vehicle VH by the support device 160. Also, in the above embodiment, the support device 160 is configured to allow manual change of the position and orientation of the image display device 150, but it is not necessarily required to be configured to allow manual change of the position and orientation of the image display device 150.

[0063] In the above embodiment, the support device 160 is configured by a robot arm having multiple joints and can change the position and orientation of the image display device 150, but the support device 160 may be configured by other devices (for example, a gimbal) as long as it can change the orientation of the image display device 150 while supporting the image display device 150. Note that if the support device 160 has three or more axes, it is preferable because, for example, the pan, tilt, and roll of the image display device 150 can be adjusted along each axis, and the orientation of the image display device 150 can be adjusted more flexibly.

[0064] In the above embodiment, in the initial state, the image display device 150 is located in front of the passenger PA, and the front surface FS of the image display device 150 is oriented so as to directly face the passenger PA, but the position and orientation of the image display device 150 in the initial state are not limited to this. For example, in the initial state, the front surface FS of the image display device 150 may be oriented upward to form a table shape, making it easy for one or more passengers PA to view the screen. Alternatively, in the initial state, the orientation of the image display device 150 may be such that it is easy for the passenger PA to perform touch input operations on the screen.

[0065] In addition, in the above embodiment, a case where one image display device 150 is used by one passenger PA (user) has been exemplified, but the technology disclosed in this specification is similarly applicable to a case where one image display device 150 is shared by multiple users. That is, since the orientation of one image display device 150 can be recognized almost accurately by each of the multiple users, it is possible to indicate an almost accurate direction to the multiple users depending on the orientation of the image display device, and to display an image corresponding to the direction on the image display device.

[0066] In the above embodiment, an in-vehicle image display device is used as the image display device 150, but other image display devices such as a smartphone or tablet terminal brought by the passenger PA may be used as the image display device 150. In such a case, for example, a holder for holding the image display device is provided in the support device 160, and the image display device is held in the holder, thereby supporting the image display device and changing its position and orientation. In addition, a projection type image display device such as a projector may be used as the image display device 150.

[0067] In the above embodiments, the control device 100 may be configured to be divided into a plurality of devices. For example, the vehicle VH may be provided with a control device for automatic driving control, a control device for image display control, and a control device for support control, which are independent of each other. In the above embodiments, a part of the configuration realized by hardware may be replaced with software, and conversely, a part of the configuration realized by software may be replaced with hardware.

[0068] Further, the contents of the image display control process in the above embodiment are merely examples and can be modified in various ways. For example, in the image display control process in the above embodiment, the spatial information acquisition unit 132 acquires, as spatial information, information on the position and orientation of the vehicle VH detected based on signals from various sensors 102 installed in the vehicle VH, but the spatial information acquisition unit 132 may acquire, as spatial information, information on the position and orientation of the image display device 150 detected based on signals from sensors other than the sensor 102 installed in the vehicle VH (for example, a sensor installed in the image display device 150). Note that the spatial information may not include information indicating the orientation of the vehicle VH or the image display device 150.

[0069] In addition, although the image display control process in this embodiment is a process of presenting information introducing tourist attractions visible from the vehicle window as infotainment to the passenger PA of the vehicle VH traveling toward a destination by automatic driving, the technology disclosed in this specification can be applied to other image display control processes. For example, the technology disclosed in this specification can be applied to image display control processes for providing other infotainment (games, videos, etc.). In addition, the technology disclosed in this specification can be applied to image display control processes in which, for example, the spatial information acquisition unit 132 acquires spatial information, the orientation control unit 142 changes the orientation of the image display device 150 so that the front surface FS of the image display device 150 is directed toward surrounding objects, and the image display control unit 130 displays an image to be shown to surrounding objects (for example, an image that calls attention to the progress of the vehicle VH, etc.) on the image display device 150.

[0070] In addition, in the above embodiment, an example of an image display system 10 installed in an autonomously driven vehicle VH has been described, but the technology disclosed in this specification is similarly applicable to image display systems used for other purposes (e.g., an image display system installed in a vehicle that is not autonomously driven, an image display system installed on a moving body other than a vehicle, an image display system installed on a fixed object (e.g., a building), a portable image display system, etc.). [Explanation of symbols]

[0071] 10: Image display system 100: Control device 102: Sensor 110: Control unit 120: Automatic driving control unit 122: Position / orientation detection unit 124: Surrounding situation detection unit 126: Route setting unit 130: Image display control unit 132: Spatial information acquisition unit 134: Target setting unit 136: Target information acquisition unit 140: Support control unit 142: Orientation control unit 150: Image display device 152: Camera 160: Support device 162: Drive motor 170: Memory unit 172: Automatic driving control program 174: Image display control program 176: Support control program 190: Bus BS: Rear FS: Front I1: Vehicle position information I2: Tourist attraction position information I3: Specifications information ID: Infotainment database MD: 3D map database PA: Passenger TA: Target object VH: Vehicle VL: Virtual line

Claims

1. 1. An image display system, comprising: An image display device; a support device that supports the image display device and is capable of changing the orientation of the image display device; A control device that controls the image display device and the support device; Equipped with The control device includes: a spatial information acquisition unit that acquires spatial information including first position information indicating a position of the image display device and second position information indicating positions of objects in the periphery of the image display device identified based on the first position information; an orientation control unit that causes the support device to change an orientation of the image display device based on the spatial information; an image display control unit that causes the image display device to display an image corresponding to the orientation of the image display device; having Image display system.

2. 2. The image display system according to claim 1, The control device further includes a target setting unit that sets one of the surrounding objects as a target object based on the spatial information, the orientation control unit causes the support device to change the orientation of the image display device so that the image display device is oriented to point at the target object; Image display system.

3. 3. The image display system according to claim 2, The control device further includes a target information acquisition unit that acquires target information that is information related to the target object, The image display control unit causes the image display device to display an image indicating the target information. Image display system.

4. 4. The image display system according to claim 3, The target information includes information indicating at least one of the name, size, and construction year of the target object. Image display system.

5. The image display system according to any one of claims 2 to 4, the support device supports the image display device so that the orientation of the image display device can be manually changed; the target setting unit sets an object located in a direction indicated by the image display device as the target object when the orientation of the image display device is manually changed; Image display system.

6. 6. The image display system according to claim 5, the image display device has a camera that captures an image of a subject located in a direction indicated by the image display device, the image display control unit causes the image display device to display an image captured by the camera when a direction of the image display device is manually changed; Image display system.

7. The image display system according to any one of claims 2 to 6, the image display device has a flat image display surface, The direction in which the image display device points is perpendicular to the image display surface and faces the opposite side to the image display surface. Image display system.

8. The image display system according to any one of claims 1 to 7, The support device has three or more rotation axes. Image display system.

9. The image display system according to any one of claims 1 to 8, The image display system is mounted on an autonomous vehicle, the support device supports the image display device on the vehicle; The spatial information acquisition unit acquires the spatial information obtained by a sensor installed in the vehicle. Image display system.

10. 1. An image display method for displaying an image using an image display device and a support device that supports the image display device and is capable of changing an orientation of the image display device, comprising: acquiring spatial information including first position information indicating a position of the image display device and second position information indicating positions of objects around the image display device identified based on the first position information; causing the support device to change the orientation of the image display device based on the spatial information; displaying an image on the image display device according to the orientation of the image display device; Equipped with Image display method.

11. A computer program for displaying an image using an image display device and a support device that supports the image display device and is capable of changing an orientation of the image display device, the computer program comprising: On the computer, A process of acquiring spatial information including first position information indicating a position of the image display device and second position information indicating positions of objects around the image display device identified based on the first position information; A process of causing the support device to change the orientation of the image display device based on the spatial information; A process of displaying an image on the image display device according to a direction of the image display device; Execute the Computer program.

Images