Display control device, display control method, and display control program

Abstract

To provide a display control device capable of immediately and easily visibly displaying information on a feature at a position visually recognizable from inside of a vehicle.SOLUTION: A display control device is configured to: identify a building BD that is visible through a windshield GL of a vehicle; acquire building-related information IF on the identified building BD; and cause the windshield GL to display a mark MK that is visible through the windshield GL and indicates the identified building BD, the acquired building-related information IF, and an indication line L connecting the feature mark MK and the building-related information IF. In this case, the mark MK is displayed so that the mark MK and the building BD are visually recognized to overlap each other on a line of sight from inside of the vehicle.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] This application belongs to the technical field of display control devices, display control methods, and display control programs. More specifically, it belongs to the technical field of display control devices and display control methods for controlling the display of information on the windows of moving objects such as vehicles, as well as display control systems and programs for such display control.

Background Art

[0002] In recent years, various researches / developments regarding the provision of information to passengers during vehicle operation have been conducted. As a prior art document disclosing such prior art, for example, Patent Document 1 below can be cited. This Patent Document 1 describes a technique for detecting gestures such as the movement of a passenger's hand and displaying information on a ground object such as a building outside the vehicle window corresponding to the gesture on a display inside the vehicle. Recently, with the development of vehicle automatic driving technology, a technique has also been developed to project the above-mentioned information onto the inside of the vehicle near the front windshield using so-called Augmented Reality (AR) technology in correspondence with the scenery seen from the front windshield.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the technique disclosed in the above Patent Document 1 and other prior arts, in a vehicle moving in automatic driving, the immediacy and visibility in presenting information on ground objects located at positions visible from inside the vehicle have not been considered.

[0005] In contrast, in environments where the effort and burden on passengers related to driving itself are reduced, such as when riding in an autonomous vehicle, being able to instantly and clearly display information about visible landmarks from inside the vehicle during the journey would contribute to improving communication among passengers, for example.

[0006] Therefore, this application was made in view of the above requirements, and one example of the problem it addresses is to provide a display control device, a display control method, and a program for said display control that can immediately and clearly display information about landmarks located in a position visible from inside a vehicle to the passengers. [Means for solving the problem]

[0007] To solve the above problems, the invention described in claim 1 comprises: identification means for identifying any of the features visible through a transparent member of a window of a moving body; feature information acquisition means for acquiring feature information about the identified feature; and control means for displaying on the transparent member a feature mark visible through the transparent member and indicating the identified feature, the acquired feature information, and an indicator line connecting the feature mark and the feature information. The control means is configured to display the feature mark on the transparent member such that the feature mark and the identified feature overlap and are visible from the line of sight from inside the moving body, and to stop displaying the feature information and the indicator line after the recording instruction is given when the recording instruction is given.

[0008] To solve the above problems, the invention described in claim 8 is a display control method performed in a display control device comprising a identifying means, a feature information acquisition means, and a control means, comprising: an identifying step of identifying any of the features visible through a transparent member of a window of a moving body using the identifying means; a feature information acquisition step of acquiring feature information about the identified feature using the feature information acquisition means; and a control step of causing the control means to display on the transparent member a feature mark visible through the transparent member and indicating the identified feature, the acquired feature information, and an indicator line connecting the feature mark and the feature information, wherein the control means is configured to display the feature mark on the transparent member such that the feature mark and the identified feature overlap and are visible from the line of sight from inside the moving body, and to stop displaying the feature information and the indicator line after the recording instruction when the recording instruction is given.

[0009] To solve the above problems, the invention described in claim 9 is a display control program that causes a computer to function as a means for identifying any of the features visible through a transparent member of a window of a moving body, a means for acquiring feature information about the identified feature, and a control means for causing a feature mark visible through the transparent member and indicating the identified feature, the acquired feature information, and an indicator line connecting the feature mark and the feature information to be displayed on the transparent member, wherein the computer functioning as the control means is configured to display the feature mark on the transparent member such that the feature mark and the identified feature overlap and are visible from the line of sight from inside the moving body, and to stop displaying the feature information and the indicator line after the recording instruction is given when the recording of the feature information is given. [Brief explanation of the drawing]

[0010] [Figure 1] This block diagram shows the outline configuration of the display control device of the embodiment. [Figure 2] This is a block diagram showing the configuration of the information display system in the embodiment. [Figure 3] This is a flowchart illustrating the display control process of an embodiment, where (a) is a flowchart showing the entire display control process, and (b) is a flowchart showing the information display process included in the display control process. [Figure 4] This figure illustrates the information display in the embodiment, where (a) is a figure showing the first example of the information display, and (b) is a figure showing the second example of the information display. [Modes for carrying out the invention]

[0011] Next, an embodiment for carrying out the present invention will be described with reference to Figure 1. Figure 1 is a block diagram showing the schematic configuration of the display control device of the embodiment.

[0012] As shown in Figure 1, the display control device S of this embodiment is configured to include a selection means 4, a feature information acquisition means 7, and a control means 1.

[0013] At this time, the identification means 4 identifies one of the features visible through the transparent member of the window of the moving body.

[0014] Meanwhile, the feature information acquisition means 7 acquires feature information about the features identified by the identification means 4.

[0015] The control means 1 then causes the transparent member to display a feature mark indicating a feature that is visible through the transparent member and identified by the identification means 4, feature information acquired by the feature information acquisition means 7, and an indicator line connecting the feature mark and the feature information. At this time, the control means 1 displays the feature mark on the transparent member so that the feature mark and the feature identified by the identification means 4 appear to overlap and be visible from the line of sight from inside the moving vehicle.

[0016] As described above, according to the configuration of the display control device S of the embodiment, when displaying a feature mark indicating a feature visible through the transparent member of the window of the mobile body, feature information about the feature, and an indicator line connecting the feature mark and the feature information on the transparent member of the window of the mobile body, the feature mark is displayed on the transparent member in such a way that the feature mark and the feature indicated by it overlap and are visible from the line of sight from inside the mobile body. Therefore, information about features that are visible from inside the mobile body can be displayed immediately and in an easy-to-see manner. [Examples]

[0017] Next, specific embodiments corresponding to the embodiments described above will be explained with reference to Figures 2 to 4. The embodiments described below are, for example, examples in which the present invention is applied to an information display system that displays various information inside a vehicle moving using autonomous driving technology.

[0018] Furthermore, Figure 2 is a block diagram showing the configuration of the information display system in the embodiment, Figure 3 is a flowchart showing the display control process in the embodiment, and Figure 4 is a diagram illustrating the information display in the embodiment. In Figures 2 and 4, the same component numbers as those used for each component in the embodiment corresponding to each component in the display control device S of the embodiment shown in Figure 1 are used for each component in the embodiment.

[0019] As shown in Figure 2, the information display system SS of the embodiment is installed in, for example, a vehicle that is mobile using autonomous driving technology. The information display system SS of the embodiment consists of a control unit 1 consisting of a CPU (Central Processing System), ROM (Read Only Memory), and RAM (Random Access Memory), a recording unit 2 consisting of an HDD (Hard Disk Drive) or SSD (Solid State Drive), a GPS (Global Positioning System) receiving unit 3, an motion detection unit 4, a display unit 6, and an external communication unit 7. These control unit 1, recording unit 2, GPS receiving unit 3, motion detection unit 4, display unit 6, and external communication unit 7 are connected to each other via a bus 5 so that data can be exchanged between them. At this time, the control unit 1 may be implemented by hardware logic circuits such as the CPU, or it may be implemented in software by the CPU reading and executing a program corresponding to the display control processing of the embodiment described later. Furthermore, the motion detection unit 4 corresponds to an example of the identification means 4 of the embodiment, the external communication unit 7 corresponds to an example of the feature information acquisition means 7 of the embodiment and an example of the "identification information acquisition means" of the present application, and the control unit 1 corresponds to an example of the control means 1 of the embodiment.The control unit 1, motion detection unit 4, and external communication unit 7 constitute an example of the display control device S of the embodiment.Furthermore, the control unit 1 corresponds to an example of the "map data acquisition means" of the present application, the GPS receiver unit 3 corresponds to an example of the "location data acquisition means" and an example of the "current location data acquisition means" of the present application, and the motion detection unit 4 corresponds to an example of the "gaze data acquisition means" and an example of the "contact location data acquisition means" of the present application, respectively.

[0020] In the above configuration, the GPS receiver unit 3 generates current position data indicating the current position of the vehicle on which the display control system SS of the embodiment is mounted based on navigation radio waves received from navigation satellites (not shown) included in GPS under the control of the control unit 1, and outputs the generated current position data to the control unit 1. This current position is, for example, the current position of the vehicle represented using latitude, longitude, and altitude. On the other hand, map data MD indicating at least features such as buildings and facilities on the moving route of the vehicle is recorded in the recording unit 2 in a non-volatile manner and is output to the control unit 1 as necessary.

[0021] Also, the external communication unit 7 is capable of wirelessly exchanging data with the outside of the vehicle under the control of the control unit 1. Further, the motion detection unit 4 detects, as a specific motion described later, for example, the direction of the line of sight or the gesture mode by hand of the vehicle occupant by the same method as in the prior art, such as using the imaging result of an in-vehicle camera (not shown), under the control of the control unit 1, and outputs the detection result to the control unit 1. Also, the display unit 6 displays various information on the glass of the vehicle by a technique such as projection. At this time, the glass of the vehicle includes the front glass, side glass, rear glass, and the glass of the sunroof portion. Thus, the control unit 1 executes the display control process of the embodiment by the display unit 6 using the current position data from the GPS receiver unit 3, the data from the external communication unit 7, the detection result by the motion detection unit 4, and the map data MD.

[0022] Next, the display control process of the embodiment executed centering on the control unit 1 will be specifically described using FIGS. 2 to 4.

[0023] First, the overall display control process of the embodiment will be explained using Figure 3(a). This display control process is executed, for example, as an interrupt process at predetermined intervals while the autonomous vehicle equipped with the display control system SS of the embodiment is in motion. That is, as the display control process of the embodiment, first the control unit 1 uses the motion detection unit 4 to detect the gaze and direction of the occupant (meaning the driver of the vehicle or another occupant; the same applies hereinafter) (step S1). Next, the control unit 1 determines, based on the detection result of the motion detection unit 4, whether the occupant has performed a specific action, such as looking at and identifying a building outside the vehicle that is visible through the windshield at that time (step S2). If the specific action (looking action) is not detected in the determination in step S2 (step S2: NO), the control unit 1 proceeds to step S1 in order to continue detecting the occupant's gaze. On the other hand, if the determination in step S2 detects the identification action of looking at and identifying any building (step S2: YES), the control unit 1 acquires various information about the building identified in the determination in step S2 (i.e., the building located in the direction from which the gaze detected as an identification action is directed (i.e., the building being looked at)) via the external communication unit 7 (step S3). In the following description, the various information acquired in step S3 will simply be referred to as "building-related information".

[0024] At this time, in step S3, the control unit 1 identifies the building that the passenger is looking at, based on, for example, the current location data from the GPS receiver 3, the detection result of the passenger's gaze by the motion detection unit 4, and the map data MD. The building-related information also includes, for example, name information indicating the name of the building identified in step S2, location information indicating the location where it exists, feature information indicating the external features of the building, or information about an exhibition hall where information related to the building is displayed.

[0025] In step S3, once building-related information about an external building identified using the passenger's gaze detection results, etc., has been acquired, the control unit 1 then displays the acquired building-related information on the vehicle's windshield, for example, using the display unit 6 (step S4). Step S4 will be explained in detail later.

[0026] Next, the control unit 1 determines whether or not to terminate the display control process of the embodiment for reasons such as the vehicle arriving at its destination (step S5). If the determination in step S5 is to terminate the display control process of the embodiment (step S5: YES), the control unit 1 terminates the display control process. On the other hand, if the determination in step S5 is to continue the display control process of the embodiment (step S5: NO), the control unit 1 returns to step S1 and continues the display control process.

[0027] Next, the display of building-related information in step S4 will be explained. Here, in the display of building-related information in step S4, it is assumed that when the scenery outside the vehicle, as illustrated in Figure 4(a), is visible from inside the vehicle through the windshield GL, the occupant's gaze is directed toward building BD, and building BD is identified based on the detection results of the direction of the gaze, etc. (see Figure 3(a) step S2: YES), and further, building-related information about building BD is also acquired (see Figure 3(a) step S3). In this case, the control unit 1 displays a marker MK on the display unit 6 at the position on the windshield GL where the detected direction of the gaze intersects with the windshield GL (in other words, the position on the windshield GL that the occupant with that gaze would see when they are intently watching building BD (i.e., intently watching through the windshield GL)) to indicate that building BD is being watched (step S40). In addition, the control unit 1 displays the building-related information IF acquired in step S3 for the building BD using the display unit 6, at a location on the windshield GL different from the display position of the marker MK (for example, above the front of the windshield GL; see Figure 4(a)), using a distinctive and conspicuous design (step S40). Furthermore, the control unit 1 also displays the indicator line L connecting the mark MK and the building-related information IF using the display unit 6. At this time, the building-related information IF is displayed as the content of the building-related information acquired in step S3, for example, as a graphic, text, or video.

[0028] Meanwhile, the vehicle is moving even while the mark MK and other indicators in step S40 are being displayed. Therefore, as illustrated in Figure 4(a), the building BD, which was visible through the windshield GL, moves relatively to the right in Figure 4(a) as the vehicle moves, and becomes visible only through the rear window via the right-hand side window. Consequently, when the position of building BD moves out of the passenger's field of view through the windshield GL as the vehicle moves, the control unit 1, based on the current position data and the map data MD, continues to display the indicator line L in the direction that building BD has moved from the perspective of someone inside the vehicle (to the right in the example in Figure 4), as illustrated in Figure 4(b), and also continues to display the building-related information IF on the windshield GL (step S40). As long as the passenger continues to see building BD, the control unit 1 continues to display the mark MK. In this case, the indicator line L and the mark MK do not need to be connected (see Figure 4(b)). Furthermore, from the time the display shown in the example in Figure 4(b) is started, the control unit 1 displays a recording icon R, for example, in the lower center of the windshield GL, which instructs the recording unit 2 to record the contents of the building-related information IF non-volatilely.

[0029] While the display of building-related information IF, etc., as illustrated in Figure 4(b) continues, the control unit 1 monitors whether, as the vehicle moves further, the building BD for which the building-related information IF was displayed on the windshield GL is no longer visible from the entire field of view inside the vehicle (step S41). In the monitoring in step S41, as long as the building BD is visible from inside the vehicle (step S41: NO), the control unit 1 continues to display the building-related information IF and indicator line L illustrated in Figure 4(b), as well as the mark MK, which is not shown in Figure 4(b) as the building BD has moved to the right rear. On the other hand, in the monitoring in step S41, if the building BD is no longer visible from inside the vehicle (step S41: YES), the control unit 1 stops displaying the mark MK that indicated the building BD, and continues to display the building-related information IF and indicator line L in the state illustrated in Figure 4(b) (step S42).

[0030] While the display of building-related information IF and indicator line L in step S42 continues, the control unit 1 monitors whether the record icon R displayed on the windshield GL, as illustrated in Figure 4(b), has been operated by the occupant (step S43). The operation in step S43 may be, for example, a gesture by the occupant detected by the motion detection unit 4, or the motion detection unit 4 may detect that the occupant has touched a touch sensor (not shown) provided on the inner surface of the windshield GL.

[0031] If no operation on the recording icon R is detected during monitoring in step S43 (step S43: NO), the control unit 1 continues to display the information on the display unit 6 as illustrated in Figure 4(b). On the other hand, if an operation on the recording icon R is detected during monitoring in step S43 (step S43: YES), the control unit 1 records the contents of the building-related information IF displayed on the windshield GL at that time, for example, in the recording unit 2, and erases the building-related information IF and indicator line L that were displayed on the windshield GL up to that point (step S44). Note that the recording of the contents of the building-related information IF in step S44 may be done not only by recording it in the recording unit 2, but also, for example, by recording it on an external server device via the external communication unit 7. Once the recording of the contents of the building-related information IF is complete, the control unit 1 proceeds to step S5 shown in Figure 3(a).

[0032] As described above, according to the display control processing of the embodiment, when displaying a mark MK indicating a building BD visible through the windshield GL of an autonomously driven vehicle, building-related information IF for the building BD, and an indicator line L connecting the mark MK and the building-related information IF on the windshield GL, the mark MK is displayed on the windshield GL so that it overlaps with the building BD in the line of sight from inside the vehicle. Therefore, information about a building BD that is visible from inside the vehicle can be displayed immediately and in an easy-to-read manner.

[0033] Furthermore, the display of mark MK stops when the identified building BD becomes invisible from inside the vehicle (see step S42 in Figure 3(b)), thus ensuring visibility after the identified building BD is no longer visible.

[0034] Furthermore, even after the identified building BD becomes invisible, the indicator line L and the display of building-related information IF continue to show the direction in which the building BD became invisible from the perspective of the vehicle (see Figure 4(b)). Therefore, even if the building BD becomes invisible, its information can be reliably recognized.

[0035] Furthermore, based on location data indicating the vehicle's current position, line-of-sight data indicating the direction of the view from inside the vehicle, and map data MD equivalent to a map including the vehicle's current position, the system identifies the building BD that lies in the line of sight. This allows information about the building BD being viewed from inside the vehicle to be displayed immediately and clearly on that line of sight.

[0036] Furthermore, when recording of building-related information IF is instructed, the display of building-related information IF and indicator line L after the recording instruction is stopped (Figure 3(b) step S44), thereby widening the passenger's field of view by stopping the display of building-related information IF and indicator line L that are no longer needed due to recording.

[0037] In the above-described embodiment, a marker MK was displayed on the windshield GL based on the detection result of the passenger's gaze (see step S1 in Figure 3(a)). However, in addition to this, if the touch panel is provided on the inside of the windshield GL, the system may be configured to identify building BD based on current location data indicating the vehicle's current location, contact location data indicating the contact location on the touch panel, and map data MD which corresponds to a map including the vehicle's current location. In this case, information on building BD identified by contact with the touch panel (in other words, the windshield GL) can be displayed immediately and in an easily viewable manner.

[0038] Furthermore, in the above-described embodiment, the case in which the identification of building BD is performed by the passenger looking at building BD was explained. However, in addition to this, the system may be configured to recognize a passenger's statement (voice), such as "What's that?", and use that statement as a trigger to identify building BD based on the gaze of the passenger who made the statement. In this case, the process of detecting the gaze of passengers other than the passenger who made the statement can be omitted, which can have effects such as reducing the processing load on the control unit 1.

[0039] Furthermore, the building BD may be identified using external information acquired via the external communication unit 7. That is, the system may be configured to identify the building BD based on current location data indicating the vehicle's current location, the name of the building BD, and other external information for identifying the building BD, as well as map data MD corresponding to a map including the vehicle's current location. In this case, the information of the building BD identified by the external identification information can be displayed immediately and in an easy-to-understand manner.

[0040] Furthermore, while the above-described embodiment applied the present invention to display building-related information IF etc. on the windshield GL of a vehicle, the present invention can also be applied to display building-related information IF etc. on a transparent portion surrounding it, for example, on a motorcycle.

[0041] Furthermore, programs corresponding to the flowcharts shown in Figure 3 can be recorded on a recording medium such as an optical disc, or obtained via a network such as the Internet, and then read and executed by a general-purpose microcomputer, thereby allowing the microcomputer to function as the control unit 1 according to the embodiment. [Explanation of symbols]

[0042] 1. Control means (control unit) 4. Identification means (motion detection unit) 6 Display section 7. Means for acquiring geographic information (external communication unit) S Display Control Device L indicator line GL windshield MK Marker IF Building-related information BD Building

Claims

1. A means for identifying any of the features visible through the transparent element of the window of a moving object, A means for acquiring feature information about the identified feature, Control means for displaying on the transparent member a feature mark that is visible through the transparent member and indicates the identified feature, the acquired feature information, and an indicator line connecting the feature mark and the feature information, Equipped with, The control means is The feature mark is displayed on the transparent member such that the feature mark and the identified feature appear to overlap and be visible from the line of sight from the moving body. A display control device characterized by stopping the display of the feature information and the indicator line after the recording instruction is given when the recording of the feature information is instructed.

2. In the display control device according to claim 1, The control means is characterized by stopping the display of the feature mark when the identified feature becomes invisible from within the moving body.

3. In the display control device according to claim 1 or claim 2, The control means is characterized in that, after the identified feature becomes invisible from the moving body, the display control device continues to display the indicator line and the feature information such that the indicator line indicates the direction in which the feature became invisible from the moving body.

4. In the display control device according to any one of claims 1 to 3, A position data acquisition means for acquiring position data indicating the current position of the moving object, A means for acquiring gaze data that acquires gaze data indicating the direction of the gaze, Map data acquisition means for acquiring map data corresponding to a map including the current location, Furthermore, The display control device is characterized in that the identifying means identifies the feature located on the line of sight based on the acquired location data, line of sight data, and map data.

5. In the display control device according to claim 4, The display control device is characterized in that the identification means identifies the feature being viewed on the line of sight based on the acquired location data, line of sight data, and map data.

6. In the display control device according to any one of claims 1 to 3, A current location data acquisition means for acquiring current location data indicating the current position of the moving object, A contact position data acquisition means for acquiring contact position data indicating the contact position on the transparent member when the transparent member is contacted from the moving body, Map data acquisition means for acquiring map data corresponding to a map including the current location, Furthermore, The display control device is characterized in that the identification means identifies the feature based on the acquired current location data, contact location data, and map data.

7. In the display control device according to any one of claims 1 to 3, A position data acquisition means for acquiring position data indicating the current position of the moving object, A means for acquiring specific information to identify the aforementioned feature from outside the mobile body, Map data acquisition means for acquiring map data corresponding to a map including the current location, Furthermore, The display control device is characterized in that the identification means identifies the feature based on the acquired location data, identification information and map data.

8. A display control method performed in a display control device comprising a specific means, a means for acquiring feature information, and a control means, A process of identifying any of the features visible through the transparent member of the window of the moving body using the identification means, A feature information acquisition step in which feature information about the identified feature is acquired by the feature information acquisition means, A control step by which the control means causes the transparent member to display a feature mark that is visible through the transparent member and indicates the identified feature, the acquired feature information, and an indicator line connecting the feature mark and the feature information. Includes, In the control process described above, The feature mark is displayed on the transparent member such that the feature mark and the identified feature appear to overlap and be visible from the line of sight from the moving body. A display control method characterized by stopping the display of the feature information and the indicator line after the recording instruction is given when the recording of the feature information is instructed.

9. Computers, A means of identifying any of the geographical features visible through the transparent material of a window of a moving object. A means for obtaining feature information about the identified feature, and Control means for displaying on the transparent member a feature mark that is visible through the transparent member and indicates the identified feature, the acquired feature information, and an indicator line connecting the feature mark and the feature information. A display control program that functions as such, The computer, which functions as the control means, The feature mark is displayed on the transparent member such that the feature mark and the identified feature appear to overlap and be visible from the line of sight from the moving body. A display control program characterized by its function to stop the display of the feature information and the indicator line after the recording instruction is given when the recording of the feature information is instructed.

Images