Display control device, display control method, and display control program

Abstract

To provide a display control device capable of immediately and easily visibly displaying information on a feature at a position visually recognizable from inside of a vehicle.SOLUTION: A display control device is configured to: identify a building BD that is visible through a windshield GL of a vehicle; acquire building-related information IF on the identified building BD; and cause the windshield GL to display a mark MK that is visible through the windshield GL and indicates the identified building BD, the acquired building-related information IF, and an indication line L connecting the feature mark MK and the building-related information IF. In this case, the mark MK is displayed so that the mark MK and the building BD are visually recognized to overlap each other on a line of sight from inside of the vehicle.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present application relates to the technical fields of a display control device, a display control method, and a display control program. More specifically, the present application relates to the technical fields of a display control device, a display control method, a display control system, and a program for the display control that control the display of information on windows of a moving body such as a vehicle. [Background technology]

[0002] In recent years, various research and development efforts have been made regarding the provision of information to passengers driving a vehicle. Patent Document 1 listed below is an example of a prior art document disclosing such conventional technology. Patent Document 1 describes a technology that detects a gesture, such as a hand movement, of a passenger and displays information about a feature, such as a building outside the vehicle window, that corresponds to the gesture on an in-vehicle display. Furthermore, with the recent development of autonomous vehicle driving technology, a technology has also been developed that uses so-called augmented reality technology to display the above-mentioned information in the vehicle, for example, near the windshield, in correspondence with the view seen through the windshield. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2016-53880 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology disclosed in Patent Document 1 above and other prior art technologies do not take into consideration the improvement of immediacy and visibility in the presentation of information about features that are visible from inside a vehicle that is moving autonomously.

[0005] In contrast, in environments where the effort and burden on passengers in driving itself is reduced, particularly when passengers are riding in an autonomous vehicle, if information about features visible from inside the vehicle can be displayed instantly and easily while the vehicle is moving, it will help improve communication between passengers, for example.

[0006] Therefore, the present application has been made in consideration of the above-mentioned demand, and one example of its objective is to provide a display control device, a display control method, and a program for said display control that can instantly and easily display information about features located in positions visible from inside the vehicle to passengers. [Means for solving the problem]

[0007] In order to solve the above problem, the invention described in claim 1 comprises an identification means for identifying any of the features visible through a transparent member of a window of a moving body, a feature information acquisition means for acquiring feature information about the identified feature, and a control means for displaying, on the transparent member, a feature mark visible through the transparent member and indicating the identified feature, the acquired feature information, and an indication line connecting the feature mark and the feature information, wherein the control means displays the feature mark on the transparent member so that the feature mark and the identified feature are visible as overlapping when viewed from the line of sight from inside the moving body, and when an instruction to record the feature information is given, stops displaying the feature information and the indication line after the instruction to record is given.

[0008] In order to solve the above problem, the invention described in claim 8 is a display control method executed in a display control device equipped with an identification means, a feature information acquisition means, and a control means, and includes: an identification step of identifying any feature visible through a transparent member of a window of a moving body by the identification means; a feature information acquisition step of acquiring feature information about the identified feature by the feature information acquisition means; and a control step of causing the control means to display, on the transparent member, a feature mark visible through the transparent member and indicating the identified feature, the acquired feature information, and an indication line connecting the feature mark and the feature information, wherein the control step is configured to display the feature mark on the transparent member so that the feature mark and the identified feature are visible overlapping in the line of sight from inside the moving body, and when an instruction to record the feature information is given, to stop displaying the feature information and the indication line after the recording instruction.

[0009] In order to solve the above problem, the invention described in claim 9 is a display control program that causes a computer to function as an identification means that identifies any of the features visible through a transparent member of a window of a moving body, a feature information acquisition means that acquires feature information about the identified feature, and a control means that causes a feature mark that is visible through the transparent member and indicates the identified feature, the acquired feature information, and an indication line connecting the feature mark and the feature information to be displayed on the transparent member, wherein the computer functioning as the control means is caused to display the feature mark on the transparent member so that the feature mark and the identified feature are visible as overlapping in the line of sight from inside the moving body, and when an instruction to record the feature information is given, to stop displaying the feature information and the indication line after the recording instruction. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a schematic configuration of a display control device according to an embodiment; [Figure 2] 1 is a block diagram showing a configuration of an information display system according to an embodiment; [Figure 3] 1A is a flowchart showing the entire display control process, and FIG. 1B is a flowchart showing information display processing included in the display control process. [Figure 4] 1A and 1B are diagrams illustrating examples of information display according to an embodiment, in which FIG. 1A is a diagram showing a first example of the information display, and FIG. 1B is a diagram showing a second example of the information display. DETAILED DESCRIPTION OF THE INVENTION

[0011] Next, an embodiment of the present invention will be described with reference to Fig. 1. Fig. 1 is a block diagram showing a schematic configuration of a display control device according to the embodiment.

[0012] As shown in FIG. 1, the display control device S of the embodiment is configured to include a specifying means 4, a feature information acquiring means 7, and a control means 1.

[0013] At this time, the identification means 4 identifies any of the features visible through the transparent member of the window of the mobile body.

[0014] On the other hand, the feature information acquisition means 7 acquires feature information about the feature identified by the identification means 4.

[0015] Then, the control means 1 displays, on the transparent member, a feature mark that indicates the feature that is visible through the transparent member and that has been identified by the identification means 4, the feature information acquired by the feature information acquisition means 7, and an indication line that connects the feature mark and the feature information. At this time, the control means 1 displays the feature mark on the transparent member so that the feature mark and the feature identified by the identification means 4 are visible and overlapping in the line of sight from inside the moving body.

[0016] As described above, according to the configuration of the display control device S of the embodiment, when a feature mark indicating a feature visible through a transparent member of a window of a moving body, feature information about the feature, and an indication line connecting the feature mark and the feature information are displayed on the transparent member of the window of the moving body, the feature mark is displayed on the transparent member so that the feature mark and the feature indicated by it are visible as overlapping when viewed from the line of sight from inside the moving body, so that information about features located in a position visible from inside the moving body can be displayed instantly and easily. [Example]

[0017] Next, specific examples corresponding to the above-described embodiments will be described with reference to Figures 2 to 4. The examples described below are examples in which the present application is applied to an information display system that displays various information inside a vehicle moving using autonomous driving technology.

[0018] Fig. 2 is a block diagram showing the configuration of an information display system according to an embodiment, Fig. 3 is a flowchart showing a display control process according to an embodiment, and Fig. 4 is a diagram showing an example of information display according to an embodiment. In Fig. 2 and Fig. 4, the same component numbers as those of the display control device S according to the embodiment shown in Fig. 1 are used for the components of the embodiment corresponding to the components of the display control device S according to the embodiment.

[0019] As shown in Fig. 2, the information display system SS of the embodiment is installed in, for example, a vehicle that can move using automatic driving technology. The information display system SS of the embodiment is composed of a control unit 1 including a central processing system (CPU), read-only memory (ROM), random access memory (RAM), etc., a recording unit 2 including a hard disk drive (HDD) or solid state drive (SSD), etc., a global positioning system (GPS) receiving unit 3, a motion detection unit 4, a display unit 6, and an external communication unit 7. The control unit 1, recording unit 2, GPS receiving unit 3, motion detection unit 4, display unit 6, and external communication unit 7 are connected via a bus 5 so that data can be exchanged between them. In this case, the control unit 1 may be realized by a hardware logic circuit such as the CPU, or may be realized in software by the CPU reading and executing a program corresponding to the display control process of the embodiment described below. Moreover, the action detection unit 4 corresponds to an example of the identification means 4 of the embodiment, the external communication unit 7 corresponds to an example of the feature information acquisition means 7 of the embodiment and an example of the "identification information acquisition means" of the present application, and the control unit 1 corresponds to an example of the control means 1 of the embodiment. The control unit 1, the action detection unit 4, and the external communication unit 7 constitute an example of the display control device S of the embodiment. Furthermore, the control unit 1 corresponds to an example of the "map data acquisition means" of the present application, the GPS receiving unit 3 corresponds to an example of the "position data acquisition means" and an example of the "current position data acquisition means" of the present application, and the action detection unit 4 corresponds to an example of the "gaze data acquisition means" and an example of the "contact position data acquisition means" of the present application.

[0020] In the above configuration, the GPS receiver 3, under the control of the controller 1, generates current position data indicating the current position of the vehicle on which the display control system SS of the embodiment is mounted, based on navigation radio waves included in the GPS and received from a navigation satellite (not shown), and outputs the generated current position data to the controller 1. This current position is the current position of the vehicle expressed using, for example, latitude, longitude, and altitude. Meanwhile, the recording unit 2 non-volatilely stores map data MD indicating at least features such as buildings and facilities on the route of travel of the vehicle, and outputs the map data MD to the controller 1 as needed.

[0021] Under the control of the control unit 1, the external communication unit 7 is capable of wirelessly exchanging data with the outside of the vehicle. Furthermore, under the control of the control unit 1, the motion detection unit 4 detects, as specific motions (described later), the direction of the vehicle occupant's gaze or the manner of hand gestures, for example, using the results of imaging by an in-vehicle camera (not shown) in a conventional manner, and outputs the detection results to the control unit 1. Under the control of the control unit 1, the display unit 6 displays various information on the vehicle's glass using techniques such as projection. The vehicle's glass includes the windshield, side windows, rear window, and sunroof glass. The control unit 1 then executes a display control process for the display unit 6 using the current location data from the GPS receiver 3, data from the external communication unit 7, the detection results by the motion detection unit 4, and map data MD.

[0022] Next, the display control process of the embodiment, which is mainly executed by the control unit 1, will be specifically described with reference to FIGS.

[0023] First, the overall display control process of the embodiment will be described with reference to FIG. 3(a). The display control process is executed, for example, as an interrupt process at preset time intervals while an autonomously driving vehicle equipped with the display control system SS of the embodiment is moving. That is, in the display control process of the embodiment, the control unit 1 first detects the line of sight and its direction of the passenger (meaning the driver of the vehicle or other passengers; the same applies hereinafter) using the motion detection unit 4 (step S1). Next, the control unit 1 determines whether the passenger has performed a specific action, for example, gazing at a building outside the vehicle that is visible through the windshield at that time, based on the detection result of the motion detection unit 4 (step S2). If the specific action (gazing action) is not detected in the determination of step S2 (step S2: NO), the control unit 1 proceeds to step S1 to continue detecting the passenger's line of sight. On the other hand, if the determination in step S2 detects the specific behavior of gazing at a building to identify it (step S2: YES), the control unit 1 acquires various information about the building identified in the determination in step S2 (i.e., the building in the direction of the gaze detected as the specific behavior (i.e., the building being gazed at)) via the external communication unit 7 (step S3). In the following description, the various information acquired in step S3 will be simply referred to as "building-related information."

[0024] At this time, in step S3, the control unit 1 identifies the building that the passenger is gazing at, for example, based on the current position data from the GPS receiving unit 3, the detection result of the passenger's line of sight by the motion detecting unit 4, and the map data MD. The building-related information includes, for example, name information indicating the name of the building identified in step S2, location information indicating the location where it is located, feature information indicating the external features of the building, or information on an exhibition hall where information related to the building is exhibited.

[0025] Once the building-related information about the building outside the vehicle identified using the detection result of the passenger's line of sight, etc., is acquired in step S3, the control unit 1 then causes the display unit 6 to display the acquired building-related information, for example, on the windshield of the vehicle (step S4). This step S4 will be described in detail later.

[0026] Next, the control unit 1 determines whether or not to end the display control process of the embodiment, for example, because the vehicle has arrived at the destination (step S5). If the determination in step S5 is to end the display control process of the embodiment (step S5: YES), the control unit 1 ends the display control process. On the other hand, if the determination in step S5 is to continue the display control process of the embodiment (step S5: NO), the control unit 1 returns to step S1 and continues the display control process.

[0027] Next, the display of the building-related information in step S4 will be described. Here, in the display of the building-related information in step S4, it is assumed that, when the view outside the vehicle as shown in FIG. 4(a) is visible from inside the vehicle through the windshield GL, the occupant's line of sight is directed toward a building BD, the building BD is identified based on the detection results of the line of sight direction, etc. (see step S2: YES in FIG. 3(a)), and building-related information about the building BD has also been acquired (see step S3 in FIG. 3(a)). In this case, the control unit 1 causes the display unit 6 to display a marker MK indicating that the building BD is being gazed at at a position on the windshield GL where the detected line of sight direction intersects with the windshield GL (in other words, the position on the windshield GL that the occupant would see when gazing at the building BD (i.e., gazing through the windshield GL)) (step S40). In addition, the control unit 1 causes the display unit 6 to display the building-related information IF acquired in step S3 for the building BD, for example, at a position on the windshield GL different from the display position of the marker MK (for example, above the front of the windshield GL; see FIG. 4(a)), using a distinctive and eye-catching design (step S40). Furthermore, the control unit 1 also displays, on the display unit 6, an indication line L connecting the mark MK and the building-related information IF. At this time, the building-related information IF displayed is the content of the building-related information acquired in step S3, for example, using figures, characters, or videos.

[0028] Meanwhile, the vehicle is moving even while the mark MK and other symbols are being displayed in step S40. Therefore, the building BD that was visible through the windshield GL as shown in FIG. 4(a) also moves relatively to the right in FIG. 4(a) as the vehicle moves, and becomes visible only through the rear window via the right-hand side window. Therefore, if the position of the building BD moves out of the occupant's field of vision through the windshield GL as the vehicle moves, the control unit 1 continues to display the indicator line L based on the current position data and the map data MD, as shown in FIG. 4(b), so as to point in the direction in which the building BD moved as viewed from inside the vehicle (to the right in the example shown in FIG. 4), and also continues to display the building-related information IF on the windshield GL (step S40). Note that the control unit 1 continues to display the mark MK as long as the occupant continues to view the building BD. In this case, the indicator line L and the mark MK do not need to be connected (see FIG. 4(b)). In addition, after the control unit 1 starts the display illustrated in Figure 4(b), it displays a recording icon R, for example at the bottom center of the windshield GL, which instructs the contents of the building-related information IF to be recorded non-volatilely in the recording unit 2, for example.

[0029] While continuing to display the building-related information IF and the like illustrated in FIG. 4(b), the control unit 1 monitors whether the building BD, whose building-related information IF was displayed on the windshield GL, has become invisible from the entire field of view from inside the vehicle as the vehicle continues to move (step S41). During the monitoring in step S41, while the building BD is visible from inside the vehicle (step S41: NO), the control unit 1 continues to display the building-related information IF and indicator line L illustrated in FIG. 4(b), as well as the mark MK, which is not shown in FIG. 4(b) as the building BD has moved to the rear right. On the other hand, during the monitoring in step S41, if the building BD becomes invisible from inside the vehicle (step S41: YES), the control unit 1 stops displaying the mark MK that indicated the building BD and continues to display the building-related information IF and indicator line L in the state illustrated in FIG. 4(b) (step S42).

[0030] While the display of the building-related information IF and the indicator line L in step S42 continues, the control unit 1 monitors whether or not the recording icon R displayed on the windshield GL has been operated by the occupant as shown in Fig. 4(b) (step S43). The operation in step S43 may be, for example, a gesture by the occupant detected by the action detection unit 4, or may be a touch by the occupant to a touch sensor (not shown) provided on the inner surface of the windshield GL detected by the action detection unit 4.

[0031] If no operation on the recording icon R is detected during the monitoring in step S43 (step S43: NO), the control unit 1 continues displaying the information on the display unit 6 as shown in FIG. 4(b). On the other hand, if an operation on the recording icon R is detected during the monitoring in step S43 (step S43: YES), the control unit 1 records the content of the building-related information IF currently displayed on the windshield GL, for example, in the recording unit 2, and erases the building-related information IF and indicator line L that had been displayed on the windshield GL up to that point (step S44). Note that the recording of the content of the building-related information IF in step S44 may be performed not only in the recording unit 2 but also in an external server device via the external communication unit 7, for example. After recording of the content of the building-related information IF is completed, the control unit 1 proceeds to step S5 shown in FIG. 3(a).

[0032] As described above, according to the display control processing of the embodiment, when a mark MK indicating a building BD visible through the windshield GL of an automatically driven vehicle, building-related information IF about the building BD, and an indication line L connecting the mark MK and the building-related information IF are displayed on the windshield GL, the mark MK is displayed on the windshield GL so that the mark MK and the building BD are visible as overlapping when viewed from inside the vehicle, so that information about a building BD that is located in a position visible from inside the vehicle can be displayed instantly and easily.

[0033] In addition, when the identified building BD becomes invisible from inside the vehicle, the display of the mark MK is stopped (see step S42 in Figure 3(b)), so that visibility can be ensured even after the identified building BD becomes invisible.

[0034] Furthermore, even after the identified building BD becomes invisible, the indicator line L and the building-related information IF continue to be displayed so that the indicator line L indicates the direction in which the building BD becomes invisible from inside the vehicle (see Figure 4(b)), so that the information about the building BD can be reliably recognized even after it becomes invisible.

[0035] Furthermore, the building BD in the line of sight is identified based on position data indicating the current position of the vehicle, line of sight data indicating the direction of the line of sight from inside the vehicle, and map data MD corresponding to a map including the current position of the vehicle, so that information about the building BD being viewed from inside the vehicle can be displayed in the line of sight instantly and easily.

[0036] In addition, when an instruction to record building-related information IF is given, the display of the building-related information IF and the indication line L after the recording instruction is stopped (step S44 in Figure 3(b)), so that the display of the building-related information IF and the indication line L that no longer need to be displayed due to recording is stopped, thereby widening the passenger's field of view.

[0037] In the above-described embodiment, the markers MK and the like are displayed on the windshield GL based on the detection result of the line of sight of the occupant (see step S1 in FIG. 3(a)). However, in addition to this, if the touch panel is provided inside the windshield GL, the building BD may be identified based on current position data indicating the current position of the vehicle, contact position data indicating the contact position on the touch panel, and map data MD corresponding to a map including the current position of the vehicle. In this case, information about the building BD identified by contact with the touch panel (in other words, the windshield GL) can be displayed instantly and easily.

[0038] In the above-described embodiment, the above-described identifying operation of the building BD is performed by the passenger's gaze at the building BD. However, other configurations are also possible, such as recognizing a passenger's utterance (voice) such as "What's that?" and using that utterance as a trigger to identify the building BD based on the gaze of the passenger who made the utterance. In this case, the process of detecting the gazes of passengers other than the passenger who made the utterance can be omitted, which can have the effect of reducing the processing load on the control unit 1, for example.

[0039] Furthermore, the building BD may be identified using external information acquired via the external communication unit 7. That is, the building BD may be identified based on current location data indicating the current location of the vehicle, the above-mentioned external information for identifying the building BD, such as the name of the building BD, and map data MD corresponding to a map including the current location of the vehicle. In this case, information about the building BD identified by external identification information can be displayed instantly and easily.

[0040] Furthermore, in the above-described embodiment, the present application is applied when displaying building-related information IF, etc. on the windshield GL of a vehicle, but the present application can also be applied when displaying building-related information IF, etc. on a transparent part around it, for example, on the transparent part of a motorcycle.

[0041] In addition, a program corresponding to the flowcharts shown in FIG. 3 may be recorded on a recording medium such as an optical disk or obtained via a network such as the Internet, and then read and executed by a general-purpose microcomputer or the like, thereby causing the microcomputer or the like to function as the control unit 1 according to the embodiment. [Explanation of symbols]

[0042] 1 Control means (control unit) 4. Identification means (motion detection unit) 6 Display section 7. Means for acquiring geographical information (external communication unit) S Display Control Device L indicator line GL windshield MK marker IF Building-related information BD Building

Claims

1. An identification means for identifying any of the features visible through a transparent member of a window of the mobile body; a feature information acquisition means for acquiring feature information about the identified feature; a control means for displaying, on the transparent member, a feature mark that is visible through the transparent member and indicates the identified feature, the acquired feature information, and an indication line that connects the feature mark and the feature information; Equipped with The control means displaying the feature mark on the transparent member so that the feature mark and the identified feature are visually recognized as overlapping when viewed from within the moving body; A display control device characterized in that, when an instruction to record the feature information is given, the display of the feature information and the indication line after the instruction to record is stopped.

2. 2. The display control device according to claim 1, The display control device is characterized in that the control means stops displaying the feature mark when the identified feature becomes invisible from inside the moving body.

3. 3. The display control device according to claim 1, The control means is a display control device characterized in that after the identified feature becomes invisible from within the moving body, the control means continues to display the indication line and the feature information so that the indication line indicates the direction in which the feature became invisible from within the moving body.

4. 4. The display control device according to claim 1, a position data acquisition means for acquiring position data indicating a current position of the mobile object; a gaze data acquisition means for acquiring gaze data indicating the direction of the gaze; map data acquisition means for acquiring map data corresponding to a map including the current position; Further provided with The display control device is characterized in that the identification means identifies the feature on the line of sight based on the acquired position data, line of sight data, and map data.

5. 5. The display control device according to claim 4, The display control device is characterized in that the identification means identifies the feature being gazed at on the line of sight based on the acquired position data, line of sight data, and map data.

6. 4. The display control device according to claim 1, a current position data acquisition means for acquiring current position data indicating the current position of the mobile object; a contact position data acquisition means for acquiring contact position data indicating a contact position on the transparent member when the transparent member is contacted from within the moving body; map data acquisition means for acquiring map data corresponding to a map including the current position; Further provided with The display control device is characterized in that the specifying means specifies the feature based on the acquired current position data, contact position data, and map data.

7. 4. The display control device according to claim 1, a position data acquisition means for acquiring position data indicating a current position of the mobile object; a specific information acquisition means for acquiring specific information for identifying the feature from outside the mobile body; map data acquisition means for acquiring map data corresponding to a map including the current position; Further provided with The display control device is characterized in that the identification means identifies the feature based on the acquired position data, identification information, and map data.

8. A display control method executed in a display control device including a specifying means, a feature information acquiring means, and a control means, an identifying step of identifying any feature visible through a transparent member of a window of the mobile body by the identifying means; a feature information acquisition step of acquiring feature information about the identified feature by the feature information acquisition means; a control step of displaying, on the transparent member, a feature mark that is visible through the transparent member and indicates the identified feature, the acquired feature information, and an indication line that connects the feature mark and the feature information, using the control means; Including, In the control step, displaying the feature mark on the transparent member so that the feature mark and the identified feature are visually recognized as overlapping when viewed from within the moving body; A display control method characterized in that, when an instruction to record the feature information is given, display of the feature information and the indication line after the instruction to record is stopped.

9. Computer, Identification means for identifying any of the features visible through the transparent member of the window of the mobile body; a feature information acquisition means for acquiring feature information about the identified feature; and a control means for displaying, on the transparent member, a feature mark that is visible through the transparent member and indicates the identified feature, the acquired feature information, and an indication line that connects the feature mark and the feature information; A display control program that functions as The computer that functions as the control means, displaying the feature mark on the transparent member so that the feature mark and the identified feature are visually recognized as overlapping when viewed from within the moving body; A display control program characterized by causing, when an instruction to record the feature information is given, the program to stop displaying the feature information and the indication line after the instruction to record the feature information.

Images