Driving information display device, driving information display method, driving information control device, and driving information control method

Abstract

This invention provides a technology that uses AR technology to display information related to driving a railway vehicle to the driver in an appropriate location. [Solution] The driving information display device of the present invention comprises an information acquisition unit that acquires driving information, an information display unit that displays the driving information, a camera image acquisition unit that generates a driver's field of view image, a vehicle feature matching unit that associates vehicle features in the field of view image with vehicle feature information recorded in a vehicle feature storage unit, and an information display control unit that sets an information display prohibited area in the field of view image based on the vehicle feature information. The information display control unit sets a display area for driving information outside the information display prohibited area and causes the information display unit to display the driving information. The information display control unit may determine the position of the front window frame of the railway vehicle in the field of view image based on the vehicle feature information and set the inside of the front window frame of the railway vehicle as the information display prohibited area.

Description

Technical Field

[0001] The present invention relates to a driving information display device, a driving information display method, a driving information control device, and a driving information control method for providing driving information to a driver during the operation of a railway vehicle.

Background Art

[0002] During the operation of a railway vehicle, the driver performs driving operations while simultaneously recognizing and processing various driving information inside and outside the vehicle. Generally, the driving information that the driver should recognize is classified into information related to ground-side facilities and information related to the vehicle state. The driver visually recognizes such information during driving. For example, for information related to ground-side facilities, the driver visually recognizes it through the glass in front of the vehicle, and for information related to the vehicle state, the driver visually recognizes it by looking at the instruments installed on the driver's cab. The driver needs to continuously monitor the outside of the vehicle in front on the running route from the perspective of safety. However, in order to recognize information related to the vehicle state, the driver has no choice but to shift the line of sight to the instruments inside the vehicle, and this line-of-sight movement poses a burden on the driver.

[0003] Here, in order to assist the recognition of driving information inside and outside the vehicle during driving, it is known to use a technology called AR (Augmented Reality). AR technology is a technology that combines digital information such as computer graphics with images and videos of the real world to create a new expression. By using AR technology, it is possible to additionally provide digital information that does not exist in the real world in a form of superimposed display on the information of the real world for the person using a display device such as an HMD (Head Mount Display).

[0004] As an example of presenting information related to ground-side facilities to the driver using AR technology, Patent Document 1 discloses a technology of extracting ground-side facilities from a camera image that captures the driver's field of view and installing an area for displaying information related to the ground-side facilities at a coordinate position within an angle of view that does not overlap with the ground-side facilities.

Prior Art Documents

[0005] [Patent Document 1] Japanese Patent Publication No. 2022-46276 [Overview of the project] [Problems that the invention aims to solve]

[0006] In the future, it is predicted that railway vehicle operation will increasingly involve drivers with less stringent qualifications than currently required, or even crew members without qualifications at all, as well as driverless, automated operation. On the other hand, even with automated operation, crew members may still take over in emergencies such as accidents. Therefore, in the future, there will be an even greater need to provide driving information in a more easily understandable format, even to those who do not possess the qualifications of a licensed driver.

[0007] However, as mentioned above, the technology disclosed in Patent Document 1 is limited to displaying only information related to ground-side equipment as driving information using AR technology, and is based on the premise of displaying AR objects within the scenery in front of the railway vehicle. Therefore, it was not satisfactory in terms of both the content and position of the displayed driving information.

[0008] Therefore, the present invention aims to provide a technology that uses AR technology to display driving-related information to the driver in an appropriate location. In particular, it aims to provide a driving information display device that presents driving-related information in a way that is easy for the driver to recognize, while avoiding interference with information related to ground-side equipment. [Means for solving the problem]

[0009] To solve the above problems, one representative driving information display device of the present invention comprises an information acquisition unit that acquires driving information, an information display unit that displays the driving information, a camera image acquisition unit that generates a driver's field of view image, a vehicle feature matching unit that associates vehicle features in the field of view image with vehicle feature information recorded in a vehicle feature storage unit, and an information display control unit that sets an information display prohibited area in the field of view image based on the vehicle feature information, the information display control unit sets a display area for driving information outside the information display prohibited area and causes the information display unit to display the driving information. [Effects of the Invention]

[0010] According to the present invention, it becomes possible to display driving information to the driver in an appropriate location using AR technology. Issues, structures, and effects other than those mentioned above will be clarified by the following explanation of the implementation methods. [Brief explanation of the drawing]

[0011] [Figure 1] Figure 1 shows an example of the configuration of the driving information display device in Example 1. [Figure 2] Figure 2 shows an example of the processing flow of the driving information display device in Example 1. [Figure 3] Figure 3 shows an example of information display in the driving information display device of Example 1. [Figure 4] Figure 4 shows an example of the processing flow of the driving information display device in Example 2. [Figure 5] Figure 5 shows an example of information display on the driving information display device of Example 2. [Figure 6] Figure 6 shows an example of the processing flow of the driving information display device in Example 3. [Figure 7] Figure 7 shows an example of information display on the driving information display device of Example 3. [Figure 8] Figure 8 shows an example of the processing flow of the driving information display device in Example 4. [Figure 9]FIG. 9 is a diagram showing an example of information display of the operation information display device according to the fourth embodiment. [Figure 10] FIG. 10 is a diagram showing an example of the configuration of the operation information display device according to the fifth embodiment. [Figure 11] FIG. 11 is a diagram showing an example of the processing flow of the operation information display device according to the fifth embodiment.

MODE FOR CARRYING OUT THE INVENTION

[0012] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the present invention is not limited by this embodiment.

[0013] First, terms commonly used in this embodiment will be described. (Operation information) In the present disclosure, operation information means information related to the operation of a railway vehicle that a driver should refer to during operation. For example, it includes the speed, position, name of the stop station, distance to the stop station, mass control operation state, opening and closing of the vehicle door, and vehicle interior temperature of the railway vehicle.

[0014] (Ground feature) In the present disclosure, a ground feature means an external object that a driver should visually recognize during operation, that is, an object to be monitored in the front. For example, it includes signal lights, tracks, and station platforms.

[0015] (Vehicle feature) In the present disclosure, vehicle features mean equipment that characterizes a railway vehicle provided around the driver's seat. For example, it includes window frames, operating devices, monitors, and instruments on the front of the vehicle.

EXAMPLE

[0018] (Camera image acquisition unit) The camera image acquisition unit 101 is a camera means that optically acquires information from the outside world and has the function of generating a driver's field of view image 121. Here, the driver's field of view image 121 is an image generated by the camera image acquisition unit 101 that corresponds to the range of the driver's actual field of view. Examples of implementations of the camera image acquisition unit 101 include a camera for acquiring field of view images mounted on a head-mounted display or camera-type device, a camera worn on the driver's head, and a camera installed inside the vehicle. The driver's field of view image 121 generated by the camera image acquisition unit 101 is output to the feature matching unit 102, the vehicle feature matching unit 103, the information display control unit 106, etc.

[0019] (Geographical Records Department) The feature recording unit 111 includes a database of features that the driver should visually confirm while driving. The database records feature information 122. Here, feature information 122 is information that uniquely identifies a feature, such as its type, absolute position (latitude, longitude, and kilometer distance can be used), shape, and color. Various recording media such as HDDs, SSDs, and SD cards can be used as the feature recording unit 111. The feature information 122 is output from the feature recording unit 111 to the feature matching unit 102. Note that the feature information 122 may change or be modified over time or depending on the season, so the feature recording unit 111 is updated at appropriate intervals.

[0020] (Local Property Matching Department) The feature matching unit 102 has the function of extracting features from the driver's view image 121 and associating them with feature information 122 from the feature recording unit 111. This association is performed using feature information 122 input from the feature recording unit 111, the driver's view image 121 input from the camera image acquisition unit 101, and train position information 127 input from the information acquisition unit 104. The result of the association is output to the information display control unit 106 as in-image feature information 124. Here, in-image feature information 124 includes the contents of feature information 122, as well as the coordinate position, shape, and color within the driver's view image 121.

[0021] Furthermore, there are no particular limitations on the technique used by the feature matching unit 102 to extract features within the driver's view image 121. For example, machine learning can be used to learn the shape and color data of features and recognize the location of features within the driver's view image 121, or pattern matching can be used to determine areas with a high degree of agreement with the shape and color of features within the driver's view image 121. In addition, the relative position of features relative to the railway vehicle can be calculated from the position of the railway vehicle in operation obtained from the train position information 127 and the position of the features obtained from the feature information 122, and features within the driver's view image 121 can be recognized based on this.

[0022] (Vehicle Features Recording Section) The vehicle feature recording unit 112 includes a database of vehicle features installed around the driver's seat. The database records vehicle feature information. Here, vehicle feature information refers to information that characterizes a railway vehicle, such as the type of vehicle feature, its three-dimensional position with the interior of the vehicle as the origin, shape, and color. Various recording media such as HDDs, SSDs, and SD cards can be used for the vehicle feature recording unit 112. The vehicle feature information 123 is output from the vehicle feature recording unit 112 to the vehicle feature matching unit 103.

[0023] (Vehicle feature matching section) The vehicle feature matching unit 103 has the function of extracting vehicle features from the driver's view image 121 and associating them with the vehicle feature information 123 from the vehicle feature recording unit 112. This association is performed using the vehicle feature information 123 input from the vehicle feature recording unit 112 and the driver's view image 121 input from the camera image acquisition unit 101. The result of the association is output to the information display control unit 106 as in-image vehicle feature information 125. Here, in-image vehicle feature information 125 includes the contents of the vehicle feature information 123, as well as the coordinate position, shape, and color within the driver's view image 121.

[0024] Furthermore, there are no particular limitations on the technique used by the vehicle feature matching unit 103 to extract vehicle features from the driver's view image 121. It is also possible to use the extraction technique described in the above-mentioned feature matching unit 102.

[0025] (Information acquisition department) The information acquisition unit 104 has the function of creating in-train vehicle information 126 and train position information 127 using data from various devices installed on the vehicle. Here, the devices installed on the vehicle may include, for example, motors, brakes, in-train air conditioning, vehicle doors, speed generators, and vehicle information control devices that comprehensively control on-board equipment. The in-train vehicle information 126 includes driving information that the driver refers to during operation, such as the vehicle's running position, speed, stopping stations, distance to stopping stations, master controller operation status, vehicle door open / closed status, and interior temperature. The in-train vehicle information 126 is output to the information display control unit 106. In addition, the vehicle's running position from the in-train vehicle information 126 is output to the terrain matching unit 102 as train position information 127.

[0026] (Information display section) The information display unit 105 has the function of displaying information in an appropriate area within the driver's field of view 301. Examples of implementations of the information display unit 105 include HMDs worn by the driver and glasses-type head-up displays.

[0027] (Information display control unit) The information display control unit 106 has the function of displaying driving information in an appropriate position within the driver's field of view on the information display unit 105, based on the image feature information 124 input from the feature matching unit 102, the image vehicle feature information 125 input from the vehicle feature matching unit 103, and the driving vehicle information 126 input from the information acquisition unit 104.

[0028] Each processing unit and recording unit in Figure 1 may be integrated into a single device, with each function implemented as software, or they may be separate devices that communicate with each other to achieve their functions. Furthermore, while the camera image acquisition unit 101 and information display unit 105, which serve as interfaces for the driver, are installed inside the vehicle or on the driver's body, the installation location of the other processing units and recording units is not restricted as long as they can communicate with each other.

[0029] <Processing flow of the driving information display device in Example 1> The processing flow of the driving information display device in Example 1 will be explained with reference to Figure 2. Figure 2 shows an example of the processing flow of the driving information display device in Example 1. The processing flow shown in Figure 2 is executed periodically during the operation of the information display control unit 106. The operation period may be the same as the display update period of the information display control unit 106.

[0030] In process 201, the driver's field of view image 121 is acquired from the camera image acquisition unit 101. The acquired driver's field of view image 121 is used in processes 202 to 206. In this embodiment, the camera image acquisition unit 101 is assumed to be a camera for acquiring field of view images mounted on a glasses-type head-mounted display.

[0031] In process 202, the feature matching unit 102 associates the feature information 122 recorded in the feature recording unit 111 with the features extracted from the driver's view image 121 acquired in process 201 to create in-image feature information 124. The created in-image feature information 124 is used in process 204.

[0032] In process 203, the vehicle feature matching unit 103 associates the vehicle features extracted from the driver's view image 121 acquired in process 201 with the vehicle feature information 123 recorded in the vehicle feature recording unit 112 to create in-image vehicle feature information 125. The created in-image vehicle feature information 125 is used in process 204. If vehicle features are not extracted from the driver's view image 121 during a certain cycle, processing 204 and subsequent steps may be performed using vehicle features extracted in the previous cycle.

[0033] In process 204, the information display control unit 106 estimates the position and orientation of the camera image acquisition unit 101 when the driver's view image 121 is acquired. There are no particular limitations on how the position and orientation of the camera image acquisition unit 101 are estimated. For example, the three-dimensional position and shape of the vehicle features, with the vehicle interior as the origin, included in the vehicle feature information 125 input from process 203, may be compared with the position and shape of the vehicle features in the driver's view image 121, and the position and orientation of the camera image acquisition unit 101 may be calculated such that the two match. The calculated position and orientation may be expressed using the three-dimensional position with the vehicle interior as the origin and the orientation angles (roll, pitch, yaw) for each axis, or the orientation may be expressed using quaternions instead of angles. The calculated position and orientation of the camera image acquisition unit 101 are used in process 205. Furthermore, the position and orientation of the camera image acquisition unit 101 can also be calculated using the image object information 124 and train position information 127 input from the processing unit 202.

[0034] In process 205, the information display control unit 106 first calculates the position of the front window frame within the driver's view image 121 from the position and orientation of the camera image acquisition unit 101 estimated in process 204 and the three-dimensional position of the front window frame of the vehicle, recorded in the vehicle feature recording unit 112 with the vehicle interior as the origin. Next, based on the calculated position of the front window frame of the vehicle, the interior image region (the region outside the front window frame of the vehicle) and the exterior image region (the region inside the front window frame of the vehicle) within the driver's view image 121 are identified, and the exterior image region is set as the information display prohibited region. The information display prohibited region is used in process 206. Note that the information display prohibited region may be set for each pixel in the driver's view image 121.

[0035] In process 206, the information display control unit 106 sets an information prohibition area in the driver's field of view corresponding to the information prohibition area within the driver's field of view image 121, and the information display unit 105 displays (projects) predetermined driving information from the vehicle information 126 outside this area. Here, the display area for the driving information is outside the information prohibition area and is the closest position to the center of the driver's field of view image 121. Furthermore, the background color of the display area should be set to a color with a large hue difference from the surrounding color of the display area in order to improve the visibility of the information. After process 206, the process returns to process 201 and proceeds to the next cycle.

[0036] In this embodiment, the position of the window frame on the front of the vehicle within the driver's field of view image 121 was determined in process 205 based on the position and orientation of the camera image acquisition unit obtained in process 204. However, the position of the window frame on the front of the vehicle may also be determined directly by performing pattern matching in the vehicle feature matching unit 103.

[0037] <Information display of the driving information display device in Example 1> The information display of the driving information display device in Example 1 will be explained with reference to Figure 3. Figure 3 shows an example of information display in the driving information display device of Example 1.

[0038] The field of view 301 indicates the driver's field of view, and the driver's field of view image 121 acquired by the camera image acquisition unit 101 is the corresponding image. Based on the relative relationship between the position of the front window frame 302 of the vehicle in the field of view 301 and the position of the front window frame of the vehicle in the driver's field of view image 121 calculated in process 205 of Figure 2, an area corresponding to the information display prohibited area in the driver's field of view image 121 set in process 205 is set within the field of view 301. In this embodiment, since the inside of the front window frame 302 of the vehicle is set as the information display prohibited area, the driving information display area 304 is projected outside the front window frame 302 of the vehicle and close to the center of the field of view 301.

[0039] This allows driving-related information to be displayed in a position that does not interfere with the visibility of objects being monitored ahead. [Examples]

[0040] The difference between Example 2 and Example 1 lies in the location where the information display prohibition area is set. In Example 2, the information display prohibition area is set around the feature object that is the target of forward monitoring.

[0041] <Processing flow of the driving information display device in Example 2> The flow of the driving information display device in Example 2 will be explained using Figure 4. Figure 4 shows an example of the processing flow of the driving information display device in Example 2. In Figure 4, process 405 is added in place of process 205 in Figure 2. The other processes are the same as in Example 1, so their explanation is omitted.

[0042] In process 405, the information display control unit 106 calculates the position of a feature within the driver's view image 121 from the position and orientation of the camera image acquisition unit 101 estimated in process 204 and the feature information 124 within the image created in process 202. At the same time, the position of the window frame on the front of the vehicle within the driver's view image 121 is also calculated using the same process as in process 205 of Embodiment 1. In this embodiment, based on the calculated position of the feature, the area surrounding the feature within the driver's view image 121 is set as an information display prohibited area. The information display prohibited area is used in process 206.

[0043] In the subsequent process 206, the information display control unit 106 sets an information display prohibition area in the driver's field of view that corresponds to the information display prohibition area within the driver's field of view image 121, and the information display unit 105 displays (projects) predetermined driving information from the vehicle information 126 outside this information display prohibition area. Here, as long as it does not overlap with the set information display prohibition area, the display area for driving information is prioritized to be within the window frame at the front of the vehicle.

[0044] <Information display of the driving information display device in Example 2> The information display of the driving information display device in Example 2 will be explained with reference to Figure 5. Figure 5 shows an example of information display on the driving information display device of Example 2.

[0045] The field of view 301 indicates the driver's field of view, and the driver's field of view image 121 acquired by the camera image acquisition unit 101 is the corresponding image. Based on the relative relationship between the position of the landmark 303 (signal light) in the field of view 301 and the position of the landmark (signal light) in the driver's field of view image 121 calculated in processing 405 of Figure 4, the information display prohibition area in the driver's field of view image 121 is set within the field of view 301. In this embodiment, the area around the landmark 303 (signal light) is set as the information display prohibition area 505, so the driving information display area 504 is projected within the window frame 302 on the front of the vehicle that does not overlap with this area.

[0046] In this embodiment, we have used one feature (signal light) 303 as an example to explain how to set an information display prohibition area, but this is not the only way, and it is possible to set multiple features. For example, although we will not go into detail here, Figure 5 shows the result of considering features (railway tracks) when setting an information display prohibition area.

[0047] This allows driving information to be displayed in a position that does not interfere with the visibility of objects being monitored ahead, and also reduces the driver's burden as the driver's eye movement can be completed within the window frame. [Examples]

[0048] The difference between Example 3 and Example 1 is that Example 3 incorporates processing to ensure that the display area for driving information does not change as much as possible even when the driver's posture (viewing angle) changes.

[0049] <Processing flow of the driving information display device in Example 3> The flow of the driving information display device in Example 3 will be explained using Figure 6. Figure 6 shows an example of the processing flow of the driving information display device in Example 3. In Figure 6, processes 607 to 609 are added before and after process 206 in Figure 2. The other processes are the same as in Example 1, so their explanation is omitted.

[0050] In process 607, the display area for the operating information set in the processing flow of the previous cycle is compared with the information display prohibition area set in the processing flow of the current cycle to determine whether there are any overlapping areas between the two. If there are overlapping areas between the two, the process proceeds to process 206; otherwise, the process proceeds to process 608.

[0051] In process 608, the display area for the operating information set in the previous cycle's processing flow, which is saved in process 609, is applied as the display area for the operating information set in the current cycle's processing flow. After process 608, the process proceeds to process 609.

[0052] Process 609 is a process that saves the display area of ​​the operating information set in the processing flow of the current cycle until the next cycle. The display area of ​​the operating information saved here will be used in process 607 of the next cycle.

[0053] <Information display of the driving information display device in Example 3> An example of information display on the driving information display device in Example 3 will be explained with reference to Figure 7. Figure 7 shows an example of information display on the driving information display device of Example 3.

[0054] Figure 7 shows a case where the driver's posture shifts forward, causing the window frame 302 at the front of the vehicle to occupy a larger portion of the field of view 301 than in Figure 2. Due to this change in the field of view 301, the display area 304 for driving information, which should have been outside the information display prohibition area (outside the window frame 302 at the front of the vehicle in Figure 2) in the previous cycle, now overlaps with the information display prohibition area (inside the window frame 703 at the front of the vehicle in Figure 7) in the current cycle.

[0055] In this embodiment, in such cases, the display area 704 of the updated driving information in process 608 is reset to a position that does not overlap with the information display prohibited area for the current cycle. On the other hand, even if there is a change in the driver's posture, if the display area 304 of the driving information from the previous cycle does not overlap with the information display prohibited area for the current cycle, the display area 304 of the driving information from the previous cycle is maintained without changing the position of the display area.

[0056] This allows driving-related information to be continuously displayed in a fixed position within the driver's field of view, as long as it does not overlap with the area where information display is prohibited. This reduces the driver's eye movement and alleviates their burden. [Examples]

[0057] The difference between Example 4 and Example 1 is that Example 4 adds a process to set the display area near the operating device related to driving information.

[0058] <Processing flow of the driving information display device in Example 4> The flow of the driving information display device in Example 4 will be explained using Figure 8. Figure 8 shows an example of the processing flow of the driving information display device in Example 4. In Figure 8, processes 807 to 809 are added between processes 205 and 206 in Figure 2. The other processes are the same as in Example 1, so their explanation is omitted.

[0059] In process 807, the information display control unit 106 calculates the position of the control device within the driver's field of view image 121 from the position and orientation of the camera image acquisition unit 101 estimated in process 204 and the vehicle feature information 125 (three-dimensional position, etc.) of the control device within the image created in process 203, and sets the area around the control device as an information display priority area. The set information display priority area is used in process 808.

[0060] In process 808, the information display priority area set in process 807 and the information display prohibition area set in process 205 are compared to determine whether there are any overlapping areas between them. If there are overlapping areas between them, the process proceeds to process 206; otherwise, the process proceeds to process 809.

[0061] In process 809, the display area for driving information is set in a position that does not overlap with the operating devices within the information display priority area set in process 807, and this is reflected in the driver's field of view 301 to display the driving information. After process 809, the process proceeds to the next cycle.

[0062] <Information display of the driving information display device in Example 4> The information display of the driving information display device in Example 4 will be explained with reference to Figure 9. Figure 9 shows an example of information display on the driving information display device of Example 4.

[0063] In this embodiment, within the driver's field of view 301, the display area 904 for driving information is projected into an information display priority area 906 set around the operating device 905.

[0064] This allows driving-related information to be displayed in a location close to the control devices, making it easier to intuitively understand the content of the driving information and reducing the burden on the driver. [Examples]

[0065] The difference between Example 5 and Example 1 is that, even if the orientation of the camera image acquisition unit 101 does not change, if the driver's line of sight changes, a process is added to change the field of view 301 to match that orientation.

[0066] <Configuration of the driving information display device in Example 5> The configuration of the driving information display device in Example 5 will be explained with reference to Figure 10. Figure 10 shows an example of the configuration of the driving information display device of Embodiment 5. In Figure 10, the eye-line direction acquisition unit 1000 and eye-line direction information 1020 are added to the configuration of Figure 1, and a camera image acquisition unit 1001 is added in place of the camera image acquisition unit 101 in Figure 1. The other configurations are the same as in Embodiment 1, so their explanation is omitted.

[0067] (Eye direction acquisition unit) The eye-line direction acquisition unit 1000 has the function of acquiring eye-line direction information 1020 from the direction of the driver's eyes. Here, the eye-line direction information 1020 is a three-dimensional vector based on the area in front of the driver's head, and consists of three elements: roll, pitch, and yaw.

[0068] The method for detecting the direction of the gaze is not particularly limited. For example, a method for detecting the direction of the eyeball using a visible light camera or a method for detecting the position of the pupil and corneal reflection using an infrared light camera may be used. The gaze direction information 1020 acquired by the gaze direction acquisition unit 1000 is output to the camera image acquisition unit 1001. The gaze direction acquisition unit 1000 may be worn on the driver's head or installed inside the vehicle.

[0069] (Camera image acquisition unit) The camera image acquisition unit 1001 has the function of acquiring a driver's field of view image 121 in which the driver's line of sight is at the center of the image, based on the eye direction information 1020. Other functions are the same as those of the camera image acquisition unit 101 in Embodiment 1.

[0070] <Processing flow of the driving information display device in Example 5> The flow of the driving information display device in Example 5 will be explained using Figure 11. Figure 11 shows an example of the processing flow of the driving information display device in Example 5. In Figure 11, processes 1100 and 1101 are added before process 202 in Figure 2.

[0071] In process 1100, the eye-direction acquisition unit 1000 acquires the driver's eye-direction information 1020. The eye-direction information 1020 is used in process 1101.

[0072] In process 1101, the camera image acquisition unit 1001 acquires a driver's field of view image 121 centered on the driver's eye direction using the eye direction information 1020 acquired in process 1100. The driver's field of view image 121 is used in processes 202 to 206.

[0073] This allows for the display of driving information to follow the driver's line of sight, thus reducing the driver's workload.

[0074] <Variation> Although embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above, and various modifications are possible without departing from the spirit of the present invention.

[0075] For example, in the embodiment described above, the driving information display device is configured to include all the processing and storage units shown in Figures 1 and 10. However, it is not limited to this configuration, and the feature recording unit, feature matching unit, and vehicle feature recording unit can also be configured as external devices.

[0076] Furthermore, among the driving information display devices of the above-described embodiment, it is also possible to configure the information acquisition unit, vehicle feature matching unit, and information display control unit related to the display control of driving information as a single driving information control device. This disclosure includes the following aspects:

[0077] (Aspect 1) In a driver information display device that displays driving information to the driver of a railway vehicle, An information acquisition unit that acquires the aforementioned driving information, An information display unit that displays the aforementioned driving information, A camera image acquisition unit that generates the driver's field of view image, A vehicle feature matching unit that associates vehicle features in the aforementioned visual image with vehicle feature information recorded in the vehicle feature storage unit, The system includes an information display control unit that sets an information display prohibited area in the field image based on the vehicle characteristic information, The information display control unit is characterized by setting a display area for the driving information outside the information display prohibited area and causing the information display unit to display the driving information.

[0078] (Aspect 2) A driving information display device according to Embodiment 1, The information display control unit determines the position of the front window frame of the railway vehicle in the view image based on the vehicle characteristic information, and sets the inside of the front window frame of the railway vehicle as the information display prohibited area.

[0079] (Aspect 3) A driving information display device according to either embodiment 1 or 2, The system further includes a feature matching unit that associates features in the aforementioned visual image with feature information recorded in the feature recording unit. The information display control unit determines the position of a feature in the field of view image based on the vehicle characteristic information and the feature information, and sets the vicinity of the position of the feature as the information display prohibited area.

[0080] (Aspect 4) A driving information display device according to any one of embodiments 1 to 3, The information display control unit sets the information display prohibited area at predetermined intervals, and if there is no overlap between the information display prohibited area of ​​the latest period and the display area of ​​the operation information of the previous period, the display position of the operation information of the previous period is not changed.

[0081] (Appendix 5) A driving information display device according to any one of embodiments 1 to 4, The driving information display device is characterized in that, when the information display control unit associates the vehicle characteristic information with the operating device in the view image, it determines the display position of the operating device in the view image, sets the vicinity of the display position of the operating device as an information display priority area, and, if the information display priority area does not overlap with the information display prohibition area, the information display unit displays the driving information in the information display priority area.

[0082] (Aspect 6) A driving information display device according to any one of embodiments 1 to 5, The unit further includes a gaze direction acquisition unit that acquires the direction of the driver's gaze, The aforementioned camera image acquisition unit is characterized by generating the field of view image according to the direction of the line of sight.

[0083] (Aspect 7) A driving information display device according to any one of embodiments 1 to 6, The information display control unit is characterized in that it sets a color with a large hue difference from the surrounding background color as the background color of the display area for the driving information.

[0084] (Pattern 8) A railway vehicle equipped with a driving information display device according to any one of claims 1 to 7.

[0085] (Aspect 9) In a method for displaying driving information to a railway vehicle driver, The information acquisition unit acquires the aforementioned operating information, The camera image acquisition unit generates the driver's field of view image, The vehicle feature matching unit associates the vehicle features in the view image with the vehicle feature information recorded in the vehicle feature storage unit. The information display control unit sets an information display prohibition area in the field of view image based on the vehicle characteristic information, and sets a display area for the driving information outside the information display prohibition area. A method for displaying driving information, characterized in that the driving information is displayed by an information display unit.

[0086] (Aspect 10) In a train information control device that controls the display of an information display unit that shows train information to the train driver, An information acquisition unit that acquires the aforementioned driving information, A vehicle feature matching unit receives the driver's field of view image and associates the vehicle features in the field of view image with the vehicle feature information recorded in the vehicle feature recording unit. The system includes an information display control unit that sets an information display prohibited area in the field image based on the vehicle characteristic information, The information display control unit is characterized by setting a display area for the driving information outside the information display prohibited area and causing the display unit to display the driving information.

[0087] (Aspect 11) The driving information control device described in embodiment 10, The information display control unit is characterized by determining the position of the front window frame of the railway vehicle in the view image based on the vehicle characteristic information, and setting the inside of the front window frame of the railway vehicle as the information display prohibited area.

[0088] (Aspect 12) An operation information control device according to either embodiment 10 or 11, The system further includes a feature matching unit that associates features in the aforementioned visual image with feature information recorded in the feature recording unit. The information display control unit is characterized by determining the position of a feature in the view image based on the vehicle characteristic information and the feature information, and setting the vicinity of the position of the feature as the information display prohibited area.

[0089] (Aspect 13) An operation information control device according to any one of embodiments 10 to 12, The information display control unit sets the information display prohibited area at predetermined intervals, and if there is no overlap between the information display prohibited area of ​​the latest period and the display area of ​​the operation information of the previous period, the operation information control unit does not change the display position of the operation information of the previous period.

[0090] (Aspect 14) An operation information control device according to any one of embodiments 10 to 13, The driving information control device is characterized in that, when the information display control unit associates the vehicle characteristic information with the operating device in the view image, it determines the display position of the operating device in the view image, sets the vicinity of the display position of the operating device as an information display priority area, and, if the information display priority area does not overlap with the information display prohibition area, causes the display device to display the driving information in the information display priority area.

[0091] (Aspect 15) An operation information control device according to any one of embodiments 10 to 14, The information display control unit is characterized in that it sets a color with a large hue difference from the surrounding background color as the background color of the display area for the driving information.

[0092] (Aspect 16) A railway vehicle equipped with a driving information control device as described in any of embodiments 1 to 15.

[0093] (Aspect 17) In a driving information control method that controls the display of an information display unit that displays driving information to the driver of a railway vehicle, The information acquisition unit acquires the aforementioned operating information, The vehicle feature matching unit receives the driver's view image and associates the vehicle features in the view image with the vehicle feature information recorded in the vehicle feature recording unit. A driving information control method characterized in that an information display control unit sets an information display prohibition area in the field of view image based on the vehicle characteristic information, sets a driving information display area outside the information display prohibition area, and displays the driving information on the display unit. [Explanation of Symbols]

[0094] 101: Camera image acquisition unit 102: Local Matching Department 103: Vehicle Feature Matching Section 104: Information acquisition department 105: Information display section 106: Information Display Control Unit 111: Geographical Records Department 112: Vehicle Features Recording Unit 121: Driver's View Image 122: Feature information 123: Vehicle Feature Information 124: Image local information 125: Vehicle feature information in the image 126: Vehicle Information While in Operation 127:Train location information 301:Viewing angle 302, 703: Window frames on the front of railway vehicles 303: Feature (traffic light) 304, 504, 704, 904: Display area for driving information 505: Information display prohibited area 905: Operation equipment 906: Information display priority area 1000: Eye-line direction acquisition unit 1001: Camera image acquisition unit 1020: Eye direction information

Claims

1. In a driver information display device that displays driving information to the driver of a railway vehicle, An information acquisition unit that acquires the aforementioned driving information, An information display unit that displays the aforementioned driving information, A camera image acquisition unit that generates the driver's field of view image, A vehicle feature matching unit that associates vehicle features in the aforementioned visual image with vehicle feature information recorded in the vehicle feature storage unit, The system includes an information display control unit that sets an information display prohibited area in the field image based on the vehicle characteristic information, The information display control unit is characterized by setting a display area for the driving information outside the information display prohibited area and causing the information display unit to display the driving information.

2. A driving information display device according to claim 1, The information display control unit determines the position of the front window frame of the railway vehicle in the view image based on the vehicle characteristic information, and sets the inside of the front window frame of the railway vehicle as the information display prohibited area.

3. A driving information display device according to claim 1, The system further includes a feature matching unit that associates features in the aforementioned visual image with feature information recorded in the feature recording unit. The information display control unit determines the position of a feature in the field of view image based on the vehicle characteristic information and the feature information, and sets the vicinity of the position of the feature as the information display prohibited area.

4. A driving information display device according to claim 1, The information display control unit sets the information display prohibited area at predetermined intervals, and if there is no overlap between the information display prohibited area of ​​the latest period and the display area of ​​the operation information of the previous period, the display position of the operation information of the previous period is not changed.

5. A driving information display device according to claim 1, The driving information display device is characterized in that, when the information display control unit associates the vehicle characteristic information with the operating device in the view image, it determines the display position of the operating device in the view image, sets the vicinity of the display position of the operating device as an information display priority area, and, if the information display priority area does not overlap with the information display prohibition area, the information display unit displays the driving information in the information display priority area.

6. A railway vehicle equipped with a driving information display device according to any one of claims 1 to 5.

7. In a method for displaying driving information to a railway vehicle driver, The information acquisition unit acquires the aforementioned operating information, The camera image acquisition unit generates the driver's field of view image, The vehicle feature matching unit associates the vehicle features in the view image with the vehicle feature information recorded in the vehicle feature storage unit. The information display control unit sets an information display prohibition area in the field of view image based on the vehicle characteristic information, and sets a display area for the driving information outside the information display prohibition area. A method for displaying driving information, characterized in that the driving information is displayed by an information display unit.

8. In a train information control device that controls the display of an information display unit that shows train information to the train driver, An information acquisition unit that acquires the aforementioned driving information, A vehicle feature matching unit receives the driver's field of view image and associates the vehicle features in the field of view image with the vehicle feature information recorded in the vehicle feature recording unit. The system includes an information display control unit that sets an information display prohibited area in the field image based on the vehicle characteristic information, The information display control unit is characterized by setting a display area for the driving information outside the information display prohibited area and causing the information display unit to display the driving information.

9. A driving information control device according to claim 8, The information display control unit is characterized by determining the position of the front window frame of the railway vehicle in the view image based on the vehicle characteristic information, and setting the inside of the front window frame of the railway vehicle as the information display prohibited area.

10. A driving information control device according to claim 8, The system further includes a feature matching unit that associates features in the aforementioned visual image with feature information recorded in the feature recording unit. The information display control unit is characterized by determining the position of a feature in the view image based on the vehicle characteristic information and the feature information, and setting the vicinity of the position of the feature as the information display prohibited area.

11. A driving information control device according to claim 8, The information display control unit sets the information display prohibited area at predetermined intervals, and if there is no overlap between the information display prohibited area of ​​the latest period and the display area of ​​the operation information of the previous period, the operation information control unit does not change the display position of the operation information of the previous period.

12. A driving information control device according to claim 8, The driving information control device is characterized in that, when the information display control unit associates the vehicle characteristic information with the operating device in the view image, it determines the display position of the operating device in the view image, sets the vicinity of the display position of the operating device as an information display priority area, and, if the information display priority area does not overlap with the information display prohibition area, causes the display device to display the driving information in the information display priority area.

13. A railway vehicle equipped with a driving information control device according to any one of claims 8 to 12.

14. In a driving information control method that controls the display of an information display unit that displays driving information to the driver of a railway vehicle, The information acquisition unit acquires the aforementioned operating information, The vehicle feature matching unit receives the driver's view image and associates the vehicle features in the view image with the vehicle feature information recorded in the vehicle feature recording unit. A driving information control method characterized in that the information display control unit sets an information display prohibition area in the field of view image based on the vehicle characteristic information, sets a driving information display area outside the information display prohibition area, and displays the driving information on the information display unit.

Images