Driving information display device, driving information display method, driving information control device, and driving information control method

Abstract

Provided is a technique for displaying driving-related information to a driver of a railway vehicle at an appropriate position by using AR technology. This driving information display device, which solves the problem described above, comprises an information acquisition unit that acquires driving information, an information display unit that displays the driving information, a camera image acquisition unit that generates an image of the field of view of a driver, a vehicle feature matching unit that associates a vehicle feature in the field-of-view image with vehicle feature information recorded in a vehicle feature storage unit, and an information display control unit that sets an information display prohibition area in the field-of-view image on the basis of the vehicle feature information. The information display control unit sets a display area for the driving information outside the information display prohibition area and causes the information display unit to display the driving information. The information display control unit may obtain the position of a window frame on the front surface of the railway vehicle in the field-of-view image on the basis of the vehicle feature information, and set the inner side of the window frame on the front surface of the railway vehicle as the information display prohibition area.

Description

Operation Information Display Device, Operation Information Display Method, Operation Information Control Device, and Operation Information Control Method

[0001] The present invention relates to an operation information display device, an operation information display method, an operation information control device, and an operation information control method for providing operation information to a driver during the operation of a railway vehicle.

[0002] During the operation of a railway vehicle, the driver performs driving operations while simultaneously recognizing and processing various operation information inside and outside the vehicle. Generally, the operation information that the driver should recognize is classified into information related to ground-side facilities and information related to the vehicle state. The driver visually recognizes such information during operation. For example, for information related to ground-side facilities, the driver visually recognizes it through the glass in front of the vehicle, and for information related to the vehicle state, the driver visually recognizes it by looking at the instruments installed on the driver's cab. The driver needs to continuously monitor the outside of the vehicle in front on the running route from a safety perspective, but in order to recognize information related to the vehicle state, the driver has to shift the line of sight to the instruments inside the vehicle, and there is a problem that this line-of-sight movement places a burden on the driver.

[0003] Here, in order to assist in recognizing operation information inside and outside the vehicle during operation, it is known to use a technology called AR (Augmented Reality). The AR technology is a technology that combines digital information such as computer graphics with images and videos of the real world to create a new expression. By using the AR technology, it is possible to additionally provide digital information that does not exist in the real world in a form of being superimposed on the information of the real world to the target person who uses a display device such as an HMD (Head Mount Display).

[0004] As an example of presenting information related to ground-side facilities to the driver using AR technology, Patent Document 1 discloses a technology for extracting ground-side facilities from a camera image that captures the driver's field of view and installing an area for displaying information related to the ground-side facilities at a coordinate position within an angle of view that does not overlap with the ground-side facilities.

[0005] Japanese Unexamined Patent Application Publication No. 2022-46276

[0006] In the future, it is predicted that railway vehicle operation will increasingly involve drivers with less stringent qualifications than currently required, or even crew members without qualifications at all, as well as driverless, automated operation. On the other hand, even with automated operation, crew members may still take over in emergencies such as accidents. Therefore, in the future, there will be an even greater need to provide driving information in a more easily understandable format, even to those who do not possess the qualifications of a licensed driver.

[0007] However, as mentioned above, the technology disclosed in Patent Document 1 is limited to displaying only information related to ground-side equipment as driving information using AR technology, and is based on the premise of displaying AR objects in the scenery in front of the railway vehicle. Therefore, it is not satisfactory in terms of both the content of the driving information displayed and the display location.

[0008] Therefore, the present invention aims to provide a technology that uses AR technology to display driving-related information to the driver in an appropriate location. In particular, it aims to provide a driving information display device that presents driving-related information in a way that is easy for the driver to recognize, while avoiding interference with information related to ground-side equipment.

[0009] To solve the above problems, one representative driving information display device of the present invention comprises an information acquisition unit that acquires driving information, an information display unit that displays the driving information, a camera image acquisition unit that generates a driver's field of view image, a vehicle feature matching unit that associates vehicle features in the field of view image with vehicle feature information recorded in a vehicle feature storage unit, and an information display control unit that sets an information display prohibited area in the field of view image based on the vehicle feature information, the information display control unit sets a display area for driving information outside the information display prohibited area and causes the information display unit to display the driving information.

[0010] According to the present invention, it becomes possible to display driving information to the driver in an appropriate location using AR technology. Other issues, configurations, and effects will be clarified by the following description of embodiments for implementation.

[0011] Figure 1 shows an example of the configuration of the operation information display device of Example 1. Figure 2 shows an example of the processing flow of the operation information display device of Example 1. Figure 3 shows an example of the information display of the operation information display device of Example 1. Figure 4 shows an example of the processing flow of the operation information display device of Example 2. Figure 5 shows an example of the information display of the operation information display device of Example 2. Figure 6 shows an example of the processing flow of the operation information display device of Example 3. Figure 7 shows an example of the information display of the operation information display device of Example 3. Figure 8 shows an example of the processing flow of the operation information display device of Example 4. Figure 9 shows an example of the information display of the operation information display device of Example 4. Figure 10 shows an example of the configuration of the operation information display device of Example 5. Figure 11 shows an example of the processing flow of the operation information display device of Example 5.

[0012] Embodiments of the present invention will be described below with reference to the drawings. However, the present invention is not limited to these embodiments.

[0013] First, let's explain the terms commonly used in this embodiment. (Driving Information) In this disclosure, driving information means information related to the operation of a railway vehicle that the driver should refer to while driving. For example, this includes the speed of the railway vehicle, its position, the names of the stations it stops at, the distance to the stations, the status of the master controller operation, the opening and closing of the vehicle doors, and the temperature inside the vehicle.

[0014] (Local Objects) In this disclosure, "local objects" means external objects that the driver should visually observe while driving, i.e., objects that are the subject of forward monitoring. Examples include signals, tracks, and station platforms.

[0015] (Vehicle Features) In this disclosure, "vehicle features" means equipment that characterizes a railway vehicle and is installed around the driver's seat. For example, this includes the window frames on the front of the vehicle, control devices, monitors, instruments, etc.

[0016] <Configuration of the Driving Information Display Device in Example 1> The configuration of the driving information display device in Example 1 will be explained using Figure 1. Figure 1 is a diagram showing an example of the configuration of the driving information display device in Example 1.

[0017] The driving information display device in this embodiment includes a camera image acquisition unit 101, a feature matching unit 102, a vehicle feature matching unit 103, an information acquisition unit 104, an information display unit 105, an information display control unit 106, a feature recording unit 111, and a vehicle feature recording unit 112 as its components. In addition, information such as driver's view image 121, feature information 122, vehicle feature information 123, feature information within the image 124, vehicle feature information within the image 125, vehicle in operation information 126, and train position information 127 are exchanged between each component. Next, the details of the information exchanged with each component will be described.

[0018] (Camera Image Acquisition Unit) The camera image acquisition unit 101 is a camera means that optically acquires information from the outside world and has the function of generating a driver's field of view image 121. Here, the driver's field of view image 121 is an image generated by the camera image acquisition unit 101 that corresponds to the range of the driver's actual field of view. Examples of implementations of the camera image acquisition unit 101 include a camera for acquiring field of view images mounted on a head-mounted display or camera-type device, a camera worn on the driver's head, and a camera installed inside the vehicle. The driver's field of view image 121 generated by the camera image acquisition unit 101 is output to the feature matching unit 102, the vehicle feature matching unit 103, the information display control unit 106, etc.

[0019] (Feature Recording Unit) The feature recording unit 111 includes a database of features that the driver should visually confirm while driving. The database records feature information 122. Here, feature information 122 is information that uniquely identifies a feature, such as its type, absolute position (latitude, longitude, and kilometer distance are available), shape, and color. Various recording media such as HDDs, SSDs, and SD cards can be used as the feature recording unit 111. The feature information 122 is output from the feature recording unit 111 to the feature matching unit 102. Note that the feature information 122 may change or be modified over time or depending on the season, so the feature recording unit 111 is updated at appropriate intervals.

[0020] (Feature Matching Unit) The feature matching unit 102 has the function of extracting features in the driver's view image 121 and associating them with feature information 122 from the feature recording unit 111. This association is performed using feature information 122 input from the feature recording unit 111, the driver's view image 121 input from the camera image acquisition unit 101, and train position information 127 input from the information acquisition unit 104. The result of the association is output to the information display control unit 106 as in-image feature information 124. Here, in-image feature information 124 includes the contents of feature information 122, as well as the coordinate position, shape, and color within the driver's view image 121.

[0021] Furthermore, there are no particular limitations on the technique used by the feature matching unit 102 to extract features within the driver's view image 121. For example, machine learning can be used to learn the shape and color data of features and recognize the location of features within the driver's view image 121, or pattern matching can be used to determine areas with a high degree of agreement with the shape and color of features within the driver's view image 121. In addition, the relative position of features relative to the railway vehicle can be calculated from the position of the railway vehicle in operation obtained from the train position information 127 and the position of the features obtained from the feature information 122, and features within the driver's view image 121 can be recognized based on this.

[0022] (Vehicle Feature Recording Unit) The vehicle feature recording unit 112 includes a database of vehicle features installed around the driver's seat. The database records vehicle feature information. Here, vehicle feature information refers to information that characterizes a railway vehicle, such as the type of vehicle feature, the three-dimensional position with the inside of the vehicle as the origin, shape, and color. Various recording media such as HDDs, SSDs, and SD cards can be used for the vehicle feature recording unit 112. The vehicle feature information 123 is output from the vehicle feature recording unit 112 to the vehicle feature matching unit 103.

[0023] (Vehicle Feature Matching Unit) The vehicle feature matching unit 103 has the function of extracting vehicle features from the driver's view image 121 and associating them with the vehicle feature information 123 from the vehicle feature recording unit 112. This association is performed using the vehicle feature information 123 input from the vehicle feature recording unit 112 and the driver's view image 121 input from the camera image acquisition unit 101. The result of the association is output to the information display control unit 106 as in-image vehicle feature information 125. Here, in-image vehicle feature information 125 includes the contents of the vehicle feature information 123, as well as the coordinate position, shape, color, etc., within the driver's view image 121.

[0024] Furthermore, there are no particular limitations on the technique used by the vehicle feature matching unit 103 to extract vehicle features from the driver's view image 121. It is also possible to use the extraction technique described in the above-mentioned feature matching unit 102.

[0025] (Information Acquisition Unit) The information acquisition unit 104 has the function of creating in-train vehicle information 126 and train position information 127 using data from various devices installed on the vehicle. Here, the devices installed on the vehicle may include, for example, motors, brakes, air conditioning inside the vehicle, vehicle doors, speed generators, and vehicle information control devices that comprehensively control on-board equipment. The in-train vehicle information 126 includes driving information that the driver refers to during operation, such as the vehicle's running position, speed, stopping stations, distance to stopping stations, master controller operation status, vehicle door open / closed status, and interior temperature. The in-train vehicle information 126 is output to the information display control unit 106. In addition, the vehicle's running position from the in-train vehicle information 126 is output to the terrain matching unit 102 as train position information 127.

[0026] (Information display unit) The information display unit 105 has the function of displaying information in an appropriate area within the driver's field of view 301. Examples of implementations of the information display unit 105 include an HMD worn by the driver and a glasses-type head-up display.

[0027] (Information Display Control Unit) The information display control unit 106 has the function of displaying driving information in the information display unit 105 at an appropriate position within the driver's field of view, based on the image feature information 124 input from the feature matching unit 102, the image vehicle feature information 125 input from the vehicle feature matching unit 103, and the driving vehicle information 126 input from the information acquisition unit 104.

[0028] Each processing unit and recording unit in Figure 1 may be integrated into a single device, with each function implemented as software, or they may be separate devices that communicate with each other to achieve their functions. Furthermore, while the camera image acquisition unit 101 and information display unit 105, which serve as interfaces for the driver, are installed inside the vehicle or on the driver's body, the installation location of the other processing units and recording units is not restricted as long as they can communicate with each other.

[0029] <Processing Flow of the Driving Information Display Device in Example 1> The processing flow of the driving information display device in Example 1 will be explained using Figure 2. Figure 2 is a diagram showing an example of the processing flow of the driving information display device in Example 1. The processing flow in Figure 2 is executed periodically during the operation of the information display control unit 106. The operation period may be the same as the display update period of the information display control unit 106.

[0030] In process 201, the driver's field of view image 121 is acquired from the camera image acquisition unit 101. The acquired driver's field of view image 121 is used in processes 202 to 206. In this embodiment, the camera image acquisition unit 101 is assumed to be a camera for acquiring field of view images mounted on a glasses-type head-mounted display.

[0031] In process 202, the feature matching unit 102 associates the feature information 122 recorded in the feature recording unit 111 with the features extracted from the driver's view image 121 acquired in process 201 to create in-image feature information 124. The created in-image feature information 124 is used in process 204.

[0032] In process 203, the vehicle feature matching unit 103 associates the vehicle features extracted from the driver's view image 121 acquired in process 201 with the vehicle feature information 123 recorded in the vehicle feature recording unit 112 to create in-image vehicle feature information 125. The created in-image vehicle feature information 125 is used in process 204. If no vehicle features are extracted from the driver's view image 121 in a certain cycle, the vehicle features extracted in the previous cycle may be used to perform processes 204 and beyond.

[0033] In process 204, the information display control unit 106 estimates the position and orientation of the camera image acquisition unit 101 when acquiring the driver's view image 121. There are no particular limitations on how the position and orientation of the camera image acquisition unit 101 are estimated. For example, the three-dimensional position and shape of the vehicle features, with the vehicle interior as the origin, included in the vehicle feature information 125 input from process 203, may be compared with the position and shape of the vehicle features in the driver's view image 121, and the position and orientation of the camera image acquisition unit 101 may be calculated such that the two match. The calculated position and orientation may be expressed using the three-dimensional position with the vehicle interior as the origin and the orientation angles (roll, pitch, yaw) for each axis, or the orientation may be expressed using quaternions instead of angles. The calculated position and orientation of the camera image acquisition unit 101 are used in process 205. It is also possible to calculate the position and orientation of the camera image acquisition unit 101 using the in-image feature information 124 and train position information 127 input from process 202.

[0034] In process 205, the information display control unit 106 first calculates the position of the front window frame within the driver's view image 121 from the position and orientation of the camera image acquisition unit 101 estimated in process 204 and the three-dimensional position of the front window frame of the vehicle, recorded in the vehicle feature recording unit 112 with the vehicle interior as the origin. Next, based on the calculated position of the front window frame of the vehicle, the system identifies the interior image area (the area outside the front window frame) and the exterior image area (the area inside the front window frame) within the driver's view image 121, and sets the exterior image area as an information display prohibited area. The information display prohibited area is used in process 206. Note that the information display prohibited area may be set for each pixel in the driver's view image 121.

[0035] In process 206, the information display control unit 106 sets an information prohibition area in the driver's field of view corresponding to the information prohibition area within the driver's field of view image 121, and the information display unit 105 displays (projects) predetermined driving information from the vehicle information 126 outside this area. Here, the display area for the driving information is outside the information prohibition area and is the closest position to the center of the driver's field of view image 121. Furthermore, the background color of the display area should be set to a color with a large hue difference from the surrounding color to improve the visibility of the information. After process 206, the process returns to process 201 and proceeds to the next cycle.

[0036] In this embodiment, the position of the front window frame of the vehicle within the driver's field of view image 121 was determined in process 205 based on the position and orientation of the camera image acquisition unit obtained in process 204. However, the position of the front window frame of the vehicle may also be determined directly by performing pattern matching in the vehicle feature matching unit 103.

[0037] <Information Display of the Driving Information Display Device in Example 1> The information display of the driving information display device in Example 1 will be explained using Figure 3. Figure 3 is a diagram showing an example of the information display of the driving information display device in Example 1.

[0038] The field of view 301 indicates the driver's field of view, and the driver's field of view image 121 acquired by the camera image acquisition unit 101 is the corresponding image. Based on the relative relationship between the position of the front window frame 302 of the vehicle in the field of view 301 and the position of the front window frame of the vehicle in the driver's field of view image 121 calculated in process 205 of Figure 2, an area corresponding to the information display prohibited area in the driver's field of view image 121 set in process 205 is set within the field of view 301. In this embodiment, since the inside of the front window frame 302 of the vehicle is set as the information display prohibited area, the driving information display area 304 is projected outside the front window frame 302 of the vehicle and close to the center of the field of view 301.

[0039] This allows driving-related information to be displayed in a position that does not interfere with the visibility of objects being monitored ahead.

[0040] The difference between Example 2 and Example 1 lies in the location where the information display prohibited area is set. In Example 2, an information display prohibited area is set around the ground object that is the target of forward monitoring.

[0041] <Example 2's Driving Information Display Device's Processing Flow> The flow of the driving information display device in Example 2 will be described using FIG. 4. FIG. 4 is a diagram showing an example of the processing flow of the driving information display device in Example 2. In FIG. 4, processing 405 is added in place of processing 205 in FIG. 2. Since the other processes are the same as those in Example 1, the description thereof is omitted.

[0042] In processing 405, in the information display control unit 106, based on the position and orientation of the camera image acquisition unit 101 estimated in processing 204 and the ground object information 124 in the image created in processing 202, the position of the ground object in the driver's field of view image 121 is calculated. At the same time, by the same process as processing 205 in Example 1, the position of the window frame in front of the vehicle in the driver's field of view image 121 is also calculated. In this embodiment, based on the calculated position of the ground object, the surrounding area of the ground object in the driver's field of view image 121 is set as the information display prohibited area. The information display prohibited area is used in processing 206.

[0043] In the subsequent processing 206, in the information display control unit 106, an information display prohibited area at the driver's viewing angle corresponding to the information display prohibited area in the driver's field of view image 121 is set, and in the information display unit 105, a predetermined driving information among the in-vehicle driving information 126 is displayed (projected) outside this information display prohibited area. Here, as long as it does not overlap with the set information display prohibited area, the display area of the driving information gives priority to the inside of the window frame in front of the vehicle.

[0044] <Example 2's Driving Information Display Device's Information Display> The information display of the driving information display device in Example 2 will be described using FIG. 5. FIG. 5 is a diagram showing an example of the information display of the driving information display device in Example 2.

[0045] The viewing angle 301 indicates the field of view of the driver, and the driver's field of view image 121 acquired by the camera image acquisition unit 101 is an image corresponding thereto. Based on the relative relationship between the position of the ground object 303 (traffic signal) in the viewing angle 301 and the position of the ground object (traffic signal) in the driver's field of view image 121 calculated in the process 405 of FIG. 4, an information display prohibited area within the driver's field of view image 121 is set within the viewing angle 301. In this embodiment, since the periphery of the ground object 303 (traffic signal) is set as the information display prohibited area 505, the display area 504 of the driving information is projected within the window frame 302 in front of the vehicle that does not overlap with this.

[0046] In this embodiment, one ground object (traffic signal) 303 is taken up and described for setting the information display prohibited area, but it is not limited to this, and it is also possible to set a plurality of ground objects. For example, although not described in detail, FIG. 5 shows the result of considering a ground object (railway line) for setting the information display prohibited area.

[0047] As a result, the driving information can be displayed at a position that does not interfere with the visual recognition of the ground object that is the target of forward monitoring, and since the movement of the driver's line of sight can be completed within the window frame, it leads to a reduction in the driver's burden.

[0048] What is different between Embodiment 3 and Embodiment 1 is that processing is added so that the display area of the driving information does not change as much as possible even if the driver's posture (viewing angle) or the like changes.

[0049] <Processing Flow of the Driving Information Display Device of Embodiment 3> The flow of the driving information display device in Embodiment 3 will be described with reference to FIG. 6. FIG. 6 is a diagram showing an example of the processing flow of the driving information display device of Embodiment 3. In FIG. 6, processes 607 to 609 are added before and after the process 206 of FIG. 2. Since the other processes are the same as those in Embodiment 1, the description thereof is omitted.

[0050] In process 607, the display area of the driving information set in the previous cycle's processing flow and the information display prohibited area set in the current cycle's processing flow are compared to determine whether there is an overlapping area between the two. If there is an overlapping area between the two, the process proceeds to process 206, and if not, the process proceeds to process 608.

[0051] In process 608, the display area for the operating information set in the processing flow of the previous cycle, which is saved in process 609, is applied as the display area for the operating information set in the processing flow of the current cycle. After process 608, the process proceeds to process 609.

[0052] Process 609 is a process that saves the display area of ​​the operating information set in the processing flow of the current cycle until the next cycle. The display area of ​​the operating information saved here is used in process 607 of the next cycle.

[0053] <Information Display of the Driving Information Display Device in Example 3> An example of the information display of the driving information display device in Example 3 will be explained using Figure 7. Figure 7 is a diagram showing an example of the information display of the driving information display device in Example 3.

[0054] Figure 7 shows a case where the driver's posture shifts forward, causing the window frame 302 at the front of the vehicle to occupy a larger portion of the field of view 301 than in Figure 2. Due to this change in the field of view 301, the display area 304 for driving information, which should have been outside the information display prohibition area (outside the window frame 302 at the front of the vehicle in Figure 2) in the previous cycle, now overlaps with the information display prohibition area (inside the window frame 703 at the front of the vehicle in Figure 7) in the current cycle.

[0055] In this embodiment, in such cases, the display area 704 of the updated driving information in process 608 is reset to a position that does not overlap with the information display prohibited area for the current cycle. On the other hand, even if there is a change in the driver's posture, if the display area 304 of the driving information from the previous cycle does not overlap with the information display prohibited area for the current cycle, the display area 304 of the driving information from the previous cycle is maintained without changing the position of the display area.

[0056] This allows driving-related information to be continuously displayed in a fixed position within the driver's field of view, as long as it does not overlap with the area where information display is prohibited. This reduces the driver's eye movement and alleviates their burden.

[0057] The difference between Example 4 and Example 1 is that Example 4 adds a process to set the display area near the operating device related to driving information.

[0058] <Processing Flow of the Driving Information Display Device in Example 4> The flow of the driving information display device in Example 4 will be explained using Figure 8. Figure 8 is a diagram showing an example of the processing flow of the driving information display device in Example 4. In Figure 8, processes 807 to 809 are added between processes 205 and 206 in Figure 2. The other processes are the same as in Example 1, so their explanation will be omitted.

[0059] In process 807, the information display control unit 106 calculates the position of the control device within the driver's field of view image 121 from the position and orientation of the camera image acquisition unit 101 estimated in process 204 and the vehicle feature information 125 (three-dimensional position, etc.) of the control device within the image created in process 203, and sets the area around the control device as an information display priority area. The set information display priority area is used in process 808.

[0060] In process 808, the information display priority area set in process 807 and the information display prohibition area set in process 205 are compared to determine whether there are any overlapping areas between them. If there are overlapping areas between them, the process proceeds to process 206; otherwise, the process proceeds to process 809.

[0061] In process 809, the display area for driving information is set in a position that does not overlap with the operating equipment within the information display priority area set in process 807, and this is reflected in the driver's field of view 301 to display the driving information. After process 809, the process proceeds to the next cycle.

[0062] <Information Display of the Driving Information Display Device in Example 4> The information display of the driving information display device in Example 4 will be explained using Figure 9. Figure 9 is a diagram showing an example of the information display of the driving information display device in Example 4.

[0063] In this embodiment, the display area 904 for driving information is projected into an information display priority area 906 set around the operating device 905 within the driver's field of view 301.

[0064] This allows driving-related information to be displayed in a location close to the control devices, making it easier to intuitively understand the content of the driving information and reducing the burden on the driver.

[0065] The difference between Example 5 and Example 1 is that, even if the orientation of the camera image acquisition unit 101 does not change, if the driver's line of sight changes, a process is added to change the field of view angle 301 to match that orientation.

[0066] <Configuration of the Driving Information Display Device in Example 5> The configuration of the driving information display device in Example 5 will be explained using Figure 10. Figure 10 is a diagram showing an example of the configuration of the driving information display device in Example 5. In Figure 10, a gaze direction acquisition unit 1000 and gaze direction information 1020 are added to the configuration in Figure 1, and a camera image acquisition unit 1001 is added in place of the camera image acquisition unit 101 in Figure 1. The other configurations are the same as in Example 1, so their explanation will be omitted.

[0067] (Eye Direction Acquisition Unit) The eye direction acquisition unit 1000 has the function of acquiring eye direction information 1020 from the direction of the driver's eyes. Here, the eye direction information 1020 is a three-dimensional vector based on the area in front of the driver's head, and is composed of three elements: roll, pitch, and yaw.

[0068] The method for detecting the direction of the gaze is not particularly limited. For example, a method that detects the direction of the eyeball using a visible light camera or a method that detects the position of the pupil and corneal reflection using an infrared light camera may be used. The gaze direction information 1020 acquired by the gaze direction acquisition unit 1000 is output to the camera image acquisition unit 1001. The gaze direction acquisition unit 1000 may be worn on the driver's head or installed inside the vehicle.

[0069] (Camera Image Acquisition Unit) The camera image acquisition unit 1001 has the function of acquiring a driver's field of view image 121 in which the driver's line of sight is at the center of the image, based on the eye direction information 1020. Other functions are the same as those of the camera image acquisition unit 101 in Embodiment 1.

[0070] <Processing Flow of the Driving Information Display Device in Example 5> The flow of the driving information display device in Example 5 will be explained using Figure 11. Figure 11 is a diagram showing an example of the processing flow of the driving information display device in Example 5. In Figure 11, processes 1100 and 1101 are added before process 202 in Figure 2.

[0071] In process 1100, the eye-line direction acquisition unit 1000 acquires the driver's eye-line direction information 1020. The eye-line direction information 1020 is used in process 1101.

[0072] In process 1101, the camera image acquisition unit 1001 uses the gaze direction information 1020 acquired in process 1100 to acquire a driver's field of view image 121 centered on the driver's gaze direction. The driver's field of view image 121 is used in processes 202 to 206.

[0073] This allows for the display of driving information to follow the driver's line of sight, thus reducing the driver's workload.

[0074] <Modifications> Although embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above, and various modifications are possible without departing from the spirit of the present invention.

[0075] For example, in the embodiment described above, the driving information display device is configured to include all the processing and storage units shown in Figures 1 and 10, but it is not limited to this configuration, and the feature recording unit, feature matching unit, and vehicle feature recording unit can also be configured as external devices.

[0076] Furthermore, among the driving information display devices of the embodiments described above, it is also possible to configure the information acquisition unit, vehicle feature matching unit, and information display control unit related to the display control of driving information as a single driving information control device. The following embodiments are included in this disclosure.

[0077] (Aspect 1) A driving information display device for displaying driving information to a driver of a railway vehicle, comprising: an information acquisition unit for acquiring the driving information; an information display unit for displaying the driving information; a camera image acquisition unit for generating a driver's field of view image; a vehicle feature matching unit for associating vehicle features in the field of view image with vehicle feature information recorded in a vehicle feature storage unit; and an information display control unit for setting an information display prohibited area in the field of view image based on the vehicle feature information, wherein the information display control unit sets a display area for the driving information outside the information display prohibited area and causes the information display unit to display the driving information.

[0078] (Aspect 2) An information display device according to the present invention, wherein the information display control unit determines the position of the front window frame of the railway vehicle in the view image based on the vehicle characteristic information, and sets the inside of the front window frame of the railway vehicle as the information display prohibited area.

[0079] (Aspect 3) A driving information display device according to either aspect 1 or 2, further comprising a feature matching unit that associates features in the view image with feature information recorded in a feature recording unit, wherein the information display control unit determines the position of features in the view image based on the vehicle feature information and the feature information, and sets the vicinity of the position of the features as the information display prohibited area.

[0080] (Aspect 4) An operation information display device according to any one of aspects 1 to 3, wherein the information display control unit sets the information display prohibited area at a predetermined interval, and if there is no overlap between the information display prohibited area of ​​the latest interval and the display area of ​​the operation information of the previous interval, the display position of the operation information of the previous interval is not changed.

[0081] (Aspect 5) A driving information display device according to any one of aspects 1 to 4, wherein the information display control unit, when the vehicle characteristic information is associated with the operating device in the view image, determines the display position of the operating device in the view image, sets the vicinity of the display position of the operating device as an information display priority area, and, if the information display priority area does not overlap with the information display prohibition area, causes the information display unit to display the driving information in the information display priority area.

[0082] (Aspect 6) A driving information display device according to any one of aspects 1 to 5, further comprising a gaze direction acquisition unit that acquires the direction of the driver's gaze, wherein the camera image acquisition unit generates the field of view image according to the direction of the gaze.

[0083] (Aspect 7) An information display device according to any one of aspects 1 to 6, wherein the information display control unit sets a color that has a large hue difference from the surrounding background color of the display area as the background color of the display area of ​​the information display area.

[0084] (Aspect 8) A railway vehicle equipped with a driving information display device as described in any of aspects 1 to 7.

[0085] (Aspect 9) A method for displaying driving information to a railway vehicle driver, characterized in that: an information acquisition unit acquires the driving information; a camera image acquisition unit generates a view image of the driver; a vehicle feature matching unit associates vehicle features in the view image with vehicle feature information recorded in a vehicle feature storage unit; an information display control unit sets an information display prohibited area in the view image based on the vehicle feature information, and sets a display area for the driving information outside the information display prohibited area; and an information display unit displays the driving information.

[0086] (Aspect 10) A driving information control device that controls the display of an information display unit that displays driving information to a driver of a railway vehicle, comprising: an information acquisition unit that acquires the driving information; a vehicle feature matching unit that receives a view image of the driver and associates the vehicle features in the view image with vehicle feature information recorded in a vehicle feature recording unit; and an information display control unit that sets an information display prohibited area in the view image based on the vehicle feature information, wherein the information display control unit sets a display area for the driving information outside the information display prohibited area and causes the display unit to display the driving information.

[0087] (Aspect 11) An information control device according to the description in Aspect 10, wherein the information display control unit determines the position of the front window frame of the railway vehicle in the view image based on the vehicle characteristic information, and sets the inside of the front window frame of the railway vehicle as the information display prohibited area.

[0088] (Aspect 12) A driving information control device according to either aspect 10 or 11, further comprising a feature matching unit that associates features in the view image with feature information recorded in a feature recording unit, wherein the information display control unit determines the position of features in the view image based on the vehicle feature information and the feature information, and sets the vicinity of the position of the features as the information display prohibited area.

[0089] (Aspect 13) An operation information control device according to any one of aspects 10 to 12, wherein the information display control unit sets the information display prohibited area at a predetermined interval, and if there is no overlap between the information display prohibited area of ​​the latest interval and the display area of ​​the operation information of the previous interval, the display position of the operation information of the previous interval is not changed.

[0090] (Aspect 14) A driving information control device according to any one of aspects 10 to 13, wherein the information display control unit, when the vehicle characteristic information is associated with the operating device in the view image, determines the display position of the operating device in the view image, sets the vicinity of the display position of the operating device as an information display priority area, and, if the information display priority area does not overlap with the information display prohibition area, causes the display device to display the driving information in the information display priority area.

[0091] (Aspect 15) An operation information control device according to any one of aspects 10 to 14, wherein the information display control unit is characterized in that it sets a color with a large hue difference from the surrounding background color as the background color of the display area of ​​the operation information.

[0092] (Aspect 16) A railway vehicle equipped with the driving information control device described in any of aspects 1 to 15.

[0093] (Aspect 17) A driving information control method for controlling the display of an information display unit that displays driving information to a driver of a railway vehicle, characterized in that: an information acquisition unit acquires the driving information; a vehicle feature matching unit receives a view image of the driver, associates the vehicle features in the view image with vehicle feature information recorded in a vehicle feature recording unit; and an information display control unit sets an information display prohibited area in the view image based on the vehicle feature information, sets a display area for the driving information outside the information display prohibited area, and displays the driving information on the display unit.

[0094] 101: Camera image acquisition unit, 102: Feature matching unit, 103: Vehicle feature matching unit, 104: Information acquisition unit, 105: Information display unit, 106: Information display control unit, 111: Feature recording unit, 112: Vehicle feature recording unit, 121: Driver's view image, 122: Feature information, 123: Vehicle feature information, 124: Feature information in image, 125: Vehicle feature information in image, 126: Vehicle information during operation, 127: Train position information, 301: Field of view, 302, 703: Window frame on the front of the railway vehicle, 303: Feature (signal), 304, 504, 704, 904: Display area for driving information, 505: Information display prohibited area, 905: Operating equipment, 906: Information display priority area, 1000: Eye direction acquisition unit, 1001: Camera image acquisition unit, 1020: Eye direction information

Claims

1. A driving information display device for displaying driving information to a driver of a railway vehicle, comprising: an information acquisition unit for acquiring the driving information; an information display unit for displaying the driving information; a camera image acquisition unit for generating a driver's field of view image; a vehicle feature matching unit for associating vehicle features in the field of view image with vehicle feature information recorded in a vehicle feature storage unit; and an information display control unit for setting an information display prohibited area in the field of view image based on the vehicle feature information, wherein the information display control unit sets a display area for the driving information outside the information display prohibited area and causes the information display unit to display the driving information.

2. A driving information display device according to claim 1, wherein the information display control unit determines the position of the front window frame of the railway vehicle in the view image based on the vehicle characteristic information, and sets the inside of the front window frame of the railway vehicle as the information display prohibited area.

3. A driving information display device according to claim 1 or 2, further comprising a feature matching unit that associates features in the view image with feature information recorded in a feature recording unit, wherein the information display control unit determines the position of features in the view image based on the vehicle feature information and the feature information, and sets the vicinity of the position of the features as the information display prohibited area.

4. An operating information display device according to any one of claims 1 to 3, wherein the information display control unit sets the information display prohibited area at a predetermined interval, and if there is no overlap between the information display prohibited area of ​​the latest interval and the display area of ​​the operating information of the previous interval, the display position of the operating information of the previous interval is not changed.

5. A driving information display device according to any one of claims 1 to 4, wherein the information display control unit, when the vehicle characteristic information is associated with the operating device in the view image, determines the display position of the operating device in the view image, sets the vicinity of the display position of the operating device as an information display priority area, and, if the information display priority area does not overlap with the information display prohibition area, causes the information display unit to display the driving information in the information display priority area.

6. A railway vehicle equipped with a driving information display device according to any one of claims 1 to 5.

7. A method for displaying driving information to a railway vehicle driver, characterized in that: an information acquisition unit acquires the driving information; a camera image acquisition unit generates a view image of the driver; a vehicle feature matching unit associates vehicle features in the view image with vehicle feature information recorded in a vehicle feature storage unit; an information display control unit sets an information display prohibited area in the view image based on the vehicle feature information, and sets a display area for the driving information outside the information display prohibited area; and an information display unit displays the driving information.

8. A driving information control device that controls the display of an information display unit that displays driving information to a driver of a railway vehicle, comprising: an information acquisition unit that acquires the driving information; a vehicle feature matching unit that receives a view image of the driver and associates the vehicle features in the view image with vehicle feature information recorded in a vehicle feature recording unit; and an information display control unit that sets an information display prohibited area in the view image based on the vehicle feature information, wherein the information display control unit sets a display area for the driving information outside the information display prohibited area and causes the information display unit to display the driving information.

9. A driving information control device according to claim 8, wherein the information display control unit determines the position of the front window frame of the railway vehicle in the view image based on the vehicle characteristic information, and sets the inside of the front window frame of the railway vehicle as the information display prohibited area.

10. A driving information control device according to claim 8 or 9, further comprising a feature matching unit that associates features in the view image with feature information recorded in a feature recording unit, wherein the information display control unit determines the position of features in the view image based on the vehicle feature information and the feature information, and sets the vicinity of the position of the features as the information display prohibited area.

11. An operation information control device according to any one of claims 8 to 10, wherein the information display control unit sets the information display prohibited area at a predetermined interval, and if there is no overlap between the information display prohibited area of ​​the latest interval and the display area of ​​the operation information of the previous interval, the display position of the operation information of the previous interval is not changed.

12. A driving information control device according to any one of claims 8 to 11, wherein the information display control unit, when the vehicle characteristic information is associated with the operating device in the view image, determines the display position of the operating device in the view image, sets the vicinity of the display position of the operating device as an information display priority area, and, if the information display priority area does not overlap with the information display prohibition area, causes the display device to display the driving information in the information display priority area.

13. A railway vehicle equipped with a driving information control device according to any one of claims 8 to 12.

14. A driving information control method for controlling the display of an information display unit that displays driving information to a driver of a railway vehicle, characterized in that: an information acquisition unit acquires the driving information; a vehicle feature matching unit receives the driver's field of view image, associates the vehicle features in the field of view image with vehicle feature information recorded in a vehicle feature recording unit; and an information display control unit sets an information display prohibited area in the field of view image based on the vehicle feature information, sets a display area for the driving information outside the information display prohibited area, and causes the information display unit to display the driving information.

Images