Projection control device and projection control method

Abstract

To prevent the viewer's gaze from being guided by objects unrelated to the destination or route when providing directions to a destination. [Solution] The projection control device 20 includes a route information acquisition unit 22 that acquires route information indicating at least one of the vehicle's destination or the route to the destination, a recognition unit 25 that recognizes objects unrelated to the destination and route from an image captured in front of the vehicle, and a projection control unit 29 that controls the projection of a mask image superimposed on the objects recognized by the recognition unit 25 onto the windshield.

Description

Technical Field

[0001] The present invention relates to a projection control device and a projection control method.

Background Art

[0002] There is known a technique for highlighting road sign information related to route guidance in a three-dimensional display so as to be seen overlapping with the corresponding real road sign information in the forward field of view (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a car navigation system, route guidance to a destination is output to a driver by voice, displayed on a screen, or displayed on a windshield by a head-up display or the like. However, when various kinds of information are displayed in the driver's field of view, it may be difficult for the driver to visually recognize signs, markings, etc. that should be confirmed in order to head towards the destination.

[0005] The present disclosure has been made in view of the above, and an object thereof is to suppress the line of sight from being guided by an object that is not related to the destination and the route during route guidance to the destination.

Means for Solving the Problems

[0006] To solve the above-mentioned problems and achieve the objective, the projection control device according to this disclosure includes: a route information acquisition unit that acquires route information indicating at least one of the vehicle's destination or the route to the destination; a recognition unit that recognizes objects unrelated to the destination and the route from an image captured in front of the vehicle; and a projection control unit that controls the projection of a mask image superimposed on the object recognized by the recognition unit onto the windshield.

[0007] The projection control method according to this disclosure is performed by a computer acting as a projection control device, which acquires route information indicating at least one of the vehicle's destination or the route to the destination, recognizes objects unrelated to the destination and the route from an image captured in front of the vehicle, and controls the projection of a mask image superimposed on the recognized objects onto the windshield. [Effects of the Invention]

[0008] According to this disclosure, the effect is to suppress the distraction of the user's gaze from being guided by objects unrelated to the destination or route when providing route guidance to a destination. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 is a block diagram showing an example configuration of a projection device according to the first embodiment. [Figure 2] Figure 2 is a schematic diagram showing an example of the driver's field of view when facing forward in a vehicle. [Figure 3] Figure 3 is a schematic diagram showing an example where the driver's field of view, facing forward, is masked, as shown in Figure 2. [Figure 4] Figure 4 is a schematic diagram illustrating another example of the driver's field of view when facing forward in a vehicle. [Figure 5] Figure 5 is a schematic diagram showing an example where the driver's field of view, facing forward, is masked, as shown in Figure 4. [Figure 6] Figure 6 is a flowchart showing an example of the processing flow in the projection device according to the first embodiment. [Figure 7] Figure 7 is a block diagram showing an example configuration of a projection device according to the second embodiment. [Figure 8] Figure 8 is a schematic diagram showing an example of the driver's field of view when facing forward in a vehicle. [Figure 9] Figure 9 is a schematic diagram showing an example where the driver's field of view, facing forward, is masked, as shown in Figure 8. [Modes for carrying out the invention]

[0010] Embodiments of the projection control device 20 and projection control method according to this disclosure will be described in detail below with reference to the attached drawings. However, the present invention is not limited to the following embodiments.

[0011] [First Embodiment] <Projection device> Figure 1 is a block diagram showing an example configuration of a projection device according to an embodiment. The projection device 1 is mounted on a vehicle. In addition to being mounted on a vehicle, the projection device 1 may also be a portable device that can be used in a vehicle. The projection device 1 projects a mask image onto the windshield that is superimposed on objects unrelated to the destination and route recognized from the image captured from the front of the vehicle.

[0012] Objects unrelated to the destination and route are signs and markings that are unnecessary for route guidance to the destination. Objects unrelated to the destination and route are hereinafter referred to as "objects". Objects are, for example, characters or symbols that indicate the destination. Objects are, for example, characters that indicate the names of intersections where turns are made on the route to the destination. Objects are, for example, characters corresponding to place names and intersection names read aloud in the voice guidance of a navigation system's route guidance. Objects are, for example, characters that allow the driver to recognize the destination or route (direction to the destination).

[0013] The mask video is a video that masks an object. The mask video is a video that masks an object so that a driver cannot recognize features such as the pattern or characters of the object. The mask video is, for example, a video of a figure according to the outer shape of the object in a single color. The mask video may be, for example, the same color as the background color of the object or another inconspicuous color.

[0014] The projection device 1 includes an external camera 11, a GNSS receiver 13, a projection unit 19, and a projection control device 20.

[0015] The external camera 11 images a range including the front of the vehicle. A plurality of external cameras 11 may be arranged. In the present embodiment, the external camera 11 is arranged in front of the vehicle and images a video of the periphery centered on the front of the vehicle. The video is, for example, a moving image composed of images at 30 frames per second. The external camera 11 outputs the imaged video to the video acquisition unit 21 of the projection control device 20.

[0016] The GNSS receiver 13 receives GNSS signals from GNSS satellites (not shown). The GNSS receiver 13 is composed of, for example, a GNSS receiver such as GPS.

[0017] The projection unit 19 includes, for example, a liquid crystal display element or an organic EL (Electro-Luminescence) display element, and projects a video onto the windshield. The projection unit 19 is arranged on the dashboard of the vehicle or the like. The projection unit 19 displays a display video on the display element based on a video signal from the projection control unit 29 of the projection control device 20. The video display light of the display video displayed on the display surface of the display element is projected onto the windshield through, for example, a concave mirror (not shown).

[0018] <Projection control device> The projection control device 20 is an arithmetic processing device composed of, for example, a CPU (Central Processing Unit), a video processing processor, etc. The projection control device 20 loads a program stored in a storage unit (not shown) into a memory and executes instructions included in the program. The projection control device 20 includes a video acquisition unit 21, a route information acquisition unit 22, a position information acquisition unit 23, a recognition unit 25, a calculation unit 27, and a projection control unit 29. The projection control device 20 includes an internal memory (not shown), and the internal memory is used for temporarily storing data in the projection control device 20. The projection control device 20 may be composed of one or a plurality of devices.

[0019] The video acquisition unit 21 acquires a video of an area including the front of the vehicle. More specifically, the video acquisition unit 21 acquires the video output by the out-of-vehicle camera 11.

[0020] The route information acquisition unit 22 acquires route information regarding the route from the departure point to the destination. The route information acquisition unit 22 acquires the route information from the navigation system.

[0021] The route information includes map information corresponding to the route to the destination of the vehicle. The map information is, for example, information of a road map including intersections.

[0022] The recognition unit 25 recognizes an object from the video of the front of the vehicle. In the present embodiment, the recognition unit 25 recognizes characters that are not related to the destination and the route from the video of the front of the vehicle.

[0023] In the present embodiment, the recognition unit 25 recognizes characters that are not related to the destination and the route, such as characters indicating the name of a store, from the video of the front of the vehicle. For example, when the destination is a store, the recognition unit 25 recognizes characters other than the characters indicating the name of the store that is the destination from the video of the front of the vehicle.

[0024] In this embodiment, the recognition unit 25 recognizes characters from the image captured in front of the vehicle, for example, place names and road names, that are not related to the destination and route. The recognition unit 25 recognizes characters from the image captured in front of the vehicle other than characters corresponding to place names and intersection names that are read aloud in the voice guidance of the navigation system's route guidance. The recognition unit 25 recognizes characters from the image captured in front of the vehicle other than characters corresponding to place names and intersection names that are displayed in the guidance display of the navigation system's route guidance.

[0025] The recognition of objects will be explained using Figure 2. Figure 2 is a schematic diagram showing an example of the view of a driver facing forward in a vehicle. In this example, it is assumed that the destination is the restaurant "CCC" and the driver is being guided along the route. In the view of the vehicle ahead, seen through the windshield, are the store's signs 151, 152, 153, 154, and 155. The recognition unit 25 recognizes the letters "EEE" from the recognized object, sign 151. The recognition unit 25 recognizes the letters "BBB" from the recognized object, sign 152. The recognition unit 25 recognizes the letters "AAA" from the recognized object, sign 153. The recognition unit 25 recognizes the letters "CCC" from the recognized object, sign 154. The recognition unit 25 recognizes the letters "DDD" from the recognized object, sign 155.

[0026] The recognition of objects will be explained using Figure 4. Figure 4 is a schematic diagram showing another example of the driver's field of view when facing forward in a vehicle. In this example, it is assumed that the destination is "OOO" (place name) and the route is being guided. In the forward field of view of the vehicle seen through the windshield, there are signs located in the upper left and upper right, a sign indicating the intersection name is located in the lower left, and a sign indicating the station name and direction is located in the lower right. The recognition unit 25 recognizes the letters "KKK" which is the place name 171, the letters "UUU" which is the place name 172, the letters "OOO" which is the place name 173, and the letters "VVV" which is the place name 174 from the sign in the upper left. The recognition unit 25 recognizes the figure 175 indicating the highway, the letters "YYY" which is the place name 176, the letters "XXX" which is the place name 177, and the letters "ZZZ" which is the place name 178 from the sign in the upper right. The recognition unit 25 recognizes the letters "TTT" which is the intersection name 181 from the sign in the lower left. The recognition unit 25 recognizes the station name 182, "PPP Station," from the sign in the lower right.

[0027] The calculation unit 27 calculates the projection position for projecting the mask image, the size of the mask image, and the background color of the mask image from the video acquired by the video acquisition unit 21. The calculation unit 27 calculates the position of the recognized object on the windshield as the projection position from the video acquired by the video acquisition unit 21. The calculation unit 27 calculates the size of the outline of the recognized object as the size of the mask image from the video acquired by the video acquisition unit 21. The calculation unit 27 calculates the background color of the mask image as the background color of the recognized object from the video acquired by the video acquisition unit 21.

[0028] The projection position is the position on the windshield where, from the driver's perspective, or in other words, within the driver's field of view, the actual object and the masked image are superimposed. The projection position is calculated based on the position of the object in the image relative to the vehicle and the position of the driver's eyes (in-vehicle image) estimated based on pre-registered information such as the driver's seating position or the position of the driver's seat.

[0029] The projection control unit 29 controls the projection in the projection unit 19. The projection control unit 29 controls the projection of a mask image superimposed on the object recognized by the recognition unit 25 onto the windshield. The projection control unit 29 outputs a video signal to the projection unit 19 for projecting the mask image superimposed on the object onto the windshield. Based on the projection position, size, and background color of the mask image calculated by the calculation unit 27, the projection control unit 29 outputs a video signal to the projection unit 19 for projecting the mask image onto the windshield.

[0030] The projection of the mask image will be explained using Figure 3. Figure 3 is a schematic diagram showing an example of a mask being applied to the driver's field of view facing forward in the vehicle shown in Figure 2. The projection control unit 29 projects the mask image over the signs 151, 152, 153, and 155, which are objects in the driver's field of view that have recognized characters other than the destination restaurant "CCC". For example, the projection control unit 29 projects the mask image 151M onto the windshield so as to be superimposed on sign 151. For example, the projection control unit 29 projects the mask image 152M onto the windshield so as to be superimposed on sign 152. For example, the projection control unit 29 projects the mask image 153M onto the windshield so as to be superimposed on sign 153. For example, the projection control unit 29 projects the mask image 155M onto the windshield so as to be superimposed on sign 155.

[0031] The projection of the mask image will be explained using Figure 5. Figure 5 is a schematic diagram showing an example of a mask being applied to the driver's field of view facing forward in the vehicle shown in Figure 4. The projection control unit 29 projects the mask image over characters that are recognized as characters other than the destination "OOO" in the driver's field of view. For example, the projection control unit 29 projects the mask image 151M onto the windshield so as to superimpose it on the characters "KKK", which is place name 171. For example, the projection control unit 29 projects the mask image 152M onto the windshield so as to superimpose it on the characters "UUU", which is place name 172. For example, the projection control unit 29 projects the mask image 153M onto the windshield so as to superimpose it on the characters "VVV", which is place name 174. For example, the projection control unit 29 projects the mask image 176M onto the windshield so as to superimpose it on the characters "YYY", which is place name 176. The projection control unit 29 projects a mask image 177M onto the windshield so as to superimpose it on the characters "XXX", which is the place name 177. The projection control unit 29 projects a mask image 178M onto the windshield so as to superimpose it on the characters "ZZZ", which is the place name 178. The projection control unit 29 projects a mask image 181M onto the windshield so as to superimpose it on the characters "TTT", which is the intersection name 181. The projection control unit 29 projects a mask image 182M onto the windshield so as to superimpose it on the characters "PPP Station", which is the station name 182.

[0032] <Projection control method> Next, the processing flow in the projection control device 20 of the projection device 1 will be explained using Figure 6. Figure 6 is a flowchart showing an example of the processing flow in the projection device according to the embodiment. While the projection device 1 is starting up, the processing shown in the flowchart in Figure 6 is executed. When the projection device 1 is starting up, the projection control device 20 acquires route information from the navigation system using the route information acquisition unit 22. While the projection device 1 is starting up, the projection control device 20 acquires video from the external camera 11 using the video acquisition unit 21.

[0033] The projection control device 20 recognizes characters from the image (step S101). More specifically, the projection control device 20, using the recognition unit 25, recognizes characters unrelated to the destination and route from the image acquired by the image acquisition unit 21, specifically from the image of the area in front of the vehicle. The projection control device 20 then proceeds to step S102.

[0034] The projection control device 20 determines whether or not a character has been recognized (step S102). The projection control device 20 determines, using the recognition unit 25, whether or not a character unrelated to the destination and route has been recognized. If the projection control device 20 determines that a character unrelated to the destination and route has been recognized (Yes in step S102), it proceeds to step S103. If the projection control device 20 does not determine that a character unrelated to the destination and route has been recognized (No in step S102), it repeats the process in step S101.

[0035] If it is determined that characters unrelated to the destination and route have been recognized (Yes in step S102), the projection control device 20 calculates the projection position, size, and background color (step S103). More specifically, the projection control device 20 uses a calculation unit 27 to calculate the position of the recognized object on the windshield as the projection position from the image acquired by the image acquisition unit 21. The projection control device 20 uses a calculation unit 27 to calculate the size of the outline of the recognized object as the size of the mask image from the image acquired by the image acquisition unit 21. The projection control device 20 uses a calculation unit 27 to calculate the background color of the mask image as the background color of the recognized object from the image acquired by the image acquisition unit 21. The projection control device 20 then proceeds to step S104.

[0036] The projection control device 20 projects the mask image (step S104). Based on the projection position, size, and background color of the mask image calculated by the calculation unit 27, the projection control device 20 outputs a video signal to the projection unit 19 for projecting the mask image onto the windshield. The projection control device 20 proceeds to step S105.

[0037] The projection control device 20 determines whether or not to terminate the process (step S105). The projection control device 20 determines that the journey has ended, for example, when it detects that the vehicle's shift position is in "parking," or that the speed has been zero for a predetermined period of time of several minutes or longer, or that the engine has stopped, or that the handbrake or footbrake has been applied. If the projection control device 20 determines that the process should be terminated (Yes in step S105), it terminates the process in this flowchart. If the projection control device 20 does not determine that the process should be terminated (No in step S105), it repeats the process in step S101.

[0038] <Effects> As described above, this embodiment can recognize characters unrelated to the destination and route from video footage taken in front of the vehicle, and project a mask image superimposed on the recognized characters onto the windshield. According to this embodiment, it is possible to make it difficult for the driver to recognize characters unrelated to the destination and route through the windshield. According to this embodiment, it is possible to suppress the fact that the driver's gaze is guided by objects unrelated to the destination and route when providing route guidance to the destination.

[0039] [Second Embodiment] The projection device 1A according to this embodiment will be described with reference to Figures 7 to 9. Figure 7 is a block diagram showing an example of the configuration of the projection device according to the second embodiment. Figure 8 is a schematic diagram showing an example of the field of view of a driver facing forward in a vehicle. Figure 9 is a schematic diagram showing an example of a mask applied to the field of view of a driver facing forward in a vehicle, as shown in Figure 8. The basic configuration of the projection device 1A is the same as that of the projection device 1 of the first embodiment. In the following description, components similar to those of the projection device 1 are denoted by the same reference numerals or corresponding reference numerals, and their detailed descriptions are omitted. The projection device 1A differs from the first embodiment in that it has a mask area storage unit 15A and in the processing in the recognition unit 25A and the projection control unit 29A.

[0040] In this embodiment, the mask image is described as being projected onto the windshield, but the mask image may also be projected only to the mask area on the windshield where the mask image can be projected, as described below.

[0041] The masked area is the area on the windshield where images can be projected. For example, the masked area is the left masked area A2 (see Figure 8) and the right masked area A3 (see Figure 8) on the windshield. The unmasked area A1 (see Figure 8) in the center of the windshield is an area where image projection is restricted in order to ensure the driver's field of vision in front of the vehicle. The masked and unmasked areas are stored in advance in the masked area storage unit 15.

[0042] The mask area storage unit 15A stores information indicating a mask area on the vehicle's windshield where the mask can be projected. The mask area storage unit 15A is, for example, a semiconductor memory element such as RAM (Random Access Memory) or ROM (Read Only Memory), or a storage device such as a hard disk, optical disk, or external storage device via a network.

[0043] In this case, the recognition unit 25A recognizes characters unrelated to the destination and route from the video acquired by the video acquisition unit 21, which are objects located in the mask area stored in the mask area storage unit 15A. The recognition unit 25A recognizes characters unrelated to the destination and route that correspond to the mask area on the windshield where the mask image can be projected. The recognition unit 25A recognizes characters unrelated to the destination and route that are visible through the mask area of ​​the windshield in the driver's field of view from the video captured in front of the vehicle. The recognition unit 25A recognizes objects located at a position where a straight line connecting the driver's eyes and the object passes through the mask area of ​​the windshield as characters unrelated to the destination and route from the video captured in front of the vehicle.

[0044] The projection control unit 29A outputs a video signal to the projection unit 19 that projects the mask image onto the mask area of ​​the windshield, based on the projection position, size, and background color of the mask image calculated by the calculation unit 27.

[0045] The recognition of objects will be explained using Figure 8. The recognition unit 25A recognizes signs 151, 152, and 155 as objects located in mask region A2 and mask region A3 from the video captured by the external camera 11.

[0046] The projection of the mask image will be explained using Figure 9. The projection control unit 29A projects the mask image over the signs 151, 152, and 155, which are objects in the driver's field of view that have been recognized as having characters other than the destination restaurant "CCC".

[0047] <Effects> As described above, this embodiment can recognize an object corresponding to the mask area of ​​the windshield from an image captured from the front of the vehicle, and project a mask image superimposed on the object onto the mask area of ​​the windshield.

[0048] Now, although the projection device relating to this disclosure has been described, it may be implemented in various other forms besides those described above.

[0049] Each component of the illustrated projection apparatus is a functional concept and does not necessarily have to be physically configured as shown. In other words, the specific form of each apparatus is not limited to that shown, and all or part of it may be functionally or physically dispersed or integrated in any unit depending on the processing load and usage conditions of each apparatus.

[0050] The configuration of the projection device is realized, for example, as software, such as a program loaded into memory. In the above embodiment, these were described as functional blocks realized through the cooperation of hardware or software. That is, these functional blocks can be realized in various forms using hardware alone, software alone, or a combination thereof.

[0051] The components described above include those that are easily conceivable by those skilled in the art, and those that are substantially identical. Furthermore, the components described above can be combined as appropriate. In addition, various omissions, substitutions, or modifications of the components are possible without departing from the spirit of the present invention. [Explanation of symbols]

[0052] 1 Projection device 11. Exterior car camera 13 GNSS receiver 19 Projection section 20 Projection control device 21 Video Acquisition Unit 22 Route Information Acquisition Unit 23 Location information acquisition unit 25 Recognition part 27 Calculation Section 29 Projection Control Unit

Claims

1. A route information acquisition unit that acquires route information indicating at least one of the vehicle's destination or the route to the destination, A recognition unit that recognizes objects unrelated to the destination and route from video footage taken of the area in front of the vehicle, A projection control unit controls the projection of a mask image superimposed on the object recognized by the recognition unit onto the windshield. A projection control device equipped with the following features.

2. The aforementioned object is at least one of the following: a name indicating a store, a place name, or a name indicating a road. The projection control device according to claim 1.

3. Obtaining route information that indicates either the vehicle's destination or the route to the destination, From the video footage taken of the area in front of the vehicle, it is possible to recognize objects unrelated to the destination and the route, Controlling the system to project a mask image superimposed on the recognized object onto the windshield, A projection control method performed by a computer acting as a projection control device.

Images