Imaging device, control method, program, and storage media

Abstract

To provide an imaging device capable of preferably recognizing a position of a vanishing point in an image even when an imaging direction for imaging the front of a vehicle changes.SOLUTION: A navigation device 1 comprises a front imaging camera 4 for capturing the scenery in front of a vehicle, a reference target movement determination unit 16, a vanishing point learning unit 17, and a control unit 19. The vanishing point learning unit 17 analyzes a front image Im captured by the front imaging camera 4 to learn a position of a vanishing point included in the front image Im. The reference target movement determination unit 16 analyzes the front image Im while the vehicle is traveling to identify a reference object Tag that is included in the front image Im and is a part of the vehicle. The control unit 19 corrects, when a position in the front image Im of an object identified by the reference target movement determination unit 16 moves, the position of the vanishing point learned by the vanishing point learning unit 17 based on the direction and distance of the movement.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a technique for learning the position of a vanishing point in a front image.

Background Art

[0002] Conventionally, techniques for learning the position of a vanishing point in an image have been known. For example, in Patent Document 1, among the vanishing point positions detected from imaging data by a camera, a vanishing point whose deviation amount from a pre-specified vanishing point position or a learned vanishing point position within the region imaged by the camera is outside a predetermined range is regarded as not being a learning target, and an imaging device is disclosed. Further, Patent Document 2 discloses a drive recorder that includes a half mirror, a liquid crystal display, and a camera unit that photographs the front of a vehicle, and is attached to a rearview mirror so as to be sandwiched from above and below by upper and lower clamps on the back.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] In Patent Document 1, the learning accuracy of the vanishing point is improved by being able to actively exclude data that is not appropriate for obtaining the deviation amount from the current vanishing point position (for example, data when the apparent vanishing point position is temporarily shifted due to a curved road). On the other hand, when trying to learn a vanishing point using an image captured by an imaging device attached to a rearview mirror as shown in Patent Document 2, for example, when the driver changes and the angle of the rearview mirror changes, the imaging direction of the camera changes steeply along with the angle of the rearview mirror. In this case, with the technique of Patent Document 1, there is a risk that the vanishing point cannot be learned after the angle of the rearview mirror changes.

[0005] The present invention has been made to solve the above-described problems, and a main object thereof is to provide an imaging device capable of suitably recognizing the position of a vanishing point in an image even when the imaging direction for imaging the front of a vehicle changes.

Means for Solving the Problems

[0006] The invention according to the claim is an imaging device incorporated in a rearview mirror of a vehicle or attached to the rearview mirror, the imaging device including: imaging means for imaging the front of the vehicle; learning means for learning the position of a vanishing point included in a front image captured by the imaging means; specifying means for specifying a reference object included in the front image; determination means for determining whether or not there is a possibility that the angle of the rearview mirror has changed; and when it is determined that there is a possibility that the angle of the rearview mirror has changed, determining whether or not the position of the reference object in the front image has moved, and based on the direction and amount of movement when it is determined that the reference object has moved, correcting means for correcting the position of the vanishing point learned by the learning means.

[0007] Further, the invention according to the claim is a control method executed by an imaging device incorporated in a rearview mirror of a vehicle or attached to the rearview mirror, the control method including: an imaging step for imaging the front of the vehicle; a learning step for learning the position of a vanishing point included in a front image captured by the imaging step; a specifying step for specifying a reference object included in the front image; a determination step for determining whether or not there is a possibility that the angle of the rearview mirror has changed; and when it is determined that there is a possibility that the angle of the rearview mirror has changed, determining whether or not the position of the reference object in the front image has moved, and based on the direction and amount of movement when it is determined that the reference object has moved, a correction step for correcting the position of the vanishing point learned by the learning step.

[0008] The invention according to the claims is a program executed by a computer mounted on a vehicle, wherein learning means for learning the position of a vanishing point included in a front image, which is an image obtained by photographing means built in or attached to the rearview mirror of the vehicle and photographing the front of the vehicle, specific means for specifying a reference object included in the front image, determination means for determining whether or not there is a possibility that the angle of the rearview mirror has changed, and when it is determined that there is a possibility that the angle of the rearview mirror has changed, it is determined whether or not the position of the reference object in the front image has moved, and correction means for correcting the position of the vanishing point learned by the learning means based on the direction and amount of movement when it is determined that the reference object has moved, characterized in that the computer is caused to function as such.

Brief Description of Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Embodiments for Carrying Out the Invention

[0010] In one preferred embodiment of the present invention, there is provided a photographing device mounted on a vehicle, comprising photographing means for photographing the front scenery of the vehicle, learning means for analyzing a front image photographed by the photographing means and learning the position of a vanishing point included in the front image, specific means for analyzing the front image while the vehicle is running and specifying an object that is part of the vehicle included in the front image, and correction means for correcting the position of the vanishing point learned by the learning means based on the direction and distance of movement when the position of the object specified by the specific means in the front image has moved.

[0011] The above imaging device includes an imaging unit that captures the front view of a vehicle, a learning unit, a specifying unit, and a correcting unit. The learning unit analyzes the front image captured by the imaging unit and learns the position of the vanishing point included in the front image. The specifying unit analyzes the front image while the vehicle is running and specifies an object that is included in the front image and is part of the vehicle. When the position of the object specified by the specifying unit moves in the front image, the correcting unit corrects the position of the vanishing point learned by the learning unit based on the direction and distance of the movement. In this manner, even when the imaging direction of the imaging unit changes steeply, the imaging device can promptly correct and utilize the learned vanishing point position.

[0012] In one aspect of the above imaging device, it is built into the rearview mirror of the vehicle or attached to the rearview mirror. In this aspect, the imaging direction changes by adjusting the angle of the rearview mirror. Even in this case, the imaging device can promptly correct and utilize the learned vanishing point position.

[0013] In another aspect of the above imaging device, when an acceleration sensor that detects the acceleration applied to the vehicle detects an acceleration equal to or greater than a reference value, the correcting unit determines whether the position of the object in the front image has moved, and corrects the position of the vanishing point when it is determined that the object has moved. In this aspect, since the imaging device determines the presence or absence of movement of the object only when it is determined that there is a possibility that the imaging direction has changed based on the output of the acceleration sensor, the processing load can be suitably reduced.

[0014] In another aspect of the above imaging device, based on the output of a contact sensor that detects contact of a passenger of the vehicle with the rearview mirror, the correcting unit determines whether the position of the object in the front image has moved, and corrects the position of the vanishing point when it is determined that the object has moved. Also in this aspect, since the imaging device determines the presence or absence of movement of the object only when it is determined that there is a possibility that the imaging direction has changed, the processing load can be suitably reduced.

[0015] In another aspect of the above-described imaging device, when the position of the vanishing point is corrected by the correction means, the learning means resumes learning the position of the vanishing point based on the corrected position of the vanishing point. According to this aspect, even when the position of the vanishing point in the front image changes, the imaging device can suitably utilize the processing result before the change in position and resume learning.

[0016] In another embodiment according to the present invention, there is provided a control method executed by an imaging device mounted on a vehicle, the method including: a learning step of analyzing a front image captured by an imaging means for capturing a front view of the vehicle and learning the position of a vanishing point included in the front image; a specifying step of analyzing the front image while the vehicle is running and specifying an object included in the front image and being part of the vehicle; and a correcting step of correcting the position of the vanishing point learned in the learning step based on the direction and distance of movement when the position of the object specified in the specifying step moves in the front image. By executing this control method, the imaging device can promptly correct and utilize the learned vanishing point position even when the imaging direction of the imaging means changes steeply.

[0017] In another embodiment according to the present invention, there is provided a program executed by a computer mounted on a vehicle, the program causing the computer to function as: a learning means for analyzing a front image captured by an imaging means for capturing a front view of the vehicle and learning the position of a vanishing point included in the front image; a specifying means for analyzing the front image while the vehicle is running and specifying an object included in the front image and being part of the vehicle; and a correcting means for correcting the position of the vanishing point learned by the learning means based on the direction and distance of movement when the position of the object specified by the specifying means moves in the front image. By executing this program, the computer can promptly correct and utilize the learned vanishing point position even when the imaging direction of the imaging means changes steeply. Preferably, the above program is stored in a storage medium.

Example

[0018] Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. Hereinafter, the "shooting direction" refers to the direction that is the front of the camera and passes through the center of the camera's field of view (shooting range).

[0019] [Configuration of Navigation Device] FIG. 1 shows the configuration of the navigation device 1 according to the embodiment. Specifically, FIG. 1(A) shows a front view of the navigation device 1 mounted on the rearview mirror 12 in the vehicle interior (also simply referred to as the "mounted state"), FIG. 1(B) shows a rear view of the navigation device 1 in the mounted state, and FIG. 1(C) shows a top view of the navigation device 1 in the mounted state. Hereinafter, the longitudinal direction of the navigation device 1 is the "Y-axis direction", the short-side direction of the navigation device 1 is the "X-axis direction", and the direction perpendicular to the X-axis and Y-axis is the "Z-axis direction", and each positive direction is defined as shown in the figure.

[0020] The navigation device 1 shown in FIG. 1 is driven by a built-in secondary battery and mainly includes a mirror unit 13, a front shooting camera 4, a display (display means) 10 existing on the back of the mirror unit 13, and clamping portions 14A to 14D. In the mounted state, the navigation device 1 is fixed to the rearview mirror 12 by the clamping portions 14A to 14D clamping the rearview mirror 12. In the mounted state of the navigation device 1, the mirror surface of the rearview mirror 12 and the back of the navigation device 1 overlap. The navigation device 1 is an example of the "shooting device" in the present invention.

[0021] The front shooting camera 4 is a camera for generating an image of the front of the vehicle (also referred to as the "front image Im") and is disposed on the back portion of the navigation device 1 that does not overlap with the rearview mirror 12 in the mounted state of the navigation device 1. Further, the front shooting camera 4 is rotatably accommodated in the housing of the navigation device 1, and the shooting direction is adjusted based on an operation to a remote controller (not shown) or the like.

[0022] The mirror unit 13 is, for example, a half mirror that transmits part of the incident light and reflects the other light. Thereby, when the display 10 on the back is in the non-emitting state, the mirror unit 13 functions as a normal mirror that reflects the rear view of the vehicle, and when the display 10 is in the emitting state, it transmits the light emitted from the display 10, allowing the driver to visually recognize the display content of the display 10. The display 10 displays, by superimposing a guidance image representing characters, figures, etc. for guiding the driver, on the front image Im captured by the front imaging camera 4.

[0023] The navigation device 1 detects lanes, the lighting state of traffic lights, preceding vehicles, etc. by performing image recognition processing on the front image Im, and superimposes and displays a text image, a graphic image, etc. based on the detection result on the front image Im, or outputs voice information. Here, in order to reduce the processing load, the navigation device 1 performs the detection processing of lanes, traffic lights, preceding vehicles, etc. included in the front image Im by limiting the area based on the vanishing point in the front image Im. For example, when detecting a traffic light, the navigation device 2 performs traffic light detection with the image area above the vanishing point as the detection range. Also, as will be described later, the navigation device 1 also functions as a drive recorder.

[0024] FIG. 2 is a functional block diagram of the navigation device 1. As shown in FIG. 2, the navigation device 1 includes a front imaging camera 4, a current position detection unit 5, an acceleration sensor 6, a display 10, an image buffer unit 11, a non-volatile memory 15, a reference target movement determination unit 16, a vanishing point learning unit 17, and a control unit 19.

[0025] The current position detection unit 5 is, for example, a GPS receiver, and supplies information indicating the current position of the vehicle to the control unit 19. The acceleration sensor 6 detects accelerations in the three-axis directions of the X-axis, Y-axis, and Z-axis, and supplies the detection signal to the control unit 19.

[0026] The non-volatile memory 15 is a storage medium such as an SD card, and stores the front image Im generated by the front imaging camera 4, etc. based on the control of the control unit 19.

[0027] When the reference object movement determination unit 16 determines that the angle of the rearview mirror 12 (also referred to as the "mirror angle") may have changed, it detects the movement direction and movement distance in the front image Im of a part of the vehicle (also referred to as the "reference object Tag") existing within the shooting range of the front shooting camera 4. Then, the reference object movement determination unit 16 supplies the detected information to the control unit 19. Here, the reference object movement determination unit 16 determines the presence or absence of the possibility of a change in the mirror angle based on the output of the acceleration sensor 6. Details of the processing of the reference object movement determination unit 16 will be described in the section of [Detection of Movement of Reference Object].

[0028] The vanishing point learning unit 17 learns an invariant vanishing point existing in the true front direction of the vehicle. Specifically, the vanishing point learning unit 17 detects, by image analysis of the front image Im, white lines painted on both sides of the driving lane, the boundary line between the road surface and the curb, etc. Then, the vanishing point learning unit 17 stores the coordinates on the front image Im of the lane vanishing point, which is the intersection of the detected left and right white lines and the extension line in the vehicle traveling direction of the boundary line, as a learning sample of the vanishing point (also referred to as the "sample vanishing point coordinate CS"). Then, the vanishing point learning unit 17 learns the vanishing point by averaging a plurality of sample vanishing point coordinates CS acquired in time series by a statistical method such as moving average. Hereinafter, the coordinates on the front image Im of the vanishing point learned by the vanishing point learning unit 17 from a plurality of sample vanishing point coordinates CS by a statistical method are also referred to as the "learned vanishing point coordinate CL".

[0029] The control unit 19 has a CPU, a ROM that stores a control program executed by the CPU, data, etc., and a RAM in which various data are sequentially read and written as a work memory when the CPU operates, and performs overall control of the navigation device 1.

[0030] For example, the control unit 19 overwrites the front image Im for a predetermined time in the image buffer unit 11, and when the acceleration sensor 6 detects an acceleration equal to or greater than a predetermined value, writes the front image Im stored in the image buffer unit 11 into the non-volatile memory 15. Further, based on the captured image Im, the control unit 19 performs various recognition processes such as detection of the vehicle ahead, detection of the lane, and detection of the lit color of the traffic signal by a known image recognition technique, and superimposes a guidance image based on the recognition result on the front image Im and displays it on the display 10. At this time, the control unit 19 limits the area of the detection target within the front image Im based on the learned vanishing point coordinates CL calculated by the vanishing point learning unit 17.

[0031] Furthermore, when the control unit 19 receives information on the moving direction and moving distance of the reference target object Tag within the front image Im from the reference target movement determination unit 16, it corrects the learned vanishing point coordinates CL based on the moving direction and moving distance. Then, the control unit 19 notifies the vanishing point learning unit 17 of the corrected learned vanishing point coordinates CL.

[0032] Note that the navigation device 1 may further include a voice output unit and the like in addition to the elements shown in FIG. 2. The reference target movement determination unit 16 is an example of the "specifying means" in the present invention, the vanishing point learning unit 17 is an example of the "learning means" in the present invention, the control unit 19 is an example of the "correcting means" in the present invention, and the reference target movement determination unit 16, the vanishing point learning unit 17, and the control unit 19 are examples of the "computer" that executes the program in the present invention.

[0033] [Processing Flow] FIG. 3 is a flowchart showing the procedure of a process for correcting the learned vanishing point coordinates CL (also referred to as "learned vanishing point correction process").

[0034] First, the reference target movement determination unit 16 compares the front images Im during the running of the vehicle in time series, sets a part of the vehicle whose position does not change in the front image Im as the reference target object Tag, and stores the color information of the reference target object Tag, the coordinate information of the contour, and the coordinate information of the characteristic parts (also referred to as "feature points") within the contour such as the corner parts of the contour (step S101).

[0035] Next, based on the detection signal of the acceleration sensor 6, the reference object movement determination unit 16 determines whether an acceleration equal to or greater than a predetermined reference value has been detected (step S102). Here, the "reference value" is a threshold value of acceleration for determining whether the mirror angle has changed, and is determined in advance based on, for example, experiments or the like. In this case, the reference object movement determination unit 16 may compare the largest acceleration among the accelerations of the XYZ axes with the reference value, or may compare the vector length of the acceleration in the three-dimensional space with the reference value.

[0036] And when the reference object movement determination unit 16 detects an acceleration equal to or greater than the reference value (step S102; Yes), it determines that the mirror angle may have changed. Therefore, in this case, the reference object movement determination unit 16 performs a process of detecting the reference object Tag from the front image Im immediately acquired from the front shooting camera 4, and calculates the movement distance and movement direction of the reference object Tag within the front image Im (step S103). Then, the reference object movement determination unit 16 supplies the calculated information on the movement distance and movement direction of the reference object Tag to the control unit 19. A specific example of the process of step S103 will be described in the section of [Detection of Movement of Reference Object].

[0037] On the other hand, when the reference object movement determination unit 16 does not detect an acceleration equal to or greater than the reference value (step S102; No), it determines that there is no change in the mirror angle and returns the process to step S101.

[0038] After the execution of step S103, based on the information on the moving distance and moving direction of the reference object Tag within the front image Im received from the reference object movement determination unit 16, the control unit 19 determines whether the reference object Tag has moved by a predetermined distance or more within the front image Im (step S104). The "predetermined distance" is determined, for example, as the number of pixels for which it is determined that the learning vanishing point coordinates CL need to be corrected. Generally, even when the user operates the room mirror 12 and the room mirror 12 temporarily shakes, the mirror angle (i.e., the shooting direction of the front shooting camera 4) may not be finally changed. Therefore, the control unit 19 finally determines whether the learning vanishing point coordinates CL need to be corrected in step S104.

[0039] And when the reference object Tag has moved by a predetermined distance or more within the front image Im (step S104; Yes), the control unit 19 corrects the learning vanishing point coordinates CL based on the moving distance and moving direction of the reference object Tag calculated in step S103 (step S105). Then, the control unit 19 notifies the vanishing point learning unit 17 of the corrected learning vanishing point coordinates CL. After that, the vanishing point learning unit 17 resumes the learning of the vanishing point based on the corrected learning vanishing point coordinates CL (step S106). In this case, for example, the vanishing point learning unit 17 deletes all the stored sample vanishing point coordinates CS and resumes the learning considering the corrected learning vanishing point coordinates CL as having weights for a predetermined number of samples. In another example, the vanishing point learning unit 17 corrects all the stored sample vanishing point coordinates CS based on the moving distance and moving direction of the reference object Tag described above, and uses the corrected sample vanishing point coordinates CS for the calculation of the subsequent learning vanishing point coordinates CL.

[0040] On the other hand, when the reference object Tag has not moved by a predetermined distance or more (step S104; No), the control unit 19 determines that there is no need to correct the learning vanishing point coordinates CL and returns the process to step S102.

[0041] [Detection of movement of reference object] Next, a specific example of the processes executed by the reference object movement determination unit 16 in steps S101 and S103 of FIG. 3 will be described.

[0042] FIG. 4(A) shows a front image Im captured by the front camera 4 before the mirror angle is adjusted. In the front image Im of FIG. 4(A), an A-pillar 32, which is a part of the vehicle, and a dashboard 33 are displayed. Also, a position 31A in FIG. 4(A) indicates the position of the learned vanishing point coordinates CL learned by the vanishing point learning unit 17. As shown in FIG. 4(A), the position 31A overlaps with the lane vanishing point.

[0043] In this case, based on step S101 in FIG. 3, the reference target movement determination unit 16 compares the front images Im in time series and regards the A-pillar 32 and the dashboard 33 whose positions do not change in the front image Im as the reference target object Tag. Then, the reference target movement determination unit 16 stores the coordinate information of the contours (see the broken line 34) of the A-pillar 32 and the dashboard 33 in the front image Im and the color information of the area within the contours. Further, the reference target movement determination unit 16 detects a position 35A, which is the joint of the A-pillar 32 and the dashboard 33 and where the angle of these contour lines changes rapidly, as a feature point of the reference target object Tag, and stores the coordinate information of the position 35A within the front image Im.

[0044] FIG. 4(B) shows a front image Im captured by the front camera 4 after the mirror angle is adjusted. In FIG. 4(B), for the sake of convenience, the broken line 34 showing the contour detected in FIG. 4(A) and the position 35A showing the feature point are also illustrated.

[0045] In the example of FIG. 4(B), since the reference object movement determination unit 16 has detected an acceleration equal to or greater than a predetermined reference value, based on step S103, it performs a process of detecting the reference object Tag from the forward image Im shown in FIG. 4(B). In this case, the reference object movement determination unit 16 detects, for example, the contour (see the dashed line 36) of the reference object Tag and the position 35B corresponding to the feature point within the current forward image Im based on the coordinate information of the contour of the reference object Tag and the color information of the reference object Tag memorized in the example of FIG. 4(A). Then, the reference object movement determination unit 16 calculates the movement distance and movement direction from the position 35A indicating the feature point detected before the mirror angle change to the position 35B. In FIG. 4(B), the reference object movement determination unit 16 detects the movement amount "dX" of the feature point on the X-axis when the right direction of the forward image Im is the positive direction of the X-axis, and the movement amount "dY" of the feature point on the Y-axis when the upward direction of the forward image Im is the positive direction of the Y-axis.

[0046] FIG. 4(C) shows the position 31B corresponding to the corrected learning vanishing point coordinates CL. In this example, the control unit 19 determines that the movement distance of the reference object Tag based on the movement amounts dX and dY is equal to or greater than a predetermined distance, and corrects the learning vanishing point coordinates CL. Specifically, the control unit 19 recognizes the coordinates of the position 31B that has been moved by the movement amount dX in the X-axis direction and the movement amount dY in the Y-axis direction from the position 31A corresponding to the learning vanishing point coordinates CL learned by the vanishing point learning unit 17 before the adjustment of the mirror angle, as the corrected learning vanishing point coordinates CL. In this case, the position 31B overlaps with the lane vanishing point of the forward image Im taken after the mirror angle is changed. In this way, even when the mirror angle is changed, the navigation device 1 can effectively utilize the learned learning vanishing point coordinates CL and accurately recognize the current learning vanishing point coordinates CL.

[0047] As described above, the navigation device 1 according to this embodiment includes a front camera 4 that captures the front view of the vehicle, a reference object movement determination unit 16, a vanishing point learning unit 17, and a control unit 19. The vanishing point learning unit 17 analyzes the front image Im captured by the front camera 4 and learns the position of the vanishing point included in the front image Im. The reference object movement determination unit 16 analyzes the front image Im while the vehicle is running and identifies a reference object Tag that is included in the front image Im and is part of the vehicle. When the position of the object identified by the reference object movement determination unit 16 moves in the front image Im of the object, the control unit 19 corrects the position of the vanishing point learned by the vanishing point learning unit 17 based on the direction and distance of the movement. By doing so, even when the shooting direction of the front camera 4 changes steeply, the navigation device 1 can quickly correct and utilize the learned vanishing point position.

[0048] [Modification Example] Hereinafter, modification examples suitable for the above-described embodiment will be described. The following modification examples may be arbitrarily combined and applied to the above-described embodiment.

[0049] (Modification Example 1) Instead of or in addition to the acceleration sensor 6, the navigation device 1 may be provided with a contact sensor for detecting that a passenger has touched the rearview mirror 12.

[0050] In this case, the contact sensor may be a touch panel provided on the mirror surface of the rearview mirror 12, or a pressure-sensitive switch provided on the housing or lever of the rearview mirror 12 that the passenger touches when changing the angle of the rearview mirror 12. In this case, in step S102 of FIG. 3, when the control unit 19 determines that there is contact with the rearview mirror 12 based on the output signal of the contact sensor, it determines that the mirror angle may have been changed and proceeds to step S103.

[0051] (Modification Example 2) The navigation device 1 does not necessarily have the function of a drive recorder. In this case, the navigation device 1 does not have the acceleration sensor 6, and the reference object movement determination unit 16 does not necessarily execute the determination process based on the output of the acceleration sensor 6 in step S102 of FIG. 3.

[0052] In this case, after executing step S101, the reference object movement determination unit 16 calculates the movement distance and movement direction of the reference object Tag in the front image Im in step S103, and determines in step S104 whether the reference object Tag has moved by a predetermined distance or more. That is, in this case, the reference object movement determination unit 16 continuously monitors the movement of the reference object Tag at all times.

[0053] In this way, even when the navigation device 1 does not include the acceleration sensor 6, it can suitably detect a change in the mirror angle and correct the learned vanishing point coordinates CL.

[0054] (Modification Example 3) In the above embodiment, the navigation device 1 includes sandwiching portions 14A to 14D for sandwiching and holding the navigation device 1 between the genuine rearview mirror 12 attached to the vehicle. Instead of this, the navigation device 1 may not include the sandwiching portions 14A to 14D, may be built into the rearview mirror, and may be replaceable with the genuine rearview mirror attached to the vehicle.

[0055] In another example, the navigation device 1 is not limited to being attached to or built into the rearview mirror 12, and may be configured integrally with a display installed on the dashboard of the vehicle, and may be a navigation device whose angle can be freely changed to a direction in which the display is easily visible to the occupant.

[0056] In yet another example, the navigation device 1 may be a drive recorder or the like that does not have a route guidance function.

Explanation of Reference Numerals

[0057] 1 Navigation device 4 Front camera 5 Current position detection unit 10 Display 11 Image buffer unit 12 Rearview mirror 13 Mirror unit 15 Non-volatile memory 16 Reference target movement determination unit 17 Vanishing point learning unit 19 Control unit

Claims

1. A photographing device that is built into or attached to a vehicle's rearview mirror, An imaging means for imaging a front view of the vehicle; A learning means for learning a position of a vanishing point included in a forward image captured by the imaging means; An identification means for identifying a reference object included in the forward image; A determination means for determining whether or not the angle of the rearview mirror may have changed; a correction means for determining whether or not a position of the reference object in the forward image has moved when it is determined that the angle of the rearview mirror may have changed, and for correcting the position of the vanishing point learned by the learning means when it is determined that the reference object has moved, based on the direction and amount of the movement; An imaging device comprising:

2. 2. The photographing device according to claim 1, wherein the determining means determines that the angle of the rearview mirror may have changed when an acceleration sensor that detects acceleration applied to the vehicle detects an acceleration equal to or greater than a reference value.

3. 2. The photographing device according to claim 1, wherein the determination means determines that the angle of the rearview mirror may have changed when a contact sensor that detects contact with the rearview mirror by a vehicle occupant detects the contact.

4. 4. The imaging device according to claim 1, wherein the specifying unit specifies a part of the vehicle included in the forward image as the reference object.

5. 5. The photographing device according to claim 1, wherein, when the position of the vanishing point is corrected by the correction means, the learning means resumes learning of the position of the vanishing point based on the corrected position of the vanishing point.

6. A control method executed by an imaging device built into or attached to a vehicle rearview mirror, comprising: an imaging step of imaging a front area of ​​the vehicle; A learning step of learning a position of a vanishing point included in the forward image captured by the photographing step; An identification step of identifying a reference object included in the forward image; a determination step of determining whether or not the angle of the rearview mirror may have changed; a correction step of determining whether or not a position of the reference object in the forward image has moved when it is determined that the angle of the rearview mirror may have changed, and correcting the position of the vanishing point learned in the learning step based on the direction and amount of the movement when it is determined that the reference object has moved; A control method comprising the steps of:

7. A program executed by a computer installed in a vehicle, A learning means for learning the position of a vanishing point included in a forward image, which is an image captured by a photographing means built into or attached to a rearview mirror of the vehicle, of the vehicle; An identification means for identifying a reference object included in the forward image; A determination means for determining whether or not the angle of the rearview mirror may have changed; a correction means for determining whether or not a position of the reference object in the forward image has moved when it is determined that the angle of the rearview mirror may have changed, and for correcting the position of the vanishing point learned by the learning means when it is determined that the reference object has moved, based on the direction and amount of the movement; A program for causing the computer to function as a

8. A storage medium storing the program according to claim 7.

Images