Vehicle imaging system

The vehicle photographing system dynamically adjusts its camera range based on driver gaze to overcome fixed imaging limitations, ensuring comprehensive information provision by capturing and displaying previously obscured areas.

JP2024158311APending Publication Date: 2024-11-08SUZUKI MOTOR CORP
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023073423
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-04-27
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

Existing vehicle imaging systems with fixed cameras fail to provide comprehensive information to drivers by limiting the imaging range, leading to potential blind spots and inadequate information provision.

Method used

A vehicle photographing system with a movable camera unit, line-of-sight detection, and a control unit that adjusts the camera's range based on the driver's gaze direction, using a line-of-sight detection unit to dynamically change the imaging area.

Benefits of technology

Enables the system to capture and display previously unseen areas, reducing blind spots and providing drivers with the necessary information by adapting the camera's field of view to their viewing preferences.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024158311000001_ABST
    Figure 2024158311000001_ABST
Patent Text Reader

Abstract

To make it possible to appropriately provide information required by a driver.SOLUTION: A vehicle imaging system comprises: an imaging unit (1) installed on a vehicle (100); a movable unit (2) for moving the imaging unit (1) to make an imaging range be variable; a display unit (3) for displaying a picture imaged by the imaging unit (1); a visual line detection unit (4) for detecting a visual line of a driver; and a control unit (5) that, on the basis of the visual line of the driver detected by the visual line detection unit (4), controls the movable unit (2) to vary the imaging range of the imaging unit (1).SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a vehicle imaging system. [Background technology]

[0002] There is known a vehicle imaging system that captures images of the surroundings of the vehicle using a camera installed in the vehicle and displays the captured images on a display device installed inside the vehicle. For example, Patent Document 1 discloses a technology that displays an image captured by an external camera on a monitor-integrated rearview mirror. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2002-120649 A Summary of the Invention [Problem to be solved by the invention]

[0004] In a vehicle imaging system, if a camera is fixedly installed in a vehicle and the imaging range is fixed, it may not be possible to obtain images outside that range, and it may not be possible to provide the driver with the information he or she needs appropriately.

[0005] The present invention has been made in view of the above circumstances, and has an object to provide information required by a driver in an appropriate manner. [Means for solving the problem]

[0006] The vehicle photography system of the present invention is characterized in that it comprises a photography unit installed in a vehicle, a movable unit that moves the photography unit to vary the photography range, a display unit that displays an image photographed by the photography unit, a gaze detection unit that detects the driver's gaze, and a control unit that controls the movable unit to change the photography range of the photography unit based on the driver's gaze detected by the gaze detection unit. Effect of the Invention

[0007] According to the present invention, it is possible to appropriately provide information required by a driver. [Brief description of the drawings]

[0008] [Figure 1] 1 is a diagram showing a configuration of a vehicle imaging system according to an embodiment; [Diagram 2] 4 is a flowchart illustrating an example of a process executed by a control device according to the embodiment. [Diagram 3] 4 is a flowchart illustrating an example of a process executed by a control device according to the embodiment. [Figure 4] 1 is a diagram for explaining gaze detection by a gaze detection sensor; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] The vehicle photography system according to one embodiment of the present invention includes a photography unit (1) installed in a vehicle (100), a movable unit (2) that moves the photography unit (1) to change the photography range, a display unit (3) that displays an image photographed by the photography unit (1), a gaze detection unit (4) that detects the driver's gaze, and a control unit (5) that controls the movable unit (2) to change the photography range of the photography unit (1) based on the driver's gaze detected by the gaze detection unit (4). This makes it possible to capture an image of a range that cannot be captured when the capture range is fixed, according to the driver's intentions, and makes it possible to provide the driver with the information he or she needs appropriately. EXAMPLES

[0010] Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. FIG. 1 is a diagram showing an example of the configuration of a vehicle-mounted imaging system according to an embodiment, where (a) is a block diagram showing the configuration of the vehicle-mounted imaging system, and (b) is a diagram for explaining the installation locations of a camera 1, a movable device 2, a display device 3, and a gaze detection sensor 4. A camera unit 1 (simply referred to as camera 1) and a movable device 2 that moves the camera 1 to change the shooting range are installed in an automobile 100, which is a vehicle. In this embodiment, the camera 1 and the movable device 2 are installed at the rear of the automobile 100, and the camera 1 shoots images behind the automobile 100. The movable device 2 also drives the camera 1 to pan and tilt, to change the shooting direction of the camera 1. In this embodiment, the camera 1 and the movable device 2 function as the shooting section and the movable section, respectively, as defined in the present invention.

[0011] A display device 3 that displays the image captured by the camera 1 is installed inside the automobile 100. In this embodiment, the display device 3 is a so-called digital inner mirror that is installed in place of a mirror-type inner mirror, and is installed in front of the ceiling of the driver's cab or in the upper center of the windshield to enable the driver to view the rear of the automobile 100. In this embodiment, the display device 3 functions as a display unit as defined in the present invention. A gaze detection sensor 4 that detects the driver's gaze is attached to the display device 3. Fig. 4 is a diagram for explaining gaze detection by the gaze detection sensor 4. As shown in Fig. 4, the gaze detection sensor 4 divides the screen 3a of the display device 3 into nine areas 1 to 9, and detects to which position (area) the driver's gaze is directed. In this embodiment, the gaze detection sensor 4 functions as a gaze detection unit as defined in the present invention.

[0012] A control device 5 is installed at an appropriate location in the automobile 100, which executes a process of displaying an image captured by the camera 1 on the display device 3 (the control device 5 is not shown in FIG. 1(b)). The control device 5, the details of which will be described later, executes a process of controlling the movable device 2 to change the shooting range of the camera 1 based on the line of sight of the driver detected by the line of sight detection sensor 4. The control device 5 configured in this manner is constituted by, for example, a computer device equipped with a CPU, ROM, RAM, etc. An existing electronic control device may be configured to realize the functions of the control device 5 as part of its functions. In this embodiment, the control device 5 functions as a control unit as defined in the present invention.

[0013] 2 and 3 are flowcharts showing examples of processing executed by the control device 5. The processing of these flowcharts is carried out by the CPU of the computer device constituting the control device 5 executing a predetermined program.

[0014] Fig. 2 shows a process for displaying an image captured by the camera 1 on the display device 3. The flowchart in Fig. 2 is repeatedly executed from when the driving source such as the engine of the automobile 100 is started until it is stopped. In step S101, the control device 5 acquires an image captured by the camera 1. Normally, in other words, the camera 1 captures images in a preset initial setting direction except when the shooting direction of the camera 1 is changed by the processing of the flowchart in Fig. 3 described below. Also, the camera 1 may capture images by changing the shooting direction by the processing of the flowchart in Fig. 3. In step S102, the control device 5 displays on the display device 3 the photographed image acquired in step S101.

[0015] 3 shows a process for controlling the movable device 2 to change the shooting direction of the camera 1 based on the line of sight of the driver detected by the line of sight detection sensor 4. The flowchart in FIG. 3 is repeatedly executed in parallel with the flowchart in FIG. In step S201, the control device 5 acquires information on the driver's line of sight detected by the line of sight detection sensor 4. In this embodiment, the control device 5 acquires information on whether the driver's line of sight is directed toward the screen 3a of the display device 3, and, if the driver's line of sight is directed toward the screen 3a of the display device 3, on which of the regions 1 to 9 the driver's line of sight is directed toward.

[0016] In step S202, the control device 5 determines whether or not the driver's line of sight is directed to a predetermined position on the screen 3a of the display device 3, based on the line of sight information acquired in step S201. In this embodiment, the predetermined position is the area 2 to 9 around the screen 3a, and the control device 5 determines whether or not the driver's line of sight is directed to any of the areas 2 to 9. If the driver's line of sight is directed to any of the areas 2 to 9, the process proceeds to step S203. On the other hand, if the driver's line of sight is not directed to any of the areas 2 to 9, the process exits from this flowchart.

[0017] In step S203, the control device 5 judges whether the time during which the driver directs his / her gaze to the same position is equal to or longer than a predetermined time. That is, the control device 5 judges whether the state in which the driver directs his / her gaze to any of the areas 2 to 9 continues for a predetermined time. If it is equal to or longer than the predetermined time, the process proceeds to step S204. On the other hand, if it is not equal to or longer than the predetermined time, the process exits from this flowchart.

[0018] In step S204, the control device 5 controls the movable device 2 to change the shooting direction of the camera 1 based on the line-of-sight information acquired in step S201. The control device 5 changes the shooting direction of the camera 1 so that the shot image displayed in the area where the driver's line of sight is directed among the areas 2 to 9 moves toward the center of the screen 3a by a predetermined amount. For example, when the driver's line of sight is directed toward the area 2 in the upper left, the control device 5 changes the shooting direction of the camera 1 so that the shot image displayed in the area 2 moves toward the center of the screen 3a by a predetermined amount, as shown by the arrow A in FIG. 4. As a result, the shot image 401 including the part that was previously outside the shooting range is displayed on the display device 3.

[0019] If the driver's line of sight is shifted away and this state continues after the shooting direction of camera 1 is changed in step S204, the shooting direction of camera 1 is returned to the initially set direction. Also, if the driver's line of sight is directed toward central area 1 and this state continues after the shooting direction of camera 1 is changed in step S204, the shooting direction of camera 1 may be returned to the initially set direction. 4, the regions 1 to 9 are shown to be of approximately the same size, but the number and sizes of the regions may be set appropriately. For example, the central region 1 may be set to be larger than the surrounding regions 2 to 9.

[0020] As described above, the movable device 2 is controlled to change the shooting range of the camera 1 based on the driver's line of sight, so that it is possible to shoot a range that cannot be shot when the shooting range is fixed, according to the driver's intention. This makes it possible to reduce blind spots that cannot be seen by the driver. For example, the rear lower area of ​​the automobile 100 is likely to be a blind spot, but when the driver directs his / her gaze toward a lower area (e.g., area 7) on the screen 3a of the display device 3, the shooting direction of the camera 1 becomes downward. This makes it possible to display on the display device 3 a shot image including the rear lower area of ​​the automobile 100 that was previously outside the shooting range, and to assist in backing up. In this way, it becomes possible to appropriately provide the information required by the driver.

[0021] In addition, the position at which the driver's line of sight is directed on the screen 3a of the display device 3 is detected, and based on that position, the movable device 2 is controlled to change the shooting range of the camera 1, so that the direction in which the shooting range is changed can be finely controlled and the range that the driver wants to see can be appropriately shot. In addition, the shooting range of the camera 1 is changed on the condition that the driver continues to direct his / her gaze at the same position for a predetermined period of time (step S203), thereby preventing the shooting range (i.e., the captured image displayed on the display device 3) from changing more than necessary and against the driver's intention.

[0022] The following describes modified examples. The vehicle may be equipped with a motion detection unit that detects the movement of the driver's head, and the control device 5 may control the movable device 2 to change the shooting range of the camera 1 based on the driver's line of sight detected by the line of sight detection sensor 4 and the movement of the driver's head detected by the motion detection unit. In a mirror-type inner mirror, when the driver changes the viewing angle by peering into the inner mirror in the up, down, left, or right directions, the range visible in the inner mirror changes. In order to reproduce a behavior similar to this, when the driver's line of sight is directed to the screen 3a of the display device 3 and head movement such as peering into the up, down, left, or right directions is detected, the shooting range of the camera 1 is changed. For example, when the driver's line of sight is directed to the screen 3a of the display device 3 and head movement such as peering into the upper left direction is detected, the control device 5 changes the shooting direction of the camera 1 so that the captured image displayed in the upper left area of ​​the screen 3a moves by a predetermined amount toward the center of the screen 3a, as explained using the arrow A in FIG. 4. As a result, the captured image including the part that was previously outside the shooting range is displayed on the display device 3.

[0023] In addition, the gaze detection unit is described as being the gaze detection sensor 4 mounted on the display device 3, but is not limited to this. Anything that is already mounted on the vehicle, such as a sensor for recognizing the driver's drowsiness or a camera, and can detect the driver's gaze, can be used. Similarly, as for the movement detection unit that detects the movement of the driver's head, any thing that is already mounted on the vehicle and can detect the movement of the driver's head can be used.

[0024] Also, although it has been described that the camera 1 is installed at the rear of the automobile 100 and the display device 3 is a digital inner mirror, this is not limited thereto. For example, the camera 1 that captures the rear view from the side of the automobile 100 may be installed at the side of the automobile 100. The display device 3 may be what is called a digital side mirror that is installed in place of a mirror-type side mirror. Furthermore, the display device 3 may be a display device other than a digital inner mirror or a digital side mirror.

[0025] Also, while the example in which the movable device 2 drives the camera 1 to pan and tilt has been described, the present invention is not limited to this. For example, at the rear of the automobile 100, the movable device 2 may be configured to move the camera 1 in up, down, left and right directions while keeping the shooting direction unchanged, thereby varying the shooting range.

[0026] Although the embodiments of the present invention have been described in detail above with reference to the drawings, each embodiment merely shows a specific example of the implementation of the present invention. The technical scope of the present invention is not limited to each embodiment. Various modifications of the present invention are possible without departing from the spirit of the present invention, and these modifications are also included in the technical scope of the present invention. [Explanation of symbols]

[0027] 1: camera, 2: movable device, 3: display device, 3a: screen, 4: gaze detection sensor, 5: control device, 100: automobile

Claims

1. A camera installed in the vehicle; A movable unit that moves the imaging unit to change the imaging range; a display unit that displays an image captured by the imaging unit; A gaze detection unit that detects the driver's gaze; a control unit that controls the movable unit to change a shooting range of the shooting unit based on the driver's line of sight detected by the line of sight detection unit.

2. The gaze detection unit detects to which position on the screen of the display unit the driver's gaze is directed, The vehicle image capturing system according to claim 1 , wherein the control unit controls the movable unit to change the image capturing range of the image capturing unit based on the position detected by the line of sight detection unit.

3. 3. The vehicle image capturing system according to claim 2, wherein the control unit controls the movable unit to change the capturing range of the capturing unit when the driver continues to direct his or her gaze at the same position for a predetermined period of time.

4. A motion detection unit is provided to detect the motion of the driver's head, 2. The vehicle image capturing system according to claim 1, wherein the control unit controls the movable unit to change the capturing range of the capturing unit based on the driver's line of sight detected by the line of sight detection unit and the movement of the driver's head detected by the movement detection unit.

5. 5. The vehicle-mounted image capturing system according to claim 1, wherein the movable portion changes an image capturing direction of the image capturing portion.

6. The photographing unit photographs a rear area of ​​the vehicle, 5. The vehicle image capturing system according to claim 1, wherein the display unit is a display unit that allows a driver to visually recognize a rear view of the vehicle.

Citation Information

Patent Citations

  • Room mirror with built-in monitor

    JP2002120649A