Image display method and image display device

Abstract

To prevent an occupant from having an illusion about an object behind a vehicle due to switching of a display state when an image captured of a range including the rear of the vehicle is displayed on a display device in a vehicle cabin.SOLUTION: In an image display method, an image of a range including the rear of a vehicle is captured by an imaging device (S1), the image captured by the imaging device is displayed on a display device arranged inside a vehicle cabin (S2), it is determined whether the occupant's line of sight is directed toward the display device (S3), and when the occupant's line of sight is directed toward the display device, the display state for displaying the image on the display device is continuously switched between a first state in which the image is displayed at a predetermined magnification and a second state in which the image is displayed at a magnification higher than the first state (S4), and when the occupant's line of sight is not directed toward the display device, the display state is not switched (S5).SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an image display method and an image display device. [Background technology]

[0002] The image display device described in Patent Document 1 is equipped with a display changeover switch that switches from a first state in which a mirror image of the rear of the vehicle is displayed in the rearview mirror to a second state in which a camera image of the rear view is displayed through the rearview mirror; when switching from the first state to the second state begins, the camera image is displayed so that it matches the mirror image in the rearview mirror, and thereafter the camera image is gradually changed over time. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2020-161919 Summary of the Invention [Problem to be solved by the invention]

[0004] Gradually changing the rear camera image in this way may give the occupants the illusion that an object behind them in the camera image appears larger because it is closer, or that the position of the object in the image has changed because the vehicle has changed lanes. The present invention aims to prevent occupants from having an illusion regarding objects behind the vehicle due to switching of the display state when video captured of an area including the rear of the vehicle is displayed on a display device inside the vehicle cabin. [Means for solving the problem]

[0005] In an image display method according to one aspect of the present invention, an image of a range including the rear of a vehicle is captured by an imaging device, the image captured by the imaging device is displayed on a display device arranged inside the vehicle cabin, it is determined whether the occupant's line of sight is directed toward the display device, and if the occupant's line of sight is directed toward the display device, the display state for displaying the image on the display device is continuously switched between a first state in which the image is displayed at a predetermined magnification and a second state in which the image is displayed at a magnification higher than the first state, and if the occupant's line of sight is not directed toward the display device, the display state is not switched. [Effects of the Invention]

[0006] According to the present invention, when an image captured of an area including the rear of a vehicle is displayed on a display device inside the vehicle, it is possible to prevent occupants from having an illusion about objects behind the vehicle due to switching of the display state. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a diagram illustrating an example of a schematic configuration of a vehicle equipped with an image display device according to an embodiment. [Figure 2] FIG. 2 is a schematic diagram illustrating an example of the arrangement of cameras and display devices. [Figure 3] 4 is a flowchart illustrating an example of an image display method according to the first embodiment. [Figure 4] 2 is a block diagram illustrating an example of a functional configuration of a controller in FIG. 1. FIG. [Figure 5] 10(a) to 10(d) are explanatory diagrams illustrating an example of an image display method according to the second embodiment. [Figure 6] 10(a) to 10(c) are explanatory diagrams illustrating an example of an image display method according to the second embodiment. [Figure 7] 10 is a flowchart illustrating an example of an image display method according to a second embodiment. [Figure 8] 10 is a flowchart illustrating an example of an image display method according to a second embodiment. [Figure 9] 10 is a flowchart illustrating an example of an image display method according to the third embodiment. [Figure 10] 13 is a flowchart illustrating an example of an image display method according to a fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the drawings are schematic and may differ from the actual product. Furthermore, the embodiments of the present invention shown below are examples of devices and methods for embodying the technical concept of the present invention, and the technical concept of the present invention does not limit the structure, arrangement, etc. of component parts to those described below. The technical concept of the present invention can be modified in various ways within the technical scope defined by the claims.

[0009] (First embodiment) (composition) 1 is a diagram showing a schematic configuration example of a vehicle equipped with an image display device according to an embodiment. The vehicle 1 is equipped with an image display device 10 according to the embodiment, an in-vehicle device 20, an object sensor 21, and a vehicle sensor 22. Image display device 10 displays camera images obtained by capturing an area including the rear of vehicle 1 on a display device arranged in the cabin of vehicle 1. Image display device 10 includes rear camera 11, in-vehicle camera 12, passenger camera 13, display device 14, and controller 15. Figure 2 is a schematic diagram of an example of the arrangement of rear camera 11, in-vehicle camera 12, passenger camera 13, and display device 14.

[0010] The rear camera 11 is installed at the rear of the vehicle 1 and captures images of the area behind the vehicle 1. The interior camera 12 is installed at the front of the vehicle cabin (for example, at the position of the rearview mirror) and captures images of the area including the interior of the vehicle and the area behind the vehicle 1. The passenger camera 13 is installed inside the vehicle cabin and captures images of the faces of passengers (for example, the driver) of the vehicle 1. Display device 14 is provided in the vehicle cabin so as to be visible to the occupants, and switches between displaying images captured by rear camera 11 and images captured by in-vehicle camera 12. For example, display device 14 may be disposed in the front of the vehicle cabin (for example, at the position of the rearview mirror).

[0011] For example, the display device 14 may be an electronic mirror (a so-called "smart rearview mirror") that includes a display 14a that displays camera images captured by the rear camera 11 and the in-vehicle camera 12, and a rearview mirror 14b that reflects the rear of the vehicle 1 and transmits the image on the display screen of the display 14a located on the back side. The display device 14 may be switchable between a display state in which the occupant sees a mirror image of the rear of the vehicle 1 through the rearview mirror 14b, and a display state in which the occupant sees the camera image displayed on the display 14a.

[0012] The controller 15 is an electronic control unit that controls the camera images displayed on the display device 14 based on images captured by the rear camera 11 and the in-vehicle camera 12, and also controls the display state of the display device 14. The controller 15 includes a processor 15a and peripheral components such as a storage device 15b. The processor 15a may be, for example, a CPU or an MPU. The storage device 15b may include a semiconductor storage device, a magnetic storage device, an optical storage device, or the like. The functions of the controller 15 described below are realized, for example, by the processor 15a executing a computer program stored in the storage device 15b. The controller 15 may also be formed by dedicated hardware for executing the information processing described below. For example, the controller 15 may include a functional logic circuit configured in a general-purpose semiconductor integrated circuit. For example, the controller 15 may include a PLD such as an FPGA.

[0013] The in-vehicle devices 20 are various devices mounted on the vehicle 1 and are operated by the driver. For example, the in-vehicle devices 20 may be a touch panel or a contact sensor provided on the display portion of the rearview mirror 14b, a physical switch provided on the rearview mirror 14b, a sensor for detecting the angle of the rearview mirror 14b, a steering switch, a console switch, or a turn signal (directional indicator).

[0014] The object sensor 21 detects objects within a predetermined distance range from the vehicle 1. The object sensor 21 detects the surrounding environment of the vehicle 1, such as the relative position of the vehicle 1 and objects present around the vehicle 1, the distance between the vehicle 1 and the objects, and the direction in which the objects are present. The object sensor 21 may include a distance measuring device such as a laser range finder, radar, LiDAR, sonar, or infrared sensor. The vehicle sensor 22 detects various pieces of information (vehicle information) about the vehicle 1. For example, the vehicle sensor 22 may include a vehicle speed sensor that detects the traveling speed of the vehicle 1, a three-axis acceleration sensor that detects the acceleration (including deceleration) in three axial directions of the vehicle 1, a sensor that detects the steering angle of the steering wheel and the turning angle of the steered wheels, and a sensor that detects the gearshift position.

[0015] Next, an overview of the control of the image display device 10 by the controller 15 will be described. The image display device 10 displays on the display 14a camera images of a range including the rear of the vehicle 1 captured by the rear camera 11 and the in-vehicle camera 12. Therefore, the angle of view of the camera images displayed on the display 14a can be changed. For example, the controller 15 may display on the display 14a a camera image with the same angle of view as the mirror image reflected in the rearview mirror 14b, or may display on the display 14a a camera image with a narrower angle of view (i.e., a higher magnification or optical magnification) than the mirror image reflected in the rearview mirror 14b.

[0016] This allows, for example, if there is an object that the occupant is paying attention to behind the vehicle 1, to enlarge and display this object on the display 14a, thereby improving the occupant's visibility of the specific object that is behind the vehicle 1. In the following description, the display state of the display device 14 in which a camera image with the same angle of view as the mirror image reflected in the rearview mirror 14b is displayed on the display 14a and the camera image transmitted through the rearview mirror 14b is visible to the occupant, or a state in which a mirror image of the rear of the vehicle 1 is visible to the occupant through the rearview mirror 14b, may be referred to as the "first state."

[0017] In addition, among the display states of the display device 14, a state in which a camera image with a narrower angle of view than the mirror image reflected in the rearview mirror 14b is displayed on the display 14a, and the camera image transmitted through the rearview mirror 14b is visible to the occupant, is sometimes referred to as the "second state." The controller 15 can gradually (continuously) switch the display state of the display device 14 between the first state and the second state.

[0018] Since the angle of view of the camera image in the second state is narrower than the angle of view of the mirror image in the rearview mirror 14b and the camera image in the first state, the magnification ratio (optical magnification) of the camera image in the second state is higher than the magnification ratio of the mirror image and the camera image in the first state. In the following description, the magnification ratio of the mirror image or camera image in the first state may be referred to as the "first magnification ratio M1," and the magnification ratio of the camera image in the second state may be referred to as the "second magnification ratio M2." For example, the second magnification ratio M2 may be defined as the magnification ratio of the camera image in the second state relative to the mirror image or camera image in the first state (in this case, the reference first magnification ratio M1 is "1").

[0019] For example, when switching from the first state to the second state, the state may be switched from one in which the mirror image reflected in the rearview mirror 14b is visible to the occupants to one in which the camera image captured by the in-vehicle camera 12 at a first magnification M1 is displayed on the display 14a. After that, the magnification of the image displayed on the display 14a may be gradually increased (i.e., the angle of view may be gradually decreased). Thereafter, when the field of view of the image displayed on display 14a is covered by rear camera 11, the camera image displayed on display 14a may be switched to the camera image of rear camera 11. Thereafter, the magnification ratio of the camera image subsequently displayed on display 14a may be gradually increased up to second magnification ratio M2.

[0020] For example, when switching from the second state to the first state, the magnification of the camera image of the rear camera 11 displayed on the display 14a is gradually reduced, and when the field of view of the image to be displayed on the display 14a can no longer be covered by the rear camera 11, the camera image displayed on the display 14a can be switched to the camera image of the in-vehicle camera 12. Thereafter, the magnification of the image displayed on the display 14a may be gradually decreased, and when the magnification of the camera image reaches the first magnification M1, the state may be switched to one in which the mirror image reflected in the rearview mirror 14b is visible to the occupants.

[0021] In this way, if the display state of display device 14 is gradually switched between the first state and the second state, a passenger who sees an object behind on display device 14 may be mistakenly led to believe that the size of the object shown in the image has changed because the object is approaching or moving away. Alternatively, the passenger may be mistakenly led to believe that the following vehicle has changed lanes due to a change in the cropping range of the camera image. Therefore, the controller 15 determines whether the occupant's line of sight is directed toward the display device 14, and if the occupant's line of sight is directed toward the display device 14, it continuously switches the display state for displaying an image on the display device 14 between a first state and a second state, and if the occupant's line of sight is not directed toward the display device 14, it does not switch the display state.

[0022] As a result, the display state of the image starts to change after the occupant looks at the display device 14, making it easier for the occupant to understand that the change in the appearance of the rear object is due to a change in the magnification rate or cropping range of the camera image, and not due to a change in the positional relationship between the vehicle 1 and the rear object. As a result, it is possible to prevent an illusion of the rear object caused by switching the display state.

[0023] 3 is a flowchart of an example of the image display method of the first embodiment. In step S1, the rear camera 11 or the in-vehicle camera 12 captures camera images of a range including the rear of the vehicle 1. In step S2, the controller 15 displays the camera images on the display device 14. In step S3, the controller 15 determines whether or not the occupant's line of sight is directed toward the display device 14. If the occupant's line of sight is directed toward the display device 14 (step S3: Y), the process proceeds to step S4.

[0024] If the passenger's line of sight is not directed toward the display device 14 (step S3: N), the process proceeds to step S5. In step S4, the controller 15 gradually switches the display state of the display device 14 between the first state and the second state, after which the process ends. In step S5, the controller 15 stops switching the display state of the display device 14. After that, the process ends.

[0025] (Second embodiment) 4 is a block diagram showing an example of the functional configuration of the controller 15. The controller 15 includes a target object detection unit 40, a line-of-sight detection unit 41, and an image control unit . The target object detection unit 40 detects a target object behind the vehicle 1 based on the surrounding environment of the vehicle 1 detected by the object sensor 21. For example, the target object detection unit 40 may detect another vehicle (car, motorcycle, or bicycle) present within a predetermined distance behind the vehicle 1 as the target object, or may detect a pedestrian present within a predetermined distance behind the vehicle 1 as the target object.

[0026] The target object detection unit 40 may detect another vehicle within a predetermined distance behind the vehicle 1 as a target object when the acceleration of the other vehicle is greater than or equal to a predetermined value, or when another vehicle within a predetermined distance behind the vehicle 1 is honking its horn or tailgating. The gaze detection unit 41 detects the gaze direction of an occupant (for example, a driver). For example, the gaze detection unit 41 detects the gaze direction of the occupant based on an image of the occupant's face captured by the occupant camera 13.

[0027] The image control unit 42 acquires camera images to be displayed on the display device 14 from the camera images captured by the rear camera 11 and the in-vehicle camera 12. The image control unit 42 also controls the display state of the display device 14 based on the detection result of the target object by the target object detection unit 40 and the detection result of the direction of the occupant's line of sight by the line of sight detection unit 41. For example, when a target object is detected behind the vehicle 1 and the occupant's line of sight is directed toward the display device 14, the image control unit 42 continuously switches the display state of the display device 14 from the first state to the second state.

[0028] Figures 5(a) to 5(d) are explanatory diagrams of an example of switching from the first state to the second state. In Figure 5(a), reference numeral 30 indicates the mirror image reflected in the rearview mirror 14b or the camera image displayed on the display 14a of the display device 14 in the first state, and reference numeral 31 indicates the target object reflected in the mirror image or the camera image. In Figures 5(b) to 5(d), reference numerals 30 and 31 respectively indicate the camera image displayed on the display 14a and the target object reflected in the camera image.

[0029] When the target object detection unit 40 detects the target object 31 in the first state, the image control unit 42 switches from a state in which the mirror image 30 reflected in the rearview mirror 14b is visible to the occupants to a state in which the camera image 30 captured by the in-vehicle camera 12 at the first magnification M1 is displayed on the display 14a. The image control unit 42 sets the range in which the target object 31 appears in the camera image 30 (the range in which the target object 31 is detected in the camera image 30) as the zoom range. As shown in FIG. 5(b), the image control unit 42 displays a graphic 32 of a first color (for example, red) that surrounds the zoom range, superimposed on the camera image 30. In this embodiment, a frame line of the first color (hereinafter referred to as a "first frame line") is displayed as the graphic 32 that surrounds the zoom range.

[0030] The image control unit 42 determines whether the occupant's line of sight is directed inside the first frame line 32 (i.e., whether the occupant is looking inside the first frame line 32). When the occupant's line of sight is directed inside the first frame line 32, the image control unit 42 gradually increases the magnification rate of the camera image as shown in FIG. 5(c). For example, the magnification rate may be gradually increased so that the camera image displayed on the display device 14 is enlarged with the portion inside the first frame line 32 as the center. In other words, the magnification rate may be gradually increased so that the portion inside the first frame line 32 expands to fill the entire screen of the display device 14.

[0031] If the occupant's line of sight is not directed within the first frame line 32 (i.e., if the occupant is not looking within the first frame line 32), the image control unit 42 stops increasing the magnification rate of the camera image and fixes the magnification rate. When the occupant continues to look within the first frame line 32 and the magnification rate of the camera image increases to the second magnification rate M2, the image control unit 42 stops increasing the magnification rate of the camera image and fixes the magnification rate at the second magnification rate M2. At this time, the image control unit 42 may continue to display the first frame line 32 superimposed on the camera image as long as the target object is detected, or may stop displaying the first frame line 32 while the magnification rate of the camera image is fixed at the second magnification rate M2.

[0032] Thereafter, when the target object detection unit 40 no longer detects the target object, the image control unit 42 gradually (continuously) returns the display state of the display device 14 to the first state if the occupant's line of sight is directed toward the display device 14. Figures 6(a) to 6(c) are explanatory diagrams of an example of switching from the second state to the first state. Reference numeral 30 in Figures 6(a) and 6(b) indicates a camera image displayed on the display 14a. Reference numeral 30 in Figure 6(c) indicates a mirror image reflected in the rearview mirror 14b in the first state or a camera image displayed on the display 14a of the display device 14.

[0033] When the target object detection unit 40 no longer detects the target object, the image control unit 42 switches the first frame line 32 shown in Fig. 5(d) to a figure 33 of a second color (e.g., green) different from the first color and displays it superimposed on the camera image 30 (Fig. 6(a)). In this embodiment, a frame line of the second color (hereinafter referred to as the "second frame line") is displayed as the figure 33. 5(d) and 6(a), the first frame line 32 and the second frame line 33 are figures that surround the same range of the camera image captured by the rear camera 11. In other words, the second frame line 33 is a figure that surrounds the range (zoom range) of the camera image that was surrounded by the first frame line 32.

[0034] The image control unit 42 determines whether the occupant's line of sight is directed inside the second frame line 33 (i.e., whether the occupant is looking inside the second frame line 33). When the occupant's line of sight is directed inside the second frame line 33, the image control unit 42 gradually decreases the magnification rate of the camera image as shown in FIG. 6(b). If the occupant's line of sight is not directed within the second frame line 33 (i.e., if the occupant is not looking within the second frame line 33), the image control unit 42 stops reducing the magnification rate of the camera image and fixes the magnification rate.

[0035] When the occupant continues to look inside second frame line 33 and the magnification rate of the camera image decreases to first magnification rate M1, image control unit 42 stops the decrease in the magnification rate of the camera image, fixes the magnification rate at first magnification rate M1, and stops displaying second frame line 33 (FIG. 6(c)). Image control unit 42 switches from a state in which camera image 30 captured by in-vehicle camera 12 is displayed on display 14a to a state in which the mirror image 30 reflected in rearview mirror 14b is visible to the occupant.

[0036] The image control unit 42 may change the second magnification ratio M2 or the target object depending on the driving scene of the vehicle 1. In a driving scene in which the vehicle 1 is driving at a low speed, the distance traveled by the vehicle 1 and other vehicles in a given time period is short, so the second magnification ratio M2 may be set higher so that a relatively narrow range centered on the target object is enlarged and displayed. On the other hand, in a driving scene in which the vehicle 1 is driving at a high speed, the distance traveled by the vehicle 1 and other vehicles in a given time period is long, so the second magnification ratio M2 may be set lower so that a relatively wide range centered on the target object is displayed.

[0037] For example, the image control unit 42 may change the second magnification ratio M2 or the target object depending on whether the driving scene is a general road, a main road, a highway, a parking lot, or a turn. The image control unit 42 may determine the driving scene depending on the gear position detected by the vehicle sensor 22 and the driving speed of the vehicle 1. For example, when the driving scene is an ordinary road, a bicycle, a motorcycle, or an automobile may be detected as an object of interest, and the second magnification ratio M2 may be set to twice the first magnification ratio. Furthermore, for example, when the driving scene is a main road or an expressway, a motorcycle or an automobile may be detected as an object of interest, and the second magnification ratio M2 may be set to 1.5 times the first magnification ratio. Furthermore, for example, when the driving scene is a parking lot or a turn, pedestrians, bicycles, motorcycles, and automobiles may be detected as objects of interest, and the second magnification ratio M2 may be set to four times the first magnification ratio.

[0038] The image control unit 42 may change the second magnification ratio M2 or the target object according to the traveling speed of the vehicle 1. For example, when the vehicle 1 is traveling at a relatively high speed, only motorcycles and automobiles may be detected as target objects, and a relatively low second magnification ratio M2 (for example, 1.5 times the first magnification ratio) may be set. When the vehicle 1 is traveling at a relatively low speed, pedestrians and bicycles may also be detected as target objects in addition to motorcycles and automobiles, and a relatively high second magnification ratio M2 (for example, 2 or 4 times the first magnification ratio) may be set. The image control unit 42 may also set the second magnification ratio M2 based on a user input.

[0039] FIG. 7 is a flowchart of an example of an image display method according to the second embodiment. In step S11, the image control unit 42 displays the camera image on the display device 14 at a first magnification ratio M1. In step S12, the image control unit 42 determines whether or not the target object detection unit 40 has detected an target object. If an target object is detected (step S12: Y), the process proceeds to step S13. If an target object is not detected (step S12: N), the process returns to step S12.

[0040] In step S13, the image control unit 42 displays the first frame line 32 on the display device 14, superimposed on the camera image. In step S14, the image control unit 42 determines whether or not the occupant is looking inside the first frame line 32. If the occupant is looking inside the first frame line 32 (step S14: Y), the process proceeds to step S15. If the occupant is not looking inside the first frame line 32 (step S14: N), the process proceeds to step S16.

[0041] In step S15, the image control unit 42 enlarges the camera image (increases the magnification ratio) around the first frame line 32. Then, the process proceeds to step S17. In step S16, the image control unit 42 fixes the magnification ratio of the camera image. Then, the process proceeds to step S17. In step S17, the image control unit 42 determines whether the magnification ratio of the camera image has increased to the second magnification ratio M2. If the magnification ratio of the camera image has increased to the second magnification ratio M2 (step S17: Y), the process proceeds to step S18. If the magnification ratio of the camera image is less than the second magnification ratio M2 (step S17: N), the process returns to step S14. In step S18, the image control unit 42 fixes the magnification ratio of the camera image to the second magnification ratio M2, after which the process ends.

[0042] FIG. 8 is a flowchart of an example of an image display method according to the second embodiment. In step S21, the image control unit 42 displays the camera image on the display device 14 at the second magnification M2. In step S22, the image control unit 42 determines whether the target object detection unit 40 has detected an target object. If the target object has not been detected (step S22: N), the process proceeds to step S23. If the target object has been detected (step S22: Y), the process returns to step S22.

[0043] In step S23, the image control unit 42 displays the second frame line 33 on the display device 14, superimposed on the camera image. In step S24, the image control unit 42 determines whether or not the occupant is looking inside the second frame line 33. If the occupant is looking inside the second frame line 33 (step S24: Y), the process proceeds to step S25. If the occupant is not looking inside the second frame line 33 (step S24: N), the process proceeds to step S26.

[0044] In step S25, the image control unit 42 reduces the magnification ratio of the camera image. Then, the process proceeds to step S27. In step S26, the image control unit 42 fixes the magnification ratio of the camera image. Then, the process proceeds to step S27. In step S27, the image control unit 42 determines whether the magnification ratio of the camera image has decreased to the first magnification ratio M1. If the magnification ratio of the camera image has decreased to the first magnification ratio M1 (step S27: Y), the process proceeds to step S28.

[0045] If the magnification rate of the camera image is greater than the first magnification rate M1 (step S27: N), the process returns to step S24. In step S28, the image control unit 42 fixes the magnification ratio of the camera image to the first magnification ratio M1. In step S29, the image control unit 42 erases the second frame line 33, and then the process ends.

[0046] (Third embodiment) In the third embodiment, the image control unit 42 determines that the occupant does not want the display image on the display device 14 to be enlarged if the occupant's line of sight is directed toward the part of the display screen of the display device 14 outside the first frame line 32 (i.e., if the occupant is looking at the part of the display screen of the display device 14 outside the first frame line 32) while the display state of the display device 14 is being continuously switched from the first state to the second state.

[0047] In this case, image control unit 42 stops increasing the magnification rate of the camera image displayed on display device 14 and gradually returns the display state of display device 14 to the first state. That is, the magnification rate of the camera image displayed on display device 14 is gradually returned to first magnification rate M1. Furthermore, in order to prevent the recurrence of the enlargement of the camera image that is not desired by the occupants, the function of detecting the target object by the target object detection unit 40 is suppressed for a certain period of time.

[0048] Fig. 9 is a flowchart of an example of an image display method according to the third embodiment. The processing in steps S31 to S34 is the same as steps S11 to S14 shown in Fig. 7. If the occupant is looking inside the first frame line 32 (step S34: Y), the processing proceeds to step S40. If the occupant is not looking inside the first frame line 32 (step S34: N), the processing proceeds to step S35. In step S35, the image control unit 42 determines whether the occupant is looking at the portion of the display screen of the display device 14 outside the first frame line 32 or not.

[0049] If the occupant is looking at the outside area (step S35: Y), the process proceeds to step S36. If the occupant is not looking at the outside area (step S35: N), the process proceeds to step S39. In step S36, the image control unit 42 determines whether the magnification ratio of the camera image has been reduced to the first magnification ratio M1. If the magnification ratio of the camera image has not yet been reduced to the first magnification ratio M1 (step S36: N), the process proceeds to step S37. If the magnification ratio of the camera image has been reduced to the first magnification ratio M1 (step S36: Y), the process proceeds to step S38. In step S37, the image control unit 42 reduces the magnification ratio of the camera image. Thereafter, the process returns to step S38.

[0050] In step S38, the target object detection unit 40 suppresses the target object detection function for a certain period of time, after which the process returns to step S32. On the other hand, if the occupant is not looking at the area outside the first frame line 32 (step S35: N), the image control unit 42 fixes the magnification rate of the camera image in step S39. Then, the process proceeds to step S41. In step S40, the target object detection unit 40 enlarges the camera image around the area of ​​the first frame line 32 (increases the magnification rate). Then, the process proceeds to step S41.

[0051] In step S41, the image control unit 42 determines whether the magnification ratio of the camera image has increased to the second magnification ratio M2. If the magnification ratio of the camera image has increased to the second magnification ratio M2 (step S41: Y), the process proceeds to step S42. If the magnification ratio of the camera image is less than the second magnification ratio M2 (step S41: N), the process returns to step S34. In step S42, the image control unit 42 fixes the magnification ratio of the camera image to the second magnification ratio M2, after which the process ends.

[0052] (Fourth embodiment) In the third embodiment, when a blinker is activated while the display state of display device 14 is being continuously switched from the first state to the second state, image control unit 42 stops increasing the magnification rate of the camera image displayed on display device 14 and gradually returns the display state of display device 14 to the first state. That is, the magnification rate of the camera image displayed on display device 14 is gradually returned to first magnification rate M1.

[0053] Additionally or alternatively, when an operation to cancel the enlargement of the camera image displayed on display device 14 is performed by the occupant while the display state of display device 14 is being continuously switched from the first state to the second state, image control unit 42 may interrupt the increase in the magnification rate of the camera image displayed on display device 14 and gradually return the display state of display device 14 to the first state. That is, the magnification rate of the camera image displayed on display device 14 may gradually return to first magnification rate M1.

[0054] For example, the image control unit 42 may accept an operation of the in-vehicle device 20 (for example, touching the display unit of the rearview mirror 14b, pressing a physical switch provided on the rearview mirror 14b, changing the angle of the rearview mirror 14b, operating a steering switch or a console switch) as an operation to cancel the enlargement of the camera image, or may accept a voice input to cancel the enlargement of the camera image.

[0055] Fig. 10 is a flowchart of an example of an image display method according to the fourth embodiment. The processing in steps S51 to S54 is the same as steps S11 to S14 shown in Fig. 7. If the occupant is looking inside the first frame line 32 (step S54: Y), the processing proceeds to step S55. If the occupant is not looking inside the first frame line 32 (step S54: N), the processing returns to step S54. In step S55, the image control unit 42 determines whether the blinkers are on. If the blinkers are on (step S55: Y), the process proceeds to step S56. If the blinkers are not on (step S55: N), the process proceeds to step S58.

[0056] In step S56, the image control unit 42 determines whether the magnification ratio of the camera image has been reduced to the first magnification ratio M1. If the magnification ratio of the camera image has not yet been reduced to the first magnification ratio M1 (step S56: N), the process proceeds to step S57. If the magnification ratio has been reduced to the first magnification ratio M1 (step S56: Y), the process returns to step S55. In step S57, the image control unit 42 reduces the magnification ratio of the camera image. Thereafter, the process returns to step S55.

[0057] If the blinker is not activated (step S55: N), in step S58 the image control unit 42 determines whether or not the occupant is looking inside the first frame lines 32. If the occupant is looking inside the first frame lines 32 (step S58: Y), the process proceeds to step S59. If the occupant is not looking inside the first frame lines 32 (step S58: N), the process proceeds to step S60. In step S59, the image control unit 42 enlarges (increases the magnification ratio of) the camera image centered on the portion of the first frame line 32. Thereafter, the process proceeds to step S61.

[0058] In step S60, the image control unit 42 fixes the magnification of the camera image, after which the process proceeds to step S61. In step S61, the image control unit 42 determines whether the magnification ratio of the camera image has increased to the second magnification ratio M2. If the magnification ratio of the camera image is less than the second magnification ratio M2 (step S61: N), the process returns to step S55. If the magnification ratio of the camera image has increased to the second magnification ratio M2 (step S61: Y), the process proceeds to step S62. In step S62, the image control unit 42 fixes the magnification ratio of the camera image to the second magnification ratio M2, after which the process ends.

[0059] (Effects of the embodiment) (1) In the image display method, an image of a range including the rear of the vehicle is captured by an imaging device, the image captured by the imaging device is displayed on a display device arranged inside the vehicle cabin, and it is determined whether the occupant's line of sight is directed toward the display device. If the occupant's line of sight is directed toward the display device, the display state for displaying the image on the display device is continuously switched between a first state in which the image is displayed at a predetermined magnification and a second state in which the image is displayed at a magnification higher than the first state. If the occupant's line of sight is not directed toward the display device, the display state is not switched.

[0060] As a result, the display state of the image starts to change after the occupant looks at the display device 14, making it easier for the occupant to understand that the change in the appearance of the rear object is due to a change in the magnification rate or cropping range of the camera image, and not due to a change in the positional relationship between the vehicle 1 and the rear object. As a result, it is possible to prevent an optical illusion from occurring regarding the rear object due to the switching of the display state.

[0061] (2) When a target object behind the vehicle is detected and the occupant's line of sight is directed toward the display device, the display state may be continuously switched from the first state to the second state, thereby improving the occupant's visibility of the target object. (3) The target object may be another vehicle that is present within a predetermined distance behind the vehicle, thereby improving visibility to following vehicles. (4) A first figure in a first color surrounding the target object may be displayed on the display device superimposed on the image, thereby drawing the occupant's attention to the target object.

[0062] (5) If the occupant's line of sight moves away from the display device while the display device is being continuously switched from the first state to the second state, the switching of the display state may be stopped. This allows the occupant to resume the switching after the occupant's line of sight moves back to the display device, making it easier for the occupant to understand that the change in the appearance of rearward objects is due to a change in the magnification rate or cropping range of the camera image. (6) When a target object is no longer detected within a predetermined distance behind the vehicle, the display state may be continuously switched from the second state to the first state. This allows the display state to be restored to the original state so that a wider range can be seen when the target object is no longer present.

[0063] (7) When a target object is no longer detected within a predetermined distance behind the vehicle after being detected, a second figure in a second color different from the first color may be displayed on the display device, superimposed on the image of the area previously surrounded by the first figure, and the display state may be continuously switched from the second state to the first state. This allows the display state to be restored so that the original wide area is visible when the target object is no longer present. This also makes it easier for occupants to understand that the display state is being continuously switched.

[0064] (8) When the occupant is looking at a portion of the image outside the first figure, the switching of the display state from the first state to the second state may be stopped and the display state may be returned to the first state. In this way, if the occupant does not want the target object to be enlarged, the display state can be returned to the original state so that a wider range can be seen. (9) When the turn signal is activated, the changeover of the display state from the first state to the second state may be interrupted and the display state may be returned to the first state, thereby allowing the display state to be restored to the original wide area when the occupant is about to change lanes.

[0065] (10) When an occupant performs an operation to cancel the enlargement of the image displayed on the display device, the switching of the display state from the first state to the second state may be interrupted and the display state may be returned to the first state. In this way, if the occupant does not want the enlarged display of the target object, the display state can be returned to the original state so that a wider range can be seen. (11) The type of target object and the magnification ratio of the second state may be set according to the scene in which the vehicle is traveling and the vehicle speed, thereby allowing the image of the range including the rear of the vehicle to be appropriately enlarged and displayed. [Explanation of symbols]

[0066] 1...vehicle, 10...image display device, 11...rear camera, 12...in-vehicle camera, 13...occupant camera, 14...display device, 14a...display, 14b...room mirror, 15...controller, 15a...processor, 15b...storage device, 20...in-vehicle equipment, 21...object sensor, 22...vehicle sensor, 40...target object detection unit, 41...gaze detection unit, 42...image control unit

Claims

1. An image of the area including the rear of the vehicle is captured by an imaging device, displaying the image captured by the imaging device on a display device arranged inside the vehicle; determining whether the line of sight of the occupant is directed toward the display device; When the line of sight of the occupant is directed toward the display device, a display state in which the image is displayed on the display device is continuously switched between a first state in which the image is displayed at a predetermined magnification rate and a second state in which the image is displayed at a magnification rate higher than that of the first state; The display state is not switched when the line of sight of the occupant is not directed toward the display device.

10. An image display method comprising:

2. 2. The image display method according to claim 1, wherein an object of interest behind the vehicle is detected, and when the occupant's line of sight is directed toward the display device, the display state is continuously switched from the first state to the second state.

3. 3. The image display method according to claim 2, wherein the target object is another vehicle present within a predetermined distance behind the vehicle.

4. 3. The image display method according to claim 2, wherein a first figure of a first color surrounding the target object is displayed on the display device in a superimposed manner on the image.

5. 2. The image display method according to claim 1, wherein the switching of the display state is stopped if the occupant's line of sight moves away from the display device while the display device is being continuously switched from the first state to the second state.

6. 2. The image display method according to claim 1, wherein the display state is continuously switched from the second state to the first state when a target object is no longer detected within a predetermined distance behind the vehicle after the target object has been detected within the predetermined distance behind the vehicle.

7. 5. The image display method according to claim 4, wherein, when a target object is no longer detected after being detected within a predetermined distance behind the vehicle, a second figure of a second color different from the first color that surrounds the image of the range surrounded by the first figure is superimposed on the image and displayed on the display device, and the display state is continuously switched from the second state to the first state.

8. The image display method according to claim 4, characterized in that, when the occupant is looking at a portion of the image outside the first figure, the switching of the display state from the first state to the second state is interrupted and the display state is returned to the first state.

9. 5. The image display method according to claim 4, wherein, when a turn signal is activated, the switching of the display state from the first state to the second state is interrupted and the display state is returned to the first state.

10. The image display method according to claim 4, characterized in that, when the occupant performs an operation to cancel the enlargement of the image displayed on the display device, the switching of the display state from the first state to the second state is interrupted and the display state is returned to the first state.

11. 3. The image display method according to claim 2, wherein the type of the target object and the magnification ratio of the second state are set according to the scene in which the vehicle is traveling.

12. 3. The image display method according to claim 2, wherein the type of the target object and the magnification ratio of the second state are set in accordance with the traveling speed of the vehicle.

13. an imaging device that captures an image of a range including the rear of the vehicle; a display device disposed in the vehicle interior and displaying the image captured by the imaging device; a controller that determines whether or not a line of sight of an occupant is directed toward the display device, and, when the line of sight of the occupant is directed toward the display device, continuously switches a display state in which the image is displayed on the display device between a first state in which the image is displayed at a predetermined magnification rate and a second state in which the image is displayed at a magnification rate higher than that of the first state, and does not switch the display state when the line of sight of the occupant is not directed toward the display device; An image display device comprising:

Images