Image display device, image display method, and image display program

Abstract

To intuitively grasp the center of a first image obtained by imaging the rear side of a vehicle, in a case where the first image and a second image for assisting parking are displayed in parallel.SOLUTION: An image display device comprises an electronic inner mirror 14 being a display unit and a control unit 20. The electronic inner mirror 14 is capable of displaying an image obtained by imaging the rear side of a vehicle when the vehicle moves forward and moves backward, in a display range. In moving back, the control unit 20 displays in the display range a first image obtained by imaging the rear side of the vehicle and a second image different from the first image and configured to check the rear side of the vehicle in parallel and displays a mark indicative of the horizontal center of the first image above and / or below the display range.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an image display device, an image display method, and an image display program for use in a vehicle. [Background technology]

[0002] Patent Document 1 discloses a technology for an electronic mirror camera system that can switch between displaying image information from an electronic mirror camera and image information from an electronic rearview camera on an electronic mirror display for a vehicle mirror. In this technology, the electronic mirror display (electronic inner mirror) switches the display from image information from the electronic mirror camera to image information from the electronic rearview camera. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-164115 A Summary of the Invention [Problem to be solved by the invention]

[0004] Prior art shows that image information displayed on an electronic inner mirror can be switched. However, the prior art does not assume a state in which multiple images are displayed in parallel on an electronic inner mirror. Images displayed on an electronic inner mirror include an image of the rear of the vehicle that is displayed when driving forward, and an image for assisting parking that is displayed when driving backward, and these multiple images may be displayed in parallel. Furthermore, the prior art does not assume how the multiple images would appear when displayed in parallel.

[0005] The present disclosure aims to provide an image display device, an image display method, and an image display program that enable intuitive grasping of the center of a first image captured behind a vehicle and a second image for assisting parking when the first image and the second image are displayed side by side. [Means for solving the problem]

[0006] The image display device described in claim 1 includes a display unit capable of displaying images captured behind the vehicle in a display range when the vehicle is moving forward and backward, and a control unit that, when moving backward, displays a first image captured behind the vehicle and a second image different from the first image for checking the rear of the vehicle side by side in the display range, and displays a mark indicating the left-right center of the first image at least either above or below the display range.

[0007] According to the image display device of claim 1, the first image and the second image are displayed in parallel, and a mark indicating the center of the first image is displayed. By using such a mark indicating the center, when the first image capturing the rear of the vehicle and the second image for assisting parking are displayed in parallel, the center of the first image can be intuitively grasped. This makes it possible to suppress a decrease in visibility of the first image even when multiple images are displayed in parallel.

[0008] The image display device of claim 2 is the image display device of claim 1, in which when the control unit changes from a state in which the first image is displayed in the entire display range when moving forward to a state in which the first image and the second image are displayed side by side in the display range when moving backward, the control unit slides the first image so that one end of the first image moves toward the center while trimming the left and right sides while maintaining the center of the first image, displays the second image in a position adjacent to the first image, and moves the mark along with the sliding of the center position of the first image.

[0009] According to the image display device of claim 2, the first image is slid from a state where it is displayed over the entire display range, and the mark is moved together with the sliding of the center position. By controlling the movement of the mark in accordance with such a slide, the driver can more intuitively grasp that the center of the first image is moving.

[0010] The image display device of claim 3 is the image display device of claim 1 or claim 2, wherein the first image is captured by a first imaging unit that captures an image of the rear of the vehicle, and the second image is captured by a second imaging unit that captures a rear range from a different viewpoint than the first imaging unit and captures an image to assist parking, and the control unit displays the first image in the entire display range when moving forward, and displays the first image and the second image side by side in the display range when moving backward.

[0011] According to the image display device of claim 3, the first image is displayed when moving forward, and the first image and the second image are displayed side by side when moving backward, thereby making it possible to switch the display between moving forward and moving backward.

[0012] The image display device described in claim 4 is the image display device described in any one of claims 1 to 3, wherein the control unit does not display the mark when moving forward, and displays the mark when moving backward.

[0013] According to the image display device of claim 4, when a plurality of images are displayed in parallel, a mark is displayed to assist the driver in intuitively grasping the center of the first image.

[0014] The image display device described in claim 5 is an image display device described in any one of claims 1 to 4, wherein the control unit is capable of setting a display mode including at least one of whether or not to display the mark and the type of display pattern.

[0015] According to the image display device of the fifth aspect, it is possible to set whether or not to display the mark and the display pattern so that the driver can easily understand the mark.

[0016] The image display device according to claim 6 is the image display device according to claim 5, wherein the control unit is capable of setting the display mode of the mark for each driving state including at least one of forward movement and reverse movement of the vehicle.

[0017] According to the image display device recited in claim 6, the detailed display mode of the mark can be set for each driving state.

[0018] The image display method described in claim 7 is a method in which a computer executes a process that enables images captured behind the vehicle to be displayed in a display range of a display unit when the vehicle is moving forward and backward, and when the vehicle is moving backward, a first image captured behind the vehicle and a second image different from the first image for checking the rear of the vehicle are displayed side by side in the display range, and a mark indicating the left-right center of the first image is displayed at least either above or below the display range.

[0019] The image display program described in claim 8 enables a display unit to display images of the rear of the vehicle when the vehicle is moving forward and backward in a display range, and causes a processor to control a computer to execute processing to cause a first image of the rear of the vehicle and a second image for checking the rear of the vehicle, which is different from the first image, to be displayed side by side in the display range of the display unit when the vehicle is moving backward, and to display a mark indicating the widthwise center of the first image at least above or below the display range. Effect of the Invention

[0020] According to the technology of the present disclosure, when a first image capturing an image of the rear of a vehicle and a second image for assisting parking are displayed side by side, the center of the first image can be intuitively grasped. [Brief description of the drawings]

[0021] [Figure 1] 1 is a diagram showing a schematic configuration of an in-vehicle system according to an embodiment; [Diagram 2] FIG. 2 is a block diagram showing a hardware configuration of a vehicle. [Diagram 3] 1A to 1C are diagrams showing examples of display when a first image is displayed on the electronic rear-view mirror and when the first image and a second image are displayed side by side. [Figure 4] 4 is a block diagram showing a functional configuration of a control unit; FIG. [Diagram 5] 1A and 1B are diagrams showing examples of the electronic rear-view mirror's (A) "display when moving forward" and (B) "display when reversing." [Figure 6] FIG. 13 is a diagram showing an example of a slide for switching from (A) "display when moving forward" to (B) "display when moving backward." [Figure 7] 13A and 13B are diagrams showing examples of a case where a depressed type and a protruding type are provided as mark patterns representing depressions or protrusions in an image. [Figure 8] 5 is a flowchart showing the flow of an image display process executed in a control unit. [Figure 9] 10 is a flowchart showing the flow of a setting process executed in a control unit. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] Hereinafter, an example of an embodiment of the present disclosure will be described in detail with reference to the drawings. FIG. 1 shows a schematic configuration of an in-vehicle system 10 according to the present embodiment. The vehicle 12 according to the in-vehicle system 10 includes an electronic inner mirror 14, an external camera 16A, a back monitor camera 16B, and a control unit 20. The installation positions of the external camera 16A and the back monitor camera 16B on the vehicle 12 in FIG. 1 are for convenience, and the external camera 16A and the back monitor camera 16B are installed anywhere on the vehicle 12. For example, the installation positions can be near the license plate, near the rear window, near the rear spoiler, etc. For example, the external camera 16A may be installed on the door mirror in addition to the rear of the vehicle. In this case, the images captured by the multiple external cameras 16A can be synthesized and displayed on the electronic inner mirror 14. The electronic inner mirror 14 is an example of a display unit according to the present disclosure. The external camera 16A is an example of a first imaging unit according to the present disclosure. The back monitor camera 16B is an example of a second imaging unit according to the present disclosure.

[0023] FIG. 2 shows a hardware configuration of the vehicle 12. As shown in FIG. 2, the vehicle 12 includes a central processing unit (CPU) 36, a read only memory (ROM) 38, a random access memory (RAM) 40, and a storage 42 provided in the control unit 20, the electronic inner mirror 14, an external camera 16A, a rear monitor camera 16B, and a liquid crystal monitor 18. Each component is connected to each other so as to be able to communicate with each other via a bus 44. In this embodiment, the ROM 38 or the storage 42 stores an image display program for performing image processing of an image to be displayed on the electronic inner mirror 14. The control unit 20, together with the electronic inner mirror 14, is an example of an image display device according to the present disclosure.

[0024] The electronic inner mirror 14 is provided on the upper side of the front windshield glass of the vehicle 12 and approximately in the center in the vehicle width direction. The electronic inner mirror 14 is provided with a display area 50 for displaying images, and is capable of displaying images captured behind the vehicle 12 when the vehicle 12 is moving forward and backward. The electronic inner mirror 14 is also capable of displaying multiple images. The images displayed on the electronic inner mirror 14 are rendered as continuously captured moving images (video).

[0025] The external camera 16A is a camera provided near the rear end of the vehicle 12 for the purpose of capturing an image (first image 52) to be displayed on the electronic inner mirror 14, and captures an image of the rear of the vehicle 12 as the first image 52. The focal length of the lens of the external camera 16A is set so that a following vehicle at a predetermined distance from the vehicle 12 falls within the angle of view, in accordance with the field of view of an optical inner mirror. Therefore, the blind spot of the external camera 16A is a relatively short range behind the vehicle 12. The first image 52 captured by the external camera 16A is displayed in the entire display range 50 of the electronic inner mirror 14, for example, when the vehicle 12 is traveling forward.

[0026] The back monitor camera 16B is a camera provided near the rear end of the vehicle 12 for the purpose of capturing an image (second image 54) to be displayed on the electronic inner mirror 14, and is a camera that captures the blind spot of the external camera 16A behind the vehicle 12 and captures an image to assist in parking. This allows the back monitor camera 16B to capture the second image 54 to assist in parking the vehicle 12. The second image 54 captured by the back monitor camera 16B is displayed in parallel with the first image 52 in the display range 50 of the electronic inner mirror 14, for example, when the vehicle 12 is traveling in reverse.

[0027] FIG. 3 shows examples of display when the first image 52 is displayed on the electronic inner mirror 14 and when the first image 52 and the second image 54 are displayed in parallel. In FIG. 3, (A) shows the display when the first image 52 is displayed, and (B) shows the display when the first image 52 and the second image 54 are displayed in parallel. When switching from the display (A) to the display (B), a problem occurs in that the center 52A of the display in the left-right direction (width direction) of the first image 52 is shifted, making it difficult to understand the center 52A of the first image 52. Therefore, in this embodiment, a mark 56 indicating the center 52A is introduced so that the driver can intuitively grasp the center 52A of the first image 52. The mark 56 will be described later.

[0028] As described above, the first image 52 captured by the external camera 16A and the second image 54 captured by the back monitor camera 16B are images with different viewpoints. The viewpoints may include cases where the viewpoint position is the same but the viewing angles are different. In other words, images with different viewpoints include images captured by cameras installed at the same position but with different viewing angles. The camera capturing the first image 52 is not limited to the external camera 16A, and similarly, the camera capturing the second image 54 is not limited to the back monitor camera 16B.

[0029] The liquid crystal monitor 18 is a touch panel type monitor that can be operated by the driver of the vehicle 12. The liquid crystal monitor 18 is capable of displaying a setting menu, which will be described later.

[0030] As shown in FIG. 4, in the control unit 20, the CPU 36 executes an image display program to function as an acquisition unit 200, a state detection unit 210, a display control unit 220, and a setting unit 230.

[0031] The acquisition unit 200 acquires a first image from the external camera 16A, and acquires a second image from the rear monitor camera 16B.

[0032] The state detection unit 210 detects the driving state of the vehicle 12. The state detection unit 210 detects at least forward or backward as the driving state, which is necessary for switching the display of the image in the electronic inner mirror 14. The driving state, forward or backward, may be detected using a value related to the shift position among values ​​of various sensors (not shown) mounted on the vehicle 12.

[0033] The display control unit 220 displays an image in the display range 50 of the electronic inner mirror 14. When the driving state is forward, the display control unit 220 displays the first image 52 in the entire display range 50 of the electronic inner mirror 14. When the driving state is reverse, the display control unit 220 displays the first image 52 and the second image 54 side by side in the display range 50 of the electronic inner mirror 14, and displays marks 56 indicating the center 52A of the first image 52 above and below the display range 50.

[0034] FIG. 5 shows examples of (A) "display when moving forward" and (B) "display when moving backward" of the electronic inner mirror 14. As shown in FIG. 5, in (A) display when moving forward, only the first image 52 is displayed, and marks 56 are displayed above and below the center 52A of the first image 52. In (B) display when moving backward, the first image 52 is slid while being trimmed so as to narrow the left and right sides of the image, and the second image 54 is displayed in a position adjacent to the first image 52. In (B) display when moving backward, the mark 56 is moved and displayed as the position of the center 52A of the first image 52 is slid. In this example, the mark 56 is triangular in shape. Also, the mark 56 is bordered with a color opposite to the color of the mark 56 itself. For example, if the color of the mark 56 itself is black, the color of the border is white. Note that the color is not limited to the opposite color, and any color scheme that takes into consideration visibility different from the color of the mark 56 itself may be adopted.

[0035] Here, the slide control of the first image 52 performed when switching from the "display when moving forward" to the "display when reversing" will be described. FIG. 6 shows an example of a slide when switching from (A) the "display when moving forward" to (B) the "display when reversing". As shown in FIG. 6, in (A) the display when moving forward, the left end 52B of the first image 52 is near the left end of the display range 50. In (B) the display when reversing, the state in which the first image 52 is displayed in its entirety is switched to a state in which the first image 52 and the second image 54 are displayed side by side. In the switching, the left end 52B of the first image 52 is slid so as to move toward the center of the display range 50. The slide control is performed by moving the left end 52B of the first image 52 toward the center of the display range 50 while trimming the left and right sides gradually narrower while maintaining the display of the original center 52A of the first image 52. As a result of this slide control, one side of the first image 52 in the display range 50 becomes empty, so the second image 54 is displayed in a position adjacent to the first image 52. The second image 54 may be displayed in such a manner that it slides in together with the sliding of the first image 52, or may fade in when the sliding is completed. In this manner, by the slide control, the center 52A of the display in the image of the first image 52 is maintained, while the position of the center 52A within the display range 50 slides as indicated by the arrow 52C. Thus, the display control unit 220 moves the display position of the mark 56 together with the sliding of the position of the center 52A.

[0036] The setting unit 230 sets the display mode of the mark 56. As the display mode, whether or not to display the mark 56 and the type of display pattern of the mark 56 can be set. The setting by the setting unit 230 is performed by displaying a setting menu on the liquid crystal monitor 18 mounted on the vehicle and accepting a setting operation. Whether or not to display the mark 56 may be set for each of the driving states including forward and reverse. For example, the mark 56 may be set to be hidden when forward and to be displayed when reverse. Also, the mark 56 may be set to be displayed only at the upper part when forward and to be displayed at the upper and lower parts when reverse. Also, the driving state when reverse may be set in detail, and the mark 56 may be set to flash for 10 seconds until the start of reverse and to be constantly displayed after 10 seconds. Also, it may be set to display only the upper part or only the lower part of the display range 50. Note that instead of the liquid crystal monitor 18, a setting menu may be displayed on a mobile terminal or the like carried by the driver to accept a setting operation. As shown in FIG. 7, the display pattern of the mark 56 may be a recessed type or a protruding type, which represents a recess or a protrusion in the image. FIG. 7 shows an example of the display pattern of the mark 56 at the bottom of the display range 50. The recessed type includes a triangular display pattern of (a1), a rounded corner type of (a2), and a rectangular display pattern of (a3), and the protruding type includes a triangular display pattern of (b1), a rounded corner type of (b2), and a rectangular display pattern of (b3). The recessed type can reduce the area covered by the display range 50 compared to the protruding type. The display pattern is not limited to this example, and a display pattern of any shape, such as another polygon, may be used. The display menu is not limited to being displayed on the liquid crystal monitor 18, and the setting menu may be displayed on the electronic inner mirror 14 so that the switch operation can be performed.

[0037] (Processing flow) Next, the operation of the control unit 20 will be described. Fig. 8 shows the flow of image display processing by the control unit 20. Fig. 9 shows the flow of setting processing by the control unit 20. In the control unit 20, the CPU 36 reads an image display program from the ROM 38 or storage 42, expands it into the RAM 40, and executes it, thereby performing image display processing and setting processing.

[0038] First, the image display process (FIG. 8) will be described. In step S100, the CPU 36 detects whether the vehicle is traveling forward or backward.

[0039] In step S101, the CPU 36 determines whether or not the vehicle is traveling forward. If the vehicle is traveling forward, the process proceeds to step S102, and if the vehicle is not traveling forward, the process proceeds to step S104.

[0040] In step S102, the CPU 36 acquires a first image 52 from the external camera 16A.

[0041] In step S103, the CPU 36 displays the first image 52 over the entire display range 50 of the electronic inner mirror 14, and displays marks 56 indicating the center 52A of the first image 52 above and below the display range 50. After this step, the process returns to step S100 and is repeated, maintaining the display of the first image 52 and the marks 56 as long as the vehicle is traveling forward.

[0042] In step S104, the CPU 36 determines whether or not the vehicle is traveling backward. If the vehicle is traveling backward, the process proceeds to step S105, and if the vehicle is not traveling backward, the process returns to step S100 and the process is repeated.

[0043] In step S105, the CPU 36 acquires a first image 52 from the external camera 16A and a second image 54 from the rear monitor camera 16B.

[0044] In step S106, the CPU 36 performs control to display the first image 52 and the second image 54 side by side by sliding the first image 52, and to move and display the mark 56. In this control, the first image 52 is slid while being trimmed so as to narrow the left and right sides of the image, the second image 54 is displayed in a position adjacent to the first image 52, and the display position of the mark 56 is moved together with the sliding of the position of the center 52A. After this step, the process returns to step S100 and is repeated, and the side by side display of the first image 52 and the second image 54 and the display of the mark 56 are maintained as long as the driving state is reverse.

[0045] Next, the setting process (FIG. 9) will be described. In step S200, the CPU 36 determines whether or not a setting operation for the liquid crystal monitor 18 has been accepted. If accepted, the process proceeds to step S201, and if not accepted, the process of this step is repeated. The setting operation is an operation relating to whether or not to display the mark 56, an operation relating to the type of display pattern of the mark 56, etc.

[0046] In step S201, the CPU 36 determines whether the received setting operation is an operation regarding the presence or absence of display. If it is an operation regarding the presence or absence of display, the process proceeds to step S202, and if it is not an operation regarding the presence or absence of display, the process proceeds to step S203.

[0047] In step S202, the CPU 36 reflects the display / non-display setting made by the setting operation in the mark 56 to be displayed on the electronic inner mirror 14.

[0048] In step S203, the CPU 36 determines whether the received setting operation is an operation related to the type of display pattern. If the received setting operation is an operation related to the type of display pattern, the process proceeds to step S204, and if the received setting operation is not an operation related to the type of display pattern, the process ends.

[0049] In step S204, the CPU 36 reflects the setting of the type of display pattern set by the setting operation in the mark 56 to be displayed on the electronic inner mirror 14. (summary) The electronic inner mirror 14 of this embodiment is provided with a display range 50 for displaying an image, and is capable of displaying multiple images. When the vehicle 12 is traveling forward, the control unit 20 displays the first image 52 in the entire display range 50 of the electronic inner mirror 14. When the vehicle 12 is traveling backward, the control unit 20 displays the first image 52 and the second image 54 in parallel in the display range 50 of the electronic inner mirror 14, and displays a mark 56 indicating the center 52A of the first image 52 above and below the display range 50. When the first image captured behind the vehicle and the second image for assisting parking are displayed in parallel, the mark 56 indicating the center 52A allows the user to intuitively grasp the center of the first image. This makes it possible to suppress a decrease in visibility of the first image even when multiple images are displayed in parallel.

[0050] In the above embodiment, various processes executed by the CPU 36 by reading the software (program) may be executed by various processors other than the CPU. In this case, examples of the processor include a PLD (Programmable Logic Device) such as an FPGA (Field-Programmable Gate Array) whose circuit configuration can be changed after manufacture, and a dedicated electric circuit such as an ASIC (Application Specific Integrated Circuit) which is a processor having a circuit configuration designed exclusively for executing a specific process. In addition, each of the above-mentioned processes may be executed by one of these various processors, or may be executed by a combination of two or more processors of the same or different types (for example, a plurality of FPGAs, a combination of a CPU and an FPGA, etc.). In addition, the hardware structure of these various processors is, more specifically, an electric circuit that combines circuit elements such as semiconductor elements.

[0051] In the above embodiment, each program has been described as being prestored (installed) in a non-transitory recording medium that can be read by a computer. For example, the image display program in the control unit 20 is prestored in the storage 42. However, the present invention is not limited to this, and each program may be provided in a form recorded in a non-transitory recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory. Also, the program may be downloaded from an external device via a network.

[0052] The process flow described in the above embodiment is merely an example, and unnecessary steps may be deleted, new steps may be added, or the process order may be changed without departing from the spirit of the invention. [Explanation of symbols]

[0053] 12 Vehicles 14 Electronic rearview mirror 16A External Camera 16B Rearview camera 20 Control section

Claims

1. a display unit capable of displaying, in a display range, an image captured behind the vehicle when the vehicle is moving forward and backward; a control unit that, during the reverse movement, displays in parallel in the display range a first image capturing an image of the rear of the vehicle and a second image for confirming the rear of the vehicle, the second image being different from the first image, and displays a mark indicating a center in the left-right direction of the first image at least either above or below the display range; An image display device comprising:

2. The control unit is When the first image and the second image are to be displayed side by side in the display range during the backward movement from a state in which the first image is displayed in the entire display range during the forward movement, 2. The image display device of claim 1, wherein the first image is slid so that one end of the first image is brought closer to the center while trimming the left and right sides while maintaining the center of the first image, the second image is displayed at a position adjacent to the first image, and the mark moves along with the sliding of the center position of the first image.

3. The first image is captured by a first imaging unit that captures an image of the rear of the vehicle, and the second image is captured by a second imaging unit that captures a rear range having a different viewpoint from that of the first imaging unit and captures an image for assisting parking, 3. The image display device according to claim 1, wherein the control unit causes the first image to be displayed in the entire display range when moving forward, and causes the first image and the second image to be displayed side by side in the display range when moving backward.

4. 4. The image display device according to claim 1, wherein the control unit does not display the mark when the vehicle is moving forward, and displays the mark when the vehicle is moving backward.

5. 5. The image display device according to claim 1, wherein the control unit is capable of setting a display mode including at least one of whether or not to display the mark and a type of a display pattern.

6. The image display device according to claim 5 , wherein the control unit is capable of setting the display mode of the mark for each running state including at least one of forward movement and reverse movement of the vehicle.

7. An image captured behind the vehicle when the vehicle is moving forward and backward can be displayed in a display range of the display unit, During the reverse movement, a first image capturing an image of the rear of the vehicle and a second image for checking the rear of the vehicle, which is different from the first image, are displayed in parallel in the display range, and a mark indicating the center of the first image in the left-right direction is displayed at least either above or below the display range. An image display method in which processing is performed by a computer.

8. An image captured behind the vehicle when the vehicle is moving forward and backward can be displayed in a display range of the display unit, During the reverse movement, a first image capturing an image of the rear of the vehicle and a second image for checking the rear of the vehicle, which is different from the first image, are displayed in parallel in the display range, and a mark indicating the center of the first image in the left-right direction is displayed at least either above or below the display range. An image display program that causes a computer to carry out processing.

Images