Vehicle periphery monitoring device

Abstract

The present invention provides a vehicle periphery monitoring device capable of easily and appropriately selecting a captured image to be displayed simultaneously with a composite image. The vehicle periphery monitoring device includes a plurality of image capturing units, an image compositing unit, a display unit, and a display control unit. The plurality of camera units can shoot images of the surrounding area of the vehicle and output the images; an image synthesis unit generating a synthesized image by synthesizing the images captured by the imaging unit; the display unit is arranged in the vehicle; and a display control unit controlling the display unit. Wherein the display control unit displays a plurality of switching icons corresponding to the plurality of captured images on the composite image when the composite image generated by the image compositing unit and at least one captured image captured by the image capturing unit are displayed at the same time, and changes the switching icons corresponding to the displayed captured images to a first display form, and when the vehicle exceeds the predetermined speed, changing a predetermined switching icon among the plurality of displayed switching icons to a second display form.

Description

Technical Field

[0001] This invention relates to a monitoring device, and more particularly to a vehicle perimeter monitoring device. Background Technology

[0002] In recent years, efforts to provide access to sustainable transportation systems that also take into account vulnerable groups among transportation participants have become increasingly active. To achieve this goal, research and development are being undertaken to further improve the safety and convenience of transportation through advancements related to autonomous driving.

[0003] Patent Document 1 discloses a vehicle surrounding monitoring device that enhances convenience by helping drivers see the area around their vehicle. It includes: multiple camera devices that capture images of the surrounding area outside the vehicle and output images; an image compositing mechanism for combining the images output by the camera devices and outputting the combined image; and a display control mechanism for displaying the combined image output by the image compositing mechanism in a display device. At least one of the multiple camera devices is mounted on a rearview mirror that can be both extended and retracted. The image compositing mechanism extracts images from each image output by the multiple camera devices according to a predetermined extraction range set for each image, and combines the extracted images to generate the combined image. When the rearview mirror is extended or retracted, the image compositing mechanism changes the extraction range according to the extended or retracted state of the rearview mirror and adjusts the boundary positions between the extracted images constituting the combined image.

[0004] Therefore, regardless of whether the rearview mirror is in the extended or retracted state, the same composite image can be generated.

[0005] [Existing Technical Documents]

[0006] [Patent Literature]

[0007] [Patent Document 1] Japanese Patent Publication No. 2010-069944 Summary of the Invention

[0008] In terms of vehicle operability, how to easily and appropriately switch between displayed images is a challenge for designers in this field.

[0009] This invention aims to improve the operability of selecting captured images to be displayed simultaneously with the synthesized image in order to address the aforementioned problem. Furthermore, it contributes to the development of sustainable delivery systems.

[0010] To achieve the aforementioned objective, the present invention provides a vehicle perimeter monitoring device, comprising multiple camera units, an image synthesis unit, a display unit, and a display control unit. The multiple camera units are capable of capturing images of the area surrounding the vehicle and outputting these images; the image synthesis unit generates a composite image by synthesizing the images captured by the camera units; the display unit is disposed within the vehicle; and the display control unit controls the display unit. Specifically, when the display control unit simultaneously displays the composite image generated by the image synthesis unit and at least one captured image from the camera units, it displays multiple switching icons corresponding to the multiple captured images on the composite image, and changes the switching icon corresponding to the displayed captured image to a first display format; and when the vehicle exceeds a predetermined speed, it changes a predetermined switching icon among the multiple displayed switching icons to a second display format.

[0011] In one embodiment of the present invention, when the vehicle exceeds a predetermined speed, the display control unit changes the switching icon corresponding to the captured image that cannot be displayed when the vehicle exceeds the predetermined speed to a second display form among the multiple switching icons displayed.

[0012] In one embodiment of the present invention, the display control unit switches to a predetermined screen when the switching icon corresponding to the displayed captured image changes to a second display format.

[0013] In one embodiment of the present invention, the display control unit switches to displaying a screen in the display unit that shows images other than the captured images and composite images taken by the camera unit as a predetermined screen.

[0014] In one embodiment of the present invention, the display control unit switches to a captured image that can be displayed as a predetermined screen in the captured image, and changes the switching icon corresponding to the displayable captured image to a first display form.

[0015] In one embodiment of the present invention, when the display control unit switches to simultaneously displaying a composite image and a captured image, it displays a switching icon on the composite image for a predetermined time, and after the predetermined time has elapsed, changes the switching icon to a second display form, and displays the switching icon on the composite image when an operation on the display unit is detected.

[0016] Based on the structure described, the captured images to be displayed simultaneously with the synthesized image can be easily and appropriately selected.

[0017] Based on the above, the vehicle perimeter monitoring device of this embodiment displays switching icons corresponding to the multiple captured images on the composite image when simultaneously displaying images captured by the camera unit and composite images synthesized from multiple captured images. Furthermore, when the vehicle exceeds a predetermined speed and / or a predetermined time, the switching icon corresponding to the displayable captured image is changed. This allows for easy and appropriate selection of the captured images to be displayed simultaneously with the composite image, making it convenient for the driver.

[0018] To make the above features and advantages of the present invention more apparent and understandable, embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0019] Figure 1 This is a schematic block diagram illustrating a vehicle perimeter monitoring device according to an embodiment of the present invention.

[0020] Figure 2 A flowchart illustrating a vehicle perimeter monitoring method according to an embodiment of the present invention.

[0021] Figure 3 This illustrates an example of a vehicle perimeter monitoring device according to an embodiment of the present invention performing a vehicle perimeter monitoring function.

[0022] Figures 4A to 4E This illustrates an example of a vehicle perimeter monitoring device according to an embodiment of the present invention performing a vehicle perimeter monitoring function.

[0023] Figure 5A and Figure 5B This illustrates an example of a vehicle perimeter monitoring device according to an embodiment of the present invention performing a vehicle perimeter monitoring function.

[0024] Figure 6 This illustrates an example of a vehicle perimeter monitoring device according to an embodiment of the present invention performing a vehicle perimeter monitoring function.

[0025] Explanation of reference numerals in the attached figures

[0026] 30, 40, 50, 60: Vehicle images

[0027] 31-35, 41-45, 51-55, 61-65: Toggle icon

[0028] 50a: Vehicle location

[0029] 100: Vehicle perimeter monitoring devices

[0030] 110: Camera Unit

[0031] 120: Image Synthesis Unit

[0032] 130: Display Unit

[0033] 140: Display control unit

[0034] 300, 300a, 400a~400e, 500, 500a, 500b, 600, 600a: Screen

[0035] 310, 410a~410e, 510, 510b, 610, 610a: Composite images

[0036] 320, 420a~420e, 430e, 520, 520b, 620, 620a: Image capture

[0037] S202~S212: Steps Detailed Implementation

[0038] Exemplary embodiments of the present invention will now be described in detail, examples of which are illustrated in the accompanying drawings. Wherever possible, the same component reference numerals are used in the drawings and description to denote the same or similar parts.

[0039] In this embodiment of the invention, when simultaneously displaying a composite image and captured images, a switching icon corresponding to each of the multiple captured images is displayed on the composite image, and the switching icon corresponding to the currently displayed captured image is changed to a first display format. When the vehicle speed exceeds a predetermined speed, a predetermined switching icon in the displayed switching icons is changed to a second display format. Therefore, the captured images to be displayed simultaneously with the composite image can be easily and appropriately selected.

[0040] Figure 1 This is a schematic block diagram illustrating a vehicle perimeter monitoring device according to an embodiment of the present invention. Please refer to... Figure 1 In this embodiment, the vehicle perimeter monitoring device 100 is, for example, configured in a vehicle to monitor the area surrounding the vehicle. In this embodiment, the vehicle may include, for example, a car powered by an internal combustion engine such as a diesel or gasoline engine, an electric car powered by an electric motor, or a hybrid vehicle that combines an internal combustion engine and an electric motor. The vehicle perimeter monitoring device 100 includes multiple camera units 110, an image synthesis unit 120, a display unit 130, and a display control unit 140.

[0041] The camera unit 110 is, for example, a camera or camcorder including a charge-coupled device (CCD), a complementary metal-oxide semiconductor (CMOS) element, or other types of photosensitive elements. It is configured, for example, outside the vehicle, and is capable of capturing images of the area surrounding the vehicle and outputting the captured images. The area surrounding the vehicle includes, for example, the area in front of, behind, to the left of, and to the right of the vehicle, and is not limited thereto.

[0042] For example, the camera unit 110 includes a front camera with its front peripheral area as the camera area; a rear camera with its rear peripheral area as the camera area; a left camera with its left peripheral area as the camera area; and a right camera with its right peripheral area as the camera area. The front camera 11a is, for example, mounted on the front grille of the vehicle; the rear camera is, for example, mounted on the rear tailgate or tailgate of the vehicle; and the left and right cameras are, for example, mounted on the lower ends of the left and right rearview mirrors of the vehicle, respectively.

[0043] The image synthesis unit 120 generates a composite image, for example, by synthesizing images captured by the camera unit 110. For instance, when generating a wide-view (GV) image, the image synthesis unit 120 corrects distortions and other defects in the images captured by the camera unit 110 and changes the viewpoint position, transforming the images output by each camera unit 110 into a top-down view image viewed vertically from above the vehicle. Specifically, the image synthesis unit 120 extracts images from each transformed image obtained through this transformation process (e.g., front, rear, left, and right images) according to a predetermined viewpoint, adjusts the boundary positions of each image, and then synthesizes these images into a composite image, which is then output as a wide-view image.

[0044] Display unit 130, for example, includes a liquid crystal display (LCD), a light-emitting diode (LED) display, or other types of display, and is configured inside the vehicle for performing various displays under the control of display control unit 140. For example, the display screen of display unit 130 can display not only images captured by each camera unit 110 and composite images generated by image synthesis unit 120, but also the vehicle's position and various directional displays can be superimposed on these images.

[0045] The display control unit 140 is used to control the display unit 130. In some embodiments, part or all of the image compositing unit 120 and the display control unit 140 described above can be implemented by a processor executing a program. In other embodiments, part or all of the image compositing unit 120 and the display control unit 140 can also be implemented by hardware such as large-scale integrated circuits (LSI) or application-specific integrated circuits (ASIC), or by a combination of software and hardware. This embodiment does not limit its implementation method.

[0046] Figure 2This is a flowchart illustrating a vehicle perimeter monitoring method according to an embodiment of the present invention. Please refer to... Figure 1 and Figure 2 The vehicle perimeter monitoring method in this embodiment is applicable to Figure 1 The following describes the detailed steps of the vehicle perimeter monitoring method of this embodiment in conjunction with the components of the vehicle perimeter monitoring device 100.

[0047] In step S202, the camera unit 110 captures images of the area surrounding the vehicle and outputs the images. The images include the vehicle's front view (FV), rear view (RV), right side view (SV), etc., and are not limited thereto.

[0048] In step S204, the image synthesis unit 120 generates a synthesized image by synthesizing the images captured by the camera unit 110. In some embodiments, the image synthesis unit 120 transforms the images output by each camera unit 110 into a top-down view image viewed from above in the vertical direction of the vehicle by correcting distortions and other defects in the images captured by the camera unit 110 and changing the viewpoint position, and outputs it as a synthesized image.

[0049] In step S206, the display control unit 140 determines whether to simultaneously display the composite image generated by the image synthesis unit 120 and at least one captured image by the camera unit 110. In some embodiments, the display control unit 140 determines whether to display simultaneously by detecting the user's operation on the vehicle or the vehicle perimeter monitoring device 100. For example, when the user activates the vehicle perimeter monitoring function or when the user performs an operation that requires activating the vehicle perimeter monitoring function, such as reversing, the display control unit 140 determines that the composite image and the camera image should be displayed simultaneously.

[0050] If it is determined that the composite image and the camera image need to be displayed simultaneously, in step S208, the display control unit 140 displays multiple switching icons corresponding to the multiple captured images on the composite image, and changes the switching icon corresponding to the displayed captured image to a first display format. The switching icon is, for example, displayed in or around the vehicle image of the composite image to indicate the viewing angle of the displayed camera image. The first display format, for example, involves changing the color, shape, and size of the switching icon to inform the user which switching icon corresponds to the currently displayed captured image.

[0051] In step S210, the display control unit 140 determines whether the vehicle exceeds a predetermined speed. In some embodiments, the display control unit 140 obtains the corresponding vehicle speed based on the detection signal output by the wheel speed sensor and compares it with a pre-stored or preset predetermined speed to determine whether the vehicle exceeds the predetermined speed. The predetermined speed is, for example, 20 or 30 kilometers per hour or other arbitrary speeds, and is not limited here.

[0052] If it is determined that the vehicle exceeds the predetermined speed, then in step S212, the display control unit 140 changes the predetermined switching icon among the multiple displayed switching icons to a second display format. The second display format may be, for example, not displaying the switching icon or displaying it in grayscale, dashed lines, or other ways, to indicate that the captured image corresponding to the predetermined switching icon cannot be displayed.

[0053] For example, Figure 3 This illustrates an example of a vehicle perimeter monitoring device according to an embodiment of the present invention performing the vehicle perimeter monitoring function. Please refer to... Figure 3 When a user activates the vehicle perimeter monitoring function or performs an operation that requires activating the function, the vehicle perimeter monitoring device displays the composite image 310 and the captured image 320 on the display unit's screen 300. The device also displays switching icons corresponding to multiple captured images on or around the vehicle image 30 in the composite image 310, including a switching icon 31 corresponding to the front view, a switching icon 32 corresponding to the right side view, a switching icon 33 corresponding to the rear side view, a switching icon 34 corresponding to the 3D view, and a switching icon 35 corresponding to the rear view. Specifically, corresponding to the displayed captured image 320, the vehicle perimeter monitoring device changes the display format of switching icon 33 to a first display format to indicate the view corresponding to captured image 320 to the user. At this time, the user can select one of the switching icons 31 to 35 to switch the captured image displayed on the screen 300.

[0054] When the vehicle perimeter monitoring device detects that the vehicle exceeds a predetermined speed, the predetermined switching icon among the multiple switching icons displayed in the composite image 310 in the display unit's screen 300a is changed to a second display format. For example, switching icons 31, 32, 34, and 35 corresponding to the front view, right side view, 3D view, and rear view are changed to not be displayed.

[0055] In some embodiments, when the vehicle exceeds the predetermined speed, the display control unit 140 changes the switching icon corresponding to a captured image that cannot be displayed when the vehicle exceeds the predetermined speed to a second display format among the multiple displayed switching icons. Captured images that cannot be displayed when the vehicle exceeds the predetermined speed include, for example, front view (FV), side view (SV), and 3D view (3DV), and therefore the switching icons corresponding to these views are hidden and not displayed. Captured images that can be displayed when the vehicle exceeds the predetermined speed include, for example, rear side view, rear view, or a combination of rear side view and rear view, and therefore the switching icons corresponding to these views remain displayed.

[0056] For example, Figures 4A to 4E This illustrates an example of a vehicle perimeter monitoring device according to an embodiment of the present invention performing a vehicle perimeter monitoring function.

[0057] Please refer to Figure 4A When the vehicle speed is lower than the predetermined speed, the vehicle perimeter monitoring device can display a wide-view (GV) image 410a and a front view (FV) 420a in the display unit screen 400a. Switching icons 41 to 45 are displayed on or around the vehicle image 40 in the wide-view image 410a, and the switching icon 41 corresponding to the front view 420a is changed to the first display form.

[0058] Please refer to Figure 4B When the vehicle speed is lower than the predetermined speed, the vehicle perimeter monitoring device can display a wide-view (GV) image 410b and a right-side view (SV) 420b in the display unit screen 400b. Switching icons 41 to 45 are displayed on or around the vehicle image 40 in the wide-view image 410b, and the switching icon 42 corresponding to the right-side view 420b is changed to the first display form.

[0059] Please refer to Figure 4C When the vehicle speed is lower than the predetermined speed, the vehicle perimeter monitoring device can display a wide-view (GV) image 410c and a 3D view 420c in the display unit screen 400c. Switching icons 41 to 45 are displayed on or around the vehicle image 40 in the wide-view image 410c, and the switching icon 44 corresponding to the 3D view 420c is changed to the first display form.

[0060] Please refer to Figure 4D When the vehicle speed is lower than the predetermined speed, the vehicle perimeter monitoring device can display a wide-view (GV) image 410d, a rear side view 422d, and a rear view 420d in the display unit screen 400d. Switching icons 41 to 45 are displayed on or around the vehicle image 40 in the wide-view image 410d, and the switching icons 45 and 43 corresponding to the rear view 420d and the rear side view 422d are changed to the first display form.

[0061] Please refer to Figure 4E When the vehicle speed exceeds the predetermined speed, the vehicle perimeter monitoring device cannot display the aforementioned front view (FV) 420a, right side view (SV) 420b, and 3D view, but can only display the rear side view 430e and rear view 420e. Therefore, only the switching icon 45 corresponding to the rear view 420e and the switching icon 43 corresponding to the rear side view 430e are displayed on or around the vehicle image 40 in the wide-angle image 410e, while the switching icons 41, 42, and 44 are changed to the second display form (i.e., not displayed).

[0062] In some embodiments, when the switching icon corresponding to the displayed captured image changes to a second display format, the display control unit 140 switches to a predetermined screen, such as a screen that displays images other than the captured image and composite image captured by the camera unit 110 in the display unit 130, or a captured image that can be displayed as a predetermined screen in the captured image, and changes the switching icon corresponding to the displayable captured image to a first display format.

[0063] For example, Figure 5A and Figure 5B This illustrates an example of a vehicle perimeter monitoring device according to an embodiment of the present invention performing a vehicle perimeter monitoring function.

[0064] Please refer to Figure 5A When the vehicle speed is lower than the predetermined speed, the vehicle perimeter monitoring device can display a wide-view (GV) image 510 and a front view (FV) 520 in the display unit screen 500. Switching icons 51 to 55 are displayed on or around the vehicle image 50 in the wide-view image 510, and the switching icon 51 corresponding to the front view 520 is changed to the first display form.

[0065] When the vehicle speed exceeds the predetermined speed, the vehicle perimeter monitoring device changes the switching icon 51 corresponding to the displayed front view (FV) 520 to the second display form (i.e., not displayed), and switches the displayed screen to a screen that displays images other than captured images and composite images, such as a navigation screen 500a indicating the vehicle position 50a.

[0066] Please refer to Figure 5B When the vehicle speed is lower than the predetermined speed, the vehicle perimeter monitoring device also displays a wide-view (GV) image 510 and a front view (FV) 520 in the display unit screen 500. Switching icons 51 to 55 are displayed on or around the vehicle image 50 in the wide-view image 510, and the switching icon 51 corresponding to the front view 520 is changed to the first display form.

[0067] When the vehicle speed exceeds the predetermined speed, the vehicle perimeter monitoring device changes the switching icon 51 corresponding to the displayed front view (FV) 520 to the second display form (i.e., not displayed), switches to the rear side view 520b that can be displayed as a predetermined screen in the captured image, and changes the switching icon 53 corresponding to the displayable rear side view 520b to the first display form.

[0068] In some embodiments, when switching to simultaneously displaying a composite image and a captured image, the display control unit 140 displays a switching icon on the composite image for a predetermined time, and changes the switching icon to a second display format after the predetermined time has elapsed. Additionally, when an operation on the display unit 130 is detected, the display control unit 140 displays the switching icon on the composite image. Furthermore, in some embodiments, when switching to simultaneously displaying a composite image and a captured image, if an operation is performed on an area outside the switching icon on the composite image, the display control unit 140 changes the switching icon to a second display format. Also, when an operation on the display unit 130 is detected, the display control unit 140 displays the switching icon on the composite image.

[0069] For example, Figure 6 This illustrates an example of a vehicle perimeter monitoring device according to an embodiment of the present invention performing a vehicle perimeter monitoring function.

[0070] Please refer to Figure 6 When the vehicle speed is lower than the predetermined speed, the vehicle perimeter monitoring device can display a wide-view (GV) image 610 and a rear side view 620 in the screen 600 of the display unit. Switching icons 61 to 65 are displayed on or around the vehicle image 60 in the wide-view image 610, and the switching icon 63 corresponding to the rear side view 620 is changed to the first display form.

[0071] When the display of switching icons 61-65 exceeds a predetermined time, the vehicle perimeter monitoring device changes all switching icons 61-62 and 64-65 in the wide-view (GV) image 610, except for the switching icon 63 corresponding to the rear side view 620, to a second display format (i.e., not displayed), retaining only the switching icon 63 corresponding to the rear side view 620. Switching icons 61-62 and 64-65 are re-displayed on the wide-view (GV) image 610 only when the vehicle speed is below a predetermined speed and user operation on the display unit is detected. Furthermore, when the vehicle speed is above a predetermined speed and operation on the display unit 130 is detected, switching icon 65 is re-displayed on the wide-view (GV) image 610. At this time, the re-displayed switching icon corresponds to the switching icon that can be displayed when the vehicle exceeds the predetermined speed.

[0072] In summary, the vehicle perimeter monitoring device of the present invention displays switching icons corresponding to the multiple captured images on the composite image when simultaneously displaying images captured by the camera unit and composite images synthesized from the captured images. Furthermore, when the vehicle exceeds a predetermined speed and / or a predetermined time, the switching icon corresponding to the displayable captured image is changed. This allows for easy and appropriate selection of the captured images to be displayed simultaneously with the composite image, making it convenient for the driver.

[0073] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A vehicle periphery monitoring device characterized by comprising: including: a plurality of imaging units capable of imaging an image of an area around a vehicle and outputting the image; an image synthesizing unit that generates a synthesized image by synthesizing the image imaged by the imaging unit; a display unit disposed in the vehicle; and a display control unit that controls the display unit, wherein the display control unit: when simultaneously displaying the synthesized image generated by the image synthesizing unit and at least one imaged image imaged by the imaging unit, displays a plurality of switching icons corresponding to a plurality of imaged images on the synthesized image, changes the switching icon corresponding to the displayed imaged image to a first display form, and changes a predetermined switching icon among the displayed plurality of switching icons to a second display form when the vehicle exceeds a predetermined speed.

2. The vehicle periphery monitoring apparatus according to claim 1, wherein the display control unit changes, when the vehicle exceeds the predetermined speed, a switching icon corresponding to an imaged image that cannot be displayed when the vehicle exceeds the predetermined speed among the displayed plurality of switching icons to the second display form.

3. The vehicle periphery monitoring apparatus according to claim 1, wherein the display control unit switches to a predetermined screen when the switching icon corresponding to the displayed imaged image is changed to the second display form.

4. The vehicle periphery monitoring apparatus according to claim 3, wherein the display control unit switches to a screen that displays, in the display unit, an image other than the imaged image imaged by the imaging unit and the synthesized image as the predetermined screen.

5. The vehicle periphery monitoring apparatus according to claim 3, wherein the display control unit switches to the imaged image that can be displayed as the predetermined screen among the imaged images, and changes a switching icon corresponding to the displayable imaged image to the first display form.

6. The vehicle periphery monitoring apparatus according to claim 1, wherein the display control unit displays the switching icon on the synthesized image for a predetermined time when switching to simultaneously display the synthesized image and the imaged image, and changes the switching icon to the second display form after the predetermined time elapses, and displays the switching icon on the synthesized image when detecting an operation to the display unit. ​

Images