Image display method and image display apparatus

Abstract

In this image display method executed by a computer that has a computing unit and an input / output unit, the computing unit generates a display image (35) of a display (75) in a vehicle cabin having a mirror face (73) that reflects the rearward of a vehicle (1) on the basis of a cropped image (33) cropped out from a rearward image (31) of the vehicle (1) shot by a camera (3). The size of the display image (35) matches the size of the display (75). When a shift position of the vehicle (1) is switched from a position other than reverse to a reverse position, the computing unit changes an angle of view value when the cropped image (33) is cropped out from the rearward image (31), to a predetermined angle of view value (θR). The predetermined angle of view value (θR) is an angle of view value that is greater than the angle of view value at a shift position immediately before the position was switched to the reverse position.

Description

Image display method and image display device

[0001] The present invention relates to an image display method and an image display device.

[0002] In the technology of Patent Document 1, a display image to be displayed on a display in a vehicle cabin that has a mirrored surface that reflects the rear of the vehicle is generated based on a cut-out image cut out from a rear image captured by a vehicle camera, according to the size of the display. When switching from a state in which the rear of the vehicle is reflected in the mirrored surface to a state in which the display image is displayed on the display, the angle of view value when cutting out the cut-out image from the rear image is determined so that the size of the following vehicle in the displayed image is equal to the size of the following vehicle reflected in the mirrored surface.

[0003] Patent No. 6624105

[0004] The technology of Patent Document 1 aims to appropriately display an image of a following vehicle on a display in a vehicle interior having a mirrored surface. Patent Document 1 does not mention displaying images of vehicles other than the following vehicle. The object of the present invention is to appropriately display an image of a vehicle reversing on a display in a vehicle interior having a mirrored surface.

[0005] In order to solve the above-mentioned problems, an image display method and an image display device according to one aspect of the present invention use a computer having a calculation unit and an input / output unit to display an image on a display in a vehicle cabin having a mirrored surface that reflects the rear of the vehicle. The calculation unit generates a display image of a size that matches the size of the display based on a cut-out image cut out from an image of the rear of the vehicle captured by a camera. When the vehicle's shift position is switched from a position other than reverse to the reverse position, the calculation unit changes the angle of view used when cutting out the cut-out image from the rear image to a predetermined angle of view. The predetermined angle of view is a value larger than the angle of view at the position immediately before the shift position is switched to the reverse position.

[0006] According to the present invention, an image of a vehicle reversing can be appropriately displayed on a display in a vehicle interior having a mirror surface.

[0007] FIG. 1 is a block diagram showing the configuration of a vehicle equipped with an image display device according to one embodiment of the present invention. FIG. 2 is a diagram showing a cropped image cropped by an image generation unit from a rear image captured by the camera of FIG. 1. FIG. 3 is a diagram showing the configuration of a rear display unit of FIG. 1. FIG. 4 is a diagram explaining an example of the cropped field angle of a cropped image before and after change by the field angle change unit of FIG. 1. FIG. 5 is a diagram explaining an example of the cropped field angle of a cropped image corresponding to the distance between the vehicle and an object of interest of FIG. 1. FIG. 6 is a flowchart showing an example of the procedure of an image display method according to one embodiment of the present invention, which is executed by the image display device of FIG. 1.

[0008] Hereinafter, an embodiment of the present invention and its modifications will be described with reference to the drawings. In the description of the drawings, the same parts are given the same reference numerals and the description thereof will be omitted.

[0009] 1 is a block diagram showing the configuration of a vehicle equipped with an image display device according to one embodiment of the present invention. As shown in FIG. 1, the vehicle 1 includes a camera 3, an object detection unit 5, a rear display unit 7, and an image display device 9.

[0010] The camera 3 is disposed at the rear end of the vehicle 1. The rear end of the vehicle 1 may be the rear end of a roof portion (not shown) of the vehicle 1. The camera 3 has an imaging element such as a charge-coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). The camera 3 captures an image of the area behind the vehicle 1. FIG. 2 shows an example of a rear image 31 of the vehicle 1 captured by the camera 3 in a parking lot 36. The captured rear image 31 is output to the image display device 9 and stored, for example, in an external storage device (not shown) of the image display device 9.

[0011] The object detection unit 5 in FIG. 1 measures the relative position of an object present around the vehicle 1 relative to the vehicle 1. The objects whose relative positions are measured by the object detection unit 5 from the vehicle 1 include targets of interest that require attention when the vehicle 1 is traveling. Targets of interest include, for example, lines drawn on the road surface, obstacles on the road, and moving objects on the road. Moving objects include pedestrians, bicycles, and other vehicles other than the vehicle 1. The relative position measured by the object detection unit 5 includes information on the distance from the vehicle 1 to the object. The detected relative position is output to the image display device 9 and stored, for example, in an external storage device (not shown) of the image display device 9. The object detection unit 5 may be any sensor that can detect the distance to an object present around the vehicle 1. For example, a stereo camera can be used as the object detection unit 5. The object detection unit 5 may also include a light detection and ranging (LiDAR), laser radar, or a time-of-flight (ToF) camera.

[0012] The rear display unit 7 is provided, for example, on the roof of the vehicle 1. The rear display unit 7 is disposed in a position forward of the driver's seat of the vehicle 1, between the driver's seat and the passenger seat. As shown in FIG. 3 , the rear display unit 7 includes a frame 71, a mirror surface 73, and a display 75. The frame 71 is attached to the roof, and a half mirror is fitted into an opening of the frame 71 facing the rear of the vehicle 1. The mirror surface 73 is formed by the surface of the half mirror fitted into the frame 71. The display 75 is housed inside the frame 71, and a screen 77 of the display 75 is disposed behind the mirror surface 73. The rear display unit 7 can switch between a state in which the rear of the vehicle is reflected in the mirror surface 73 through the rear window and a state in which the display 75 displays an image on the screen 77. For example, when the angle of the frame 71 relative to the roof is changed from an angle at which the rear of the vehicle reflected in the mirror surface 73 can be seen from the driver's seat to another angle, the display 75 may be configured to display an image on the screen 77.

[0013] The image display device 9 of FIG. 1 includes, for example, a general-purpose microcontroller as a computer and the external storage device described above. The microcontroller includes a calculation unit and an input / output unit. The microcontroller includes a central processing unit (CPU) and a memory. The memory includes a read-only memory (ROM) and a random access memory (RAM). The microcontroller can virtually configure multiple information processing circuits by having the CPU execute programs stored in the memory. The multiple information processing circuits can constitute the calculation unit of the image display device 9. A shift position signal indicating the shift position of the vehicle 1 is input to the input / output unit. The input / output unit can receive the shift position signal, for example, from an electronic control unit (ECU) (not shown) installed in the vehicle 1. The calculation unit can constitute an image generation unit 91 and a field of view change unit 93 (described later) of the image display device 9.

[0014] In this embodiment, an example is shown in which multiple information processing circuits built in a microcontroller are realized by software. Of course, it is also possible to configure the information processing circuits by providing dedicated hardware for executing the information processing described below for each unit 91, 93 of the image display device 9. Alternatively, the multiple information processing circuits may be configured by individual hardware. The dedicated hardware includes devices such as application specific integrated circuits (ASICs) and conventional circuit components arranged to perform the functions of each unit 91, 93.

[0015] 2, the image generation unit 91 cuts out a portion of the rear image 31 of the vehicle 1 stored in the external storage device, and generates a display image 35 based on the cut-out image 33. The display image 35 is an image to be displayed on the screen 77 of the display 75. The size of the display image 35 matches the size of the screen 77. The range in which the image generation unit 91 cuts out the cut-out image 33 from the rear image 31 can be set by the angle of view value. When the set angle of view value is large, the cut-out image 33 will be a wider portion of the rear image 31 than when the set angle of view value is small.

[0016] 1 changes the angle of view used to extract the extracted image 33 from the rear image 31 to a predetermined angle of view when the shift position of the vehicle 1 indicated by the shift position signal is switched from a position other than reverse to the reverse position. As shown in FIG. 4, the predetermined angle of view θR is a larger angle of view than the normal angle of view θN at the position immediately before the shift position is switched to the reverse position. The predetermined angle of view θR can be set to a value at most equal to the angle of view determined by the optical system of the camera 3.

[0017] In particular, when the microcontroller determines that a target of interest exists in the rear image 31, the angle-of-view changing unit 93 may set a predetermined angle-of-view value θR as the angle-of-view value at which the display image 35 including at least the target of interest is displayed on the display 75. The microcontroller may, for example, identify the imaging range of the rear image 31 based on the orientation of the vehicle 1 and the angle of view determined by the optical system of the camera 3, and determine whether the target of interest, the relative position of which is measured by the object detection unit 5, exists in the rear image 31. The orientation of the vehicle 1 may be detected or estimated, for example, using position information of the vehicle 1. The position information of the vehicle 1 may be acquired, for example, from the output of a GNSS (Global Navigation Satellite System) sensor or a gyro sensor, or from the result of detecting the vehicle 1's own position using odometry. The presence of the target of interest in the rear image 31 may be determined, for example, by the microcontroller performing image analysis of the rear image 31 captured by the camera 3. For example, edge detection filtering, moving object detection using optical flow, or the like may be used for the image analysis.

[0018] When a target of interest is present in the rear image 31, the predetermined angle of view θR may be a fixed value regardless of the distance between the target of interest and the vehicle 1, or may be a variable value set corresponding to the distance between the target of interest and the vehicle 1 measured by the object detection unit 5, for example. Fig. 5 shows examples of predetermined angle of view values ​​θRn and θRf for allowing the display image 35 to include the target of interest Sn and Sf, respectively, when the target of interest Sn and Sf have different distances Ln and Lf from the vehicle 1 and are present in the rear image 31. When the target of interest Sn and Sf in the rear image 31 are to be included in the display image 35, the predetermined angle of view θRn when the distance Ln between the target of interest Sn and the vehicle 1 is short may be set to a larger value than the predetermined angle of view θRf when the distance Lf between the target of interest Sf and the vehicle 1 is long.

[0019] As described above, the attention targets Sn and Sf include, for example, lines drawn on the road surface, obstacles on the road, and moving objects on the road. For example, the parking stalls 37 in the parking lot 36 shown in FIG. 2 or lane markings on the road correspond to lines drawn on the road surface and are included in the attention targets Sn and Sf. For example, the bollards 38 installed at the entrances to the parking stalls in the parking lot 36, the pillars or walls between the parking stalls, or fixed objects such as parked vehicles and construction signs correspond to obstacles on the road and are included in the attention targets Sn and Sf. For example, other vehicles 39 moving in the parking lot 36, or pedestrians, bicycles, and other vehicles around the vehicle 1 correspond to moving objects on the road and are included in the attention targets Sn and Sf. The angle-of-view change unit 93 changes the angle-of-view values ​​of the cropped image 33 to predetermined angle-of-view values ​​θR, θRf, and θRn so that the display image 35 includes the attention targets Sn and Sf determined by the microcontroller to be present in the rear image 31.

[0020] An example of the procedure of an image display method according to one embodiment of the present invention, which is executed by the image display device 9, will be described with reference to the flowchart of Fig. 6. The microcontroller sets the field of view value when cropping the cropped image 33 from the rear image 31 to a normal field of view value θN when the vehicle 1 is in a shift position other than reverse (step S101). The normal field of view value θN may be a fixed value or a variable value that changes depending on the condition of the vehicle 1 itself or the conditions around the vehicle 1. The microcontroller checks whether the shift position of the vehicle 1, indicated by the shift position signal input to the input / output unit, has been switched from a position other than reverse to the reverse position (step S103).

[0021] If the vehicle has not been switched to the reverse position (NO in step S103), step S103 is repeated. If the vehicle has been switched to the reverse position (YES in step S103), the microcontroller checks whether any of road lines, road obstacles, and moving objects are present in the rear image 31 (steps S105 to S109). If none of road lines, road obstacles, and moving objects are present in the rear image 31 (NO in steps S105 to S109), the angle-of-view changing unit 93 changes the angle-of-view value of the cropped image 33 to the first angle-of-view value (step S111). If any of road lines, road obstacles, and moving objects are present in the rear image 31 (YES in any of steps S105 to S109), the angle-of-view changing unit 93 changes the angle-of-view value of the cropped image 33 to the second angle-of-view value (step S113). The first and second angle of view values ​​each correspond to a predetermined angle of view θR and are larger than the normal angle of view θN.

[0022] When the normal angle of view θN is a variable value, the first angle of view value and the second pixel value corresponding to the predetermined angle of view θR are set to values ​​greater than the normal angle of view θN when the vehicle 1 was in the position immediately before switching to the reverse position. The second pixel value may be set for each type of attention target Sn, Sf confirmed to exist in the rear image 31, or may be set commonly regardless of the type of attention target Sn, Sf. The types of attention targets Sn, Sf include, for example, a set of lines drawn on the road surface, obstacles on the road, moving objects on the road, etc., which constitute one type of attention target Sn, Sf.

[0023] After changing the angle of view of the cropped image 33 to the first angle of view or the second angle of view in step S111 or S113, the microcontroller checks whether the shift position of the vehicle 1 has been switched to a position other than reverse (step S115). If the shift position remains reverse (NO in step S115), the microcontroller returns to step S105. If the shift position has been switched to a position other than reverse (YES in step S115), the microcontroller returns to step S101.

[0024] In the image display device 9 of this embodiment, when the shift position of the vehicle 1 is switched from a position other than reverse to reverse, the angle of view when cutting out the cropped image 33 from the rear image 31 is changed to a predetermined angle of view θR that is larger than the angle of view when the shift position was immediately before. When the vehicle 1 is switched to the reverse position, a wider range of the rear image 31 is cropped into the cropped image 33 than when the shift position was immediately before, other than reverse, and the display image 35 generated based on the cropped image 33 includes a wider range of the rear image 31. When the shift position of the vehicle 1 is switched to the reverse position, the display image 35 of a wider range of the rear image 31 can be appropriately displayed on the screen 77 of the rear display unit 7, making it easier to check the condition behind the vehicle 1 in the display image 35.

[0025] In this embodiment, even if the microcontroller cannot determine the attention targets Sn and Sf in the rear image 31, the angle of view for cutting out the cut-out image 33 is increased from the normal angle of view θN to a predetermined angle of view θR that is larger than the normal angle of view θN. Even if the microcontroller cannot determine the attention targets Sn and Sf in the rear image 31, the state behind the vehicle 1 can be easily confirmed on the display image 35.

[0026] For example, in a parking lot 36, by changing the angle of view value of the cropped image 33 to a predetermined angle of view value θR, the display image 35 including the parking spaces 37 and the car stops 38 in the rear image 31 can be displayed, making it easier to check the availability of the parking spaces in each parking space 37 in the parking lot 36.

[0027] In this embodiment, when the attention targets Sf and Sn are present in the rear image 31, the angle-of-view changing unit 93 sets a predetermined angle-of-view value θR as the angle-of-view value at which the display image 35 including at least the attention targets Sf and Sn is displayed on the display 75. This configuration makes it easier to confirm the attention targets Sf and Sn in the display image 35. This configuration may be omitted.

[0028] In this embodiment, when targets of interest Sn and Sf in the rear image 31 are included in the display image 35, the predetermined angle of view θRn when the distance Ln between the target of interest Sn and the vehicle 1 is short is set to a larger angle of view than the predetermined angle of view θRf when the distance Lf between the target of interest Sf and the vehicle 1 is long. With this configuration, even if the position of the target of interest Sn is close to the vehicle 1, the cropped image 33 can be cropped from the rear image 31 so that the target of interest Sn is displayed in the display image 35. This configuration may be omitted.

[0029] The above-described embodiment is an example of the present invention. Therefore, the present invention is not limited to the above-described embodiment, and various modifications can be made to the design and other aspects of the present invention without departing from the technical concept of the present invention.

[0030] REFERENCE SIGNS LIST 1 Vehicle 9 Image display device 31 Rear image 33 Extracted image 35 Display image 37 Parking space (line) 38 Car stop (obstacle) 39 Other vehicle (moving object) 73 Mirror surface 75 Display 91 Image generation unit 93 Viewing angle change unit Lf, Ln Distance Sf, Sn Target of interest θN Normal view angle value θR Predetermined view angle value θRf Predetermined view angle value θRn Predetermined view angle value

Claims

1. An image display method by a computer having an arithmetic unit and an input / output unit, wherein the arithmetic unit generates a display image of a size adapted to the size of the display, which is to be displayed on a display in a vehicle interior having a mirror surface reflecting the rear of the vehicle, based on a cut-out image cut out from a rear image of the vehicle captured by a camera, and changes an angle-of-view value at which the cut-out image is cut out from the rear image to a predetermined angle-of-view value greater than the angle-of-view value at the position immediately before the shift position of the vehicle switches to the reverse position when the shift position of the vehicle switches from a position other than the reverse position to the reverse position.

2. The image display method according to claim 1, wherein when a target of interest exists in the rear image, the arithmetic unit sets the predetermined angle-of-view value to an angle-of-view value at which the display image including at least the target of interest is displayed on the display.

3. The image display method according to claim 2, wherein the target of interest includes at least one of a line drawn on a road surface, an obstacle on the road, and a moving object on the road.

4. The image display method according to claim 2 or 3, wherein the predetermined angle-of-view value when the distance between the target of interest and the vehicle is short is greater than the predetermined angle-of-view value when the distance is long.

5. An image display device using a computer having an arithmetic unit and an input / output unit, wherein the arithmetic unit includes an image generation unit that generates a display image of a size adapted to the size of the display, which is to be displayed on a display in a vehicle interior having a mirror surface reflecting the rear of the vehicle, based on a cut-out image cut out from a rear image of the vehicle captured by a camera, and an angle-of-view change unit that changes an angle-of-view value at which the cut-out image is cut out from the rear image to an angle-of-view value greater than the angle-of-view value at the position immediately before the shift position of the vehicle switches to the reverse position when the shift position of the vehicle switches from a position other than the reverse position to the reverse position.

Images