Display control method, display control device, and recording medium

Abstract

The controller (20) performs the following steps: acquires a captured image (Im1) captured by a first camera (10) mounted in front of a vehicle (50), generates a corrected image (Im2) by performing a correction process that corrects distortion of the captured image (Im1), generates a display image (Im4) based on the corrected image (Im2), and displays the display image (Im4) on a display unit (15). The correction process corrects distortion so that distortion of an image at a position shifted along an image lateral direction from an image center in the image lateral direction is smaller than distortion of the image at the image center.

Description

Technical Field

[0001] This invention relates to a display control method, a display control device, and a recording medium. Background Technology

[0002] Patent Document 1 discloses an image processing apparatus. A first region is the area of ​​the ground predicted to be traversed by the vehicle when it travels a predetermined distance from an arbitrary position with a straight-ahead steering angle in the first travel phase. A second region is the area of ​​the ground predicted to be traversed by the vehicle when it travels a predetermined distance from an arbitrary position with a steering angle other than straight-ahead. A third region is a region of the second region that is not included in the first region. The image processing apparatus performs image transformation processing such that the distortion of the third region in the image output from the image processing apparatus during the second travel phase is less than the distortion of the third region in the image output from the image processing apparatus during the first travel phase.

[0003] Patent Document 1: Japanese Patent Application Publication No. 2018-148491 Summary of the Invention

[0004] However, in the method of Patent Document 1, it may be impossible to correctly identify objects located in areas that would be blind spots for passengers in actual space in the image.

[0005] The purpose of this invention is to provide a display control method, display control device, and recording medium capable of accurately identifying objects located in areas that are blind spots for passengers.

[0006] One aspect of the present invention relates to a display control method comprising the following steps: generating a corrected image by performing a correction process to correct distortion in a captured image; generating a display image based on the corrected image; and displaying the display image on a monitor. The correction process corrects distortion such that the distortion of the image at a position offset laterally from the image center is less than the distortion at the image center.

[0007] The effects of the invention

[0008] According to one aspect of the present invention, it is possible to accurately identify objects in actual space located in areas that are blind spots for passengers in the displayed image on the display screen. Attached Figure Description

[0009] Figure 1 This is a diagram schematically illustrating the structure of the display control device according to this embodiment.

[0010] Figure 2 It is a diagram showing the environment in which the vehicle is driving.

[0011] Figure 3 This is a flowchart illustrating the display control method.

[0012] Figure 4 It is a conceptual diagram representing a captured image.

[0013] Figure 5 It is a conceptual diagram representing the distortion of an image.

[0014] Figure 6 It is a conceptual diagram representing the distortion of an image.

[0015] Figure 7 This is a conceptual diagram representing a corrected image.

[0016] Figure 8 This is a conceptual diagram representing the cut-out image extracted from the corrected image.

[0017] Figure 9 It is a graph that represents the relationship between the cut-out image and the display image cut out from the cut-out image.

[0018] Figure 10 It is a conceptual diagram representing a display image magnified to the size of a monitor.

[0019] Figure 11 It is a conceptual diagram representing a display image magnified to the size of a monitor.

[0020] Figure 12 It is a schematic diagram showing the displayed image on the monitor. Detailed Implementation

[0021] Reference Figure 1 The display control device 1 according to this embodiment will be described. The display control device 1 is mounted on a vehicle. The display control device 1 includes a first camera 10, a second camera 11, a vehicle sensor 12, a GPS receiver 13, a map database 14, a display unit 15, and a controller 20.

[0022] like Figure 2As shown, the first camera 10 is mounted in front of the vehicle 50, specifically at the front end of the vehicle 50. The first camera 10 has a fisheye lens with a wide field of view, and its shooting range covers an area Ra of approximately 180° on the front side of the vehicle 50. For example, the first camera 10 is configured such that the optical axis center of the fisheye lens is parallel to the longitudinal direction of the vehicle 50, and the optical axis center of the fisheye lens is at a predetermined downward angle. The first camera 10 has a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The first camera 10 captures images of the scenery located in front of the vehicle 50 and outputs the captured images. The first camera 10 captures images at a predetermined shooting cycle.

[0023] The second camera 11 is positioned closer to the center of the vehicle than the first camera 10 in the longitudinal direction, for example, near the interior rearview mirror. The second camera 11 has a narrower field of view than the first camera 10, and its shooting range is smaller than area Ra, covering a certain area of ​​the front of the vehicle 50. For example, the shooting range of the second camera 11 corresponds to the field of vision of a passenger (typically the driver) facing forward. The second camera 11 has a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The second camera 11 captures images of the scenery in front and outputs the captured images. The second camera captures images at a predetermined shooting cycle. Furthermore, the display control device 1 does not necessarily need to have the second camera 11.

[0024] like Figure 1 As shown, vehicle sensor 12 is a sensor that detects the state of vehicle 50. In this embodiment, vehicle sensor 12 detects the state of the vehicle 50's direction indicator. GPS receiver 13 receives radio waves from multiple GPS satellites to calculate the position of vehicle 50.

[0025] Map database 14 is a storage device for storing map data. Map data represents map information related to roads and their associated structures. Road information includes, for example, road type, shape, and lane markings. Structure information includes the location and type of road surface displays such as stop lines at intersections, as well as the location, shape, and type of three-dimensional structures. Three-dimensional structures include traffic signs and traffic lights. Map database 14 can also obtain map data from external map data servers via cloud computing.

[0026] Display unit 15 includes one or more displays for displaying information to the driver. In this embodiment, display unit 15 includes a main display 16 and an instrument display 17. The main display 16 displays information from the navigation system, information related to audio equipment, etc. The instrument display 17 is a display that displays instruments such as the speedometer, tachometer, and fuel gauge, as well as various warning lights.

[0027] Considering driver visibility, the instrument cluster display 17 is positioned in front of the driver's seat on the dashboard. The main display 16 is positioned in the center of the dashboard, adjacent to the side of the instrument cluster display 17. The main display 16 and the instrument cluster display 17 can display information independently, or they can be treated as a single, continuous display. In the following description, unless otherwise specified, the term "display unit 15" will specifically refer to the main display 16.

[0028] The controller 20 is a general-purpose microcomputer with a CPU (central processing unit), memory, and input / output units. A computer program is installed on the microcomputer. By executing the computer program, the microcomputer functions as the multiple information processing circuits within the controller 20.

[0029] In this embodiment, an example is shown where the controller 20 has multiple information processing circuits implemented in software. Of course, multiple information processing circuits for performing the various information processing operations described below can also be constructed using dedicated hardware. Alternatively, multiple information processing circuits can be constructed using separate hardware.

[0030] The controller 20 displays a predetermined display image on the display unit 15 based on the captured image taken by the first camera 10. The controller 20 has a distortion correction unit 21, a correction corresponding image storage unit 22, an image cropping unit 23, an image adjustment unit 24, a recognition processing unit 25, and a display control unit 26 as multiple information processing circuits.

[0031] The distortion correction unit 21 acquires the captured image from the first camera 10. The distortion correction unit 21 generates a corrected image by performing a correction process to correct the distortion of the captured image. The image cropping unit 23 generates a cropped image with a size larger than the vertical dimension of the display unit 15. The image adjustment unit 24 generates a display image by cropping the cropped image according to the shape of the display unit 15. The recognition processing unit 25 recognizes objects by performing image processing on the display image. The display control unit 26 displays the display image on the display unit 15, or overlays an attention-grabbing image onto the display image.

[0032] like Figure 2 As shown, an example of a T-shaped intersection 105 where the first road 101 and the second road 102 intersect will be used to explain the display control method according to this embodiment. A vehicle 50 wants to enter the intersection 105 from the first road 101 connected to the intersection 105, and the front end of the vehicle 50 is near the intersection 105. Figure 2 In the example shown, the intersection road that intersects with the first road 101, namely the second road 102, intersects the first road 101 at a right angle. At such an intersection 105, there are sometimes walls or buildings on both sides of the first road 101, and the second road 102 is included in the area that becomes a blind spot for the driver. That is, it is difficult for the driver to visually observe the traffic conditions of the second road 102. Therefore, the display control device 1 according to this embodiment displays a display image based on the image captured by the first camera 10 and processed on the display unit 15, thereby enabling the driver to correctly identify objects on the second road 102.

[0033] In the following description, the area of ​​actual space relative to vehicle 50, defined by its lateral width, is referred to as the first area, Rac. Specifically, the first area, Rac, is the area located inside two lines extending in the longitudinal direction, separated by the lateral width of vehicle 50. The areas of actual space located further to the left and right of the first area are referred to as the second areas, Ral and Rar, respectively. In particular, the area located further to the left of the first area, Rac, is called the left second area, Ral, and the area located further to the right of the first area is called the right second area, Rar.

[0034] Reference Figure 3 Specifically, the display control method involved in this embodiment will be described as a control method for displaying a display image on the display unit 15. Figure 3 The flowchart shown is executed by controller 20.

[0035] The display control unit 26 obtains the current position of the vehicle 50 from the GPS receiver 13. Locations meeting certain conditions, such as intersections with poor visibility, are registered as registration locations in the map data of the map database 14. Based on the current position of the vehicle 50 and the map data, the display control unit 26 determines whether the vehicle 50 is within the vicinity of a registration location; specifically, whether the vehicle 50 is within a predetermined radius centered on the registration location (S10). If the vehicle 50 is within the vicinity of a registration location, step S11 and subsequent processing are performed.

[0036] The distortion correction unit 21 acquires the captured image Im1 output from the first camera 10 (S11). The position of each pixel in the captured image Im1 is determined by the coordinate values ​​of a two-dimensional coordinate system. Figure 4 In the captured image Im1 shown, the first virtual line La is virtually shown in Figure 2 In the second road 102 shown, the road edge 102a is located away from the vehicle 50. The second virtual line Lb is virtually shown in... Figure 2 In the second road 102 shown, the road edge 102b is located near the vehicle 50. As described above, Figure 2 The second road 102 intersects the first road 101 at a right angle; therefore, each road edge 102a, 102b corresponds to a straight line orthogonal to the longitudinal direction of the vehicle 50. The first virtual line La corresponds to a straight road edge 102a orthogonal to the longitudinal direction of the vehicle 50 at a position on the ground where the vehicle 50 is located, at a distance equal to the width (reference distance) of the second road 102 from the first camera 10. Similarly, the second virtual line Lb corresponds to a straight road edge 102b orthogonal to the longitudinal direction of the vehicle 50 at a position on the ground where the vehicle 50 is located, at a certain distance from the first camera 10.

[0037] like Figure 4 As shown, the captured image Im1 exhibits distortion caused by the fisheye lens. The distortion of the captured image Im1 increases with distance from the optical axis center of the fisheye lens. Figure 4 The image region Rir shown in the captured image Im1 is the area that is reflected. Figure 2 The second region Ral and Rar shown are, specifically, regions that project any position on the right side of the second region Rar. This position is located on the right side when viewed from vehicle 50, at a distance (e.g., 50m) from the first camera 10 on the second road 102. Furthermore, this position exists further outward than the field of view of the second camera 11, creating a blind spot for the driver. In contrast, the image region Ric is the region projected... Figure 2 The area shown as Region 1 Rac is, more specifically, the area that is reflected at any position on the front of a vehicle on Road 2 102.

[0038] Figure 5 and Figure 6 The distortion of the image in each image region Rir and Ric is illustrated schematically. Figure 5 The image distortion in the image region Rir is schematically shown. Figure 6 This schematically illustrates the image distortion within the image region Ric. In the absence of image distortion, Figure 5 and Figure 6 The "+" signs shown are arranged at equal intervals in both the horizontal and vertical directions. Figure 5 The image region shown has a Rir generation ratio Figure 6 The image region shown has large distortion (Ric).

[0039] The distortion correction unit 21 generates a corrected image Im2 by performing a correction process to correct the distortion of the captured image Im1 (S12). The distortion correction unit 21 generates the image Im2 by performing a coordinate transformation based on the captured image Im1. Figure 7 The corrected image Im2 is shown as such. The coordinate transformation is performed according to the correction correspondence map stored in the correction correspondence map storage unit 22. The correction correspondence map records the correspondence between the coordinate values ​​of each pixel of the captured image Im1 and the coordinate values ​​of the two-dimensional coordinate system defined for the corrected image Im2. By performing coordinate transformation on the coordinate values ​​of each pixel of the captured image Im1, the captured image Im1 can be rotated, moved horizontally, and enlarged or reduced.

[0040] The image display feature of this embodiment is to enable the driver to correctly identify the condition of the second road 102. The first region Rac, directly opposite the vehicle 50, is an area that the driver can visually observe. On the other hand, the second regions Ral and Rar contain areas that the driver cannot visually observe. In particular, the further away from the vehicle 50 a position is in the second regions Ral and Rar, the more difficult it is for the driver to visually observe it. Therefore, the distortion correction unit 21 corrects the distortion of the captured image Im1 so that objects existing in the second regions Ral and Rar can be correctly identified. Specifically, the distortion correction unit 21 corrects the distortion of the captured image Im1 so that the distortion of the image at a position laterally offset from the image center in the corrected image Im2 is less than the distortion of the image at the image center. That is, the distortion of the image related to the second regions Ral and Rar in the corrected image Im2 is less than the distortion of the image related to the first region Rac in the corrected image Im2.

[0041] Specifically, the distortion correction unit 21 corrects the distortion of the captured image Im1 so that the second road 102 is closer to horizontal as it moves from the image center of the corrected image Im2 towards the horizontal direction. At this time, the distortion correction unit 21 corrects the distortion of the captured image Im1 so that, using an object on the second road 102 located 50m away from the vehicle 50 as a reference, the aspect ratio of the object in the corrected image Im2 is most similar to the aspect ratio of the object in actual space. That is, the second regions Ral and Rar in the corrected image Im2 contain the actual size ratio region where the aspect ratio of the object in the corrected image Im2 is most similar to the aspect ratio of the object in actual space. The actual size ratio region is set in the area that becomes the driver's blind spot, that is, the area in the second regions Ral and Rar that is further outward than the viewing angle of the second camera 11.

[0042] exist Figure 4 and 5In the image region Rir shown, which is the area with a size ratio to the actual size, the distortion correction unit 21 corrects the distortion of the captured image Im1 so that the top and bottom edges of the rectangle are close to horizontal. Furthermore, the distortion correction unit 21 enlarges the vertical rectangle size and, in matching the vertical magnification, enlarges the horizontal rectangle size so that the visual perception is close to that of an object in actual space.

[0043] On the other hand, the distortion correction unit 21 corrects the distortion of the captured image Im1 so that the closer the image is to the horizontal center in the corrected image Im2, the more the image shrinks horizontally. Figure 4 and Figure 6 In the image region Ric shown, the distortion correction unit 21 corrects the distortion of the captured image Im1 so that the rectangle is compressed along the horizontal direction of the image.

[0044] Furthermore, the distortion correction unit 21 corrects the distortion so that in the corrected image Im2, the first virtual line La is located further up in the image as it extends from the image center towards the horizontal direction of the image. The first virtual line La virtually represents the road edge 102a on the far side of the second road 102.

[0045] The first virtual line La of the corrected image Im2 slopes downwards towards the image center of the corrected image Im2, preferably within the range of 1 to 8°, and more preferably within the range of 2 to 4°. The first virtual line La has a shape that curves downwards towards the image within a certain range along the horizontal direction of the corrected image Im2, including the image center (e.g., the first region Rac). Additionally, the first virtual line La extends linearly within a certain range located in the corrected image Im2 after being offset horizontally from the image center (e.g., the second region Ral, Rar).

[0046] However, if the distortion of the image over the aforementioned large-area real-size region is made too close to horizontal, that is, if the distortion of the image over the large-area real-size region is completely eliminated, the center of the first virtual line La will be unnaturally and significantly concave. Therefore, it is preferable that a small amount of distortion remains for the large-area real-size region. Therefore, in Figure 7 In the corrected image Im2, the right or left end of the first virtual line La is drawn as a straight line, but there is actually a region where the slope of the first virtual line La gradually becomes gentler the further it is towards the horizontal end of the image.

[0047] The aforementioned correction mapping is pre-set based on the concept of correction. The correction mapping is created such that, on the ground where vehicle 50 is located, at a position separated from the first camera 10 by a reference distance, for example, 6-8 meters (the width of a typical road), there exists a straight, virtual line orthogonal to the longitudinal direction of vehicle 50. This virtual line follows the same trajectory as the aforementioned first virtual line La. The distortion correction unit 21 corrects the captured image Im1 by performing coordinate transformation according to the correction mapping stored in the correction mapping storage unit 22. Furthermore, the distortion correction unit 21 generates the corrected captured image Im1, i.e., the corrected image Im2.

[0048] The image cropping unit 23 performs cropping processing on the corrected image Im2 (S13). For example... Figure 7 As shown, due to the coordinate transformation during the correction process, there exists a region (represented by the shaded area) in the corrected image Im2 where no image equivalent to the captured image Im1 exists. Therefore, as Figure 8 As shown, the image cropping unit 23 generates a cropped image Im3, which is a region without an image removed from the corrected image Im2, by cropping the region outside the corrected image Im2. This cropped image Im3 is cropped by a size larger than the vertical dimension of the display unit 15.

[0049] The image adjustment unit 24 crops the cut-out image Im3 according to the shape of the display unit 15, thereby generating the display image Im4 (S14: first image processing). The display unit 15 has two display modes: one where the display image Im4 is displayed only on the main display 16, and another where the display image Im4 is displayed on both the main display 16 and the instrument display 17. The display mode is selected, for example, through a user setting operation. Figure 9 As shown, the image adjustment unit 24 generates a display image Im4 based on a shape corresponding to the display mode.

[0050] In the above correction process, the distortion in the second region Ral and Rar of the corrected image Im2 is reduced first, while the distortion in the first region Rac of the corrected image Im2 is preserved. The correction is performed such that in the first region Rac of the corrected image Im2, the image is compressed laterally, and the virtual line La curves downwards. Therefore, the road width near the center of the cut-out image Im3 is stretched longitudinally, giving the driver an impression different from the actual depth of space. Therefore, the image adjustment unit 24 crops the image by removing the lower region within a certain distance from the bottom edge of the cut-out image Im3. As a result, the proportion of the second road 102 in the longitudinal direction near the center of the displayed image Im4 is reduced, giving the driver an impression closer to the actual depth of space.

[0051] The image adjustment unit 24, matching the size of the display unit 15, enlarges the displayed image Im4 (S15: Second image processing). For example... Figure 10 As shown, when the mode in which the display image Im4 is displayed only on the main display 16 is selected, the image adjustment unit 24 enlarges the display image Im4 to match the size of the main display 16. Additionally, as... Figure 11 As shown, when the mode of displaying the image Im4 on the main display 16 and the instrument display 17 is selected, the image adjustment unit 24 enlarges the image Im4 to match the size of the virtual display after integrating the main display 16 and the instrument display 17.

[0052] The recognition processing unit 25 performs image processing on the displayed image Im4 using well-known image processing techniques to identify objects present on the second road 102 (S16). From the identified objects, the recognition processing unit 25 identifies moving objects that are moving on the second road 102. At this time, the recognition processing unit 25 can also determine the type, speed, or direction of movement of moving objects such as pedestrians, bicycles, and cars.

[0053] The display control unit 26 performs display processing (S17) to display the display image Im4 on the display unit 15. Figure 12 An example is shown where the display image Im4 is displayed on the main display 16. This display image Im4 is a single image where the first region Rac and the second regions Ral and Rar in actual space are consecutive. In this embodiment, a single camera, namely the first camera 10, is used to generate this consecutive image.

[0054] As described above, in the correction process, the closer to the image center of the corrected image Im2, the more the image is compressed laterally. Therefore, in the display image Im4 cut from the corrected image Im2, the horizontal width corresponding to the first region Rac becomes smaller than the horizontal width corresponding to the second regions Ral and Rar in actual space. The horizontal width of the region corresponding to the second regions Ral and Rar in actual space refers to the sum of the horizontal widths corresponding to the left second region Ral and the right second region Rar.

[0055] Furthermore, when the recognition processing unit 25 recognizes a moving object, the display control unit 26 overlays an image that evokes attention to the moving object (hereinafter referred to as an "attention-evoking image") onto the display image Im4. Figure 12 In the example shown, the attention-grabbing image is a rectangular box Iwc that surrounds the moving object. Alternatively, the attention-grabbing image can also include an image representing the direction of movement of the moving object, such as an arrow image Iwa.

[0056] At this time, the display control unit 26 can also dynamically increase the size of the rectangular frame Iwc as the moving object approaches the vehicle 50. The display control unit 26 can also dynamically change the color or display mode of the rectangular frame Iwc and the arrow image Iwa in a way that the moving object is highlighted more as it gets closer to the vehicle 50. Changes in display mode may include, for example, changing from a lit display to a flashing display, or changing the transmittance. Furthermore, when the recognition processing unit 25 determines the type and speed of the moving object, the display control unit 26 can also, in a way that can recognize different types and speeds of moving objects, change the color or display mode of the attention-grabbing image accordingly.

[0057] Furthermore, the display control unit 26 can display attention-grabbing images for all moving objects on the second road 102, but it can also display attention-grabbing images only for moving objects that meet certain conditions. For example, the display control unit 26 can filter moving objects that meet certain conditions based on the state of the vehicle 50's turn indicator. For example, in a traffic environment where the vehicle 50 is traveling on the left, if the vehicle 50's turn indicator is moving in the direction of a left turn, the display control unit 26 will only display attention-grabbing images for moving objects that are located on the right side of the vehicle 50 and are moving toward the vehicle 50. On the other hand, if the vehicle 50's turn indicator is moving in the direction of a right turn, the display control unit 26 will display attention-grabbing images for moving objects that are located on both the right and left sides of the vehicle 50 and are moving toward the vehicle 50. In addition, if the display control unit 26 determines that a moving object exists within a pre-defined object range, such as within the driver's visual observation range, it can also prevent the display of attention-grabbing images for that moving object.

[0058] In addition, such as Figure 12 As shown, the display control unit 26 overlays the display menu Iwm, used for switching the content displayed on the display unit 15, onto the display image Im4. The display control unit 26 displays the display menu Iwm in a transparent state. This prevents the content of the display image Im4 from being covered or hidden by the display menu Iwm.

[0059] As described above, the displayed image Im4 is a cropped image of the lower region of the cut-out image Im3. Therefore, a portion of the intersection 105 (second road 102) in front of the vehicle 50 is not reflected in the displayed image Im4. Consequently, the driver cannot determine whether the front of the vehicle 50 has entered the intersection 105 based on the displayed image Im4. Therefore, the display control unit 26 can also perform image processing based on the cut-out image Im3 to determine whether the front of the vehicle 50 has entered the intersection 105. Furthermore, if the front of the vehicle 50 has entered the intersection 105, the display control unit 26 can display an image notifying the driver of entry into the intersection 105 on the display unit 15, or output voice or sound.

[0060] If the controller 20 determines that the display has ended, for example, that the intersection 105 has been passed, it terminates the series of processes (S18: YES). On the other hand, if the display has not ended, it returns to the process of step S11.

[0061] Thus, in the display control method of this embodiment, the controller 20 corrects the distortion of the captured image Im1 so that the distortion of the image at a position offset laterally from the image center in the corrected image Im2 is less than the distortion of the image at the image center. Consequently, the distortion of objects in the areas that become blind spots for the driver, specifically in the second regions Ral and Rar of actual space, is reduced in both the corrected image Im2 and the display image Im4 cropped from it. As a result, the driver can correctly identify objects in the areas that become blind spots for the driver in actual space.

[0062] In the display control method of this embodiment, the controller 20 corrects distortion so that in the corrected image Im2, the first virtual line La is located higher up in the image, further from the image center towards the horizontal direction. The first virtual line La extends linearly in a certain range within a region offset laterally from the image center of the corrected image Im2. The first virtual line La also has a downward-curving shape in a certain range along the horizontal direction of the corrected image Im2, including the image center. According to this method, in the corrected image Im2 and the display image Im4 cut from the corrected image Im2, an image effect can be created where a moving object approaching the vehicle 50 on the second road 102 is heading towards the vehicle 50, thus enabling the driver to correctly identify the moving object.

[0063] In the display control method of this embodiment, in the displayed image Im4, the horizontal width of the first region Rac in the actual space is smaller than the horizontal width of the second regions Ral and Rar in the actual space. Specifically, the controller 20 corrects the distortion of the captured image Im1 so that in the corrected image Im2, the closer to the image center in the horizontal direction, the smaller the image appears in the horizontal direction. According to this method, the images of the second regions Ral and Rar that should be of interest can be displayed at a large scale in the displayed image Im4. As a result, moving objects moving on the second road 102 can be correctly identified.

[0064] In the display control method of this embodiment, the second regions Ral and Rar in the corrected image Im2 contain the actual size ratio region where the aspect ratio of the object on the corrected image Im2 is most similar to that of the object in actual space. According to this method, in the corrected image Im2 and the display image Im4 cropped from the corrected image Im2, there is a region where the object on the image is displayed in a state where the aspect ratio of the object is most similar to that of the object in actual space. Therefore, the object is displayed in a natural form, and thus, the object can be correctly identified. In this embodiment, the actual size ratio region is set in the second regions Ral and Rar, which is located further outward than the viewing angle of the second camera 11. Therefore, objects existing in areas that are blind spots for the driver can be correctly identified.

[0065] Furthermore, in this embodiment, the actual size ratio region is set at a predetermined position in the corrected image Im2. However, the distortion correction unit 21 can also dynamically correct the correction correspondence map and dynamically change the position of the actual size ratio region. For example, the recognition processing unit 25 identifies intersecting roads extending in a direction intersecting the travel direction of the vehicle 50 through image processing. Furthermore, the distortion correction unit 21 can also set the actual size ratio region on the intersecting road in the second region Ral, Rar. Thus, moving objects on the intersecting road can be accurately identified.

[0066] In the display control method of this embodiment, the displayed image Im4 is a single image in which the first region Rac and the second regions Ral and Rar of the actual space are continuous. According to this method, an image of a large area Ra in front of the vehicle 50 can be displayed seamlessly.

[0067] In the display control method of this embodiment, the controller 20 generates a cutout image Im3 cut from the correction image Im2 by means of a size larger than the vertical dimension of the display unit 15. According to this method, the selection range for cutting out a display image Im4 for display on the display unit 15 can be ensured.

[0068] In the display control method of this embodiment, the controller 20 generates a display image Im4 by cropping the cut-out image Im3 in a manner that removes the lower region of the cut-out image Im3, corresponding to the shape of the display unit 15. According to this method, the intersecting roads reflected in the display image Im4 can be displayed with a natural sense of depth.

[0069] In the display control method of this embodiment, when the controller 20 detects a moving object within the shooting range of the first camera 10, it displays an attention-provoking image superimposed on the display image Im4 based on the state of the vehicle 50's direction indicator. According to this method, the driver can correctly identify the moving object that the driver should pay attention to.

[0070] In the display control method of this embodiment, the controller 20 identifies intersecting roads extending in a direction intersecting the travel direction of the vehicle 50 through image processing. The controller 20 determines whether the vehicle 50 has entered the intersecting road. Although the intersecting road near the front of the vehicle 50 is not displayed in the display image Im4, the aforementioned determination allows it to be determined that the front of the vehicle 50 has entered the intersecting road. This allows the driver to be alerted.

[0071] Furthermore, in the above embodiment, the first and second cameras 10 and 11 are positioned at the front of the vehicle 50. However, the first and second cameras 10 and 11 may also be positioned at the rear of the vehicle 50.

[0072] In this embodiment, the controller 20 displays the display image Im4 on the display unit 15 when the vehicle 50 is located around the registration location. However, the controller 20 may also display the display image Im4 on the display unit 15 when a display request is received from the driver's operation.

[0073] Additionally, a display control device 1, including a first camera 10 and a controller 20, is also included as part of this embodiment. The controller 20 displays a display image Im4 on the display unit 15 based on the image captured by the first camera 10. The controller 20 executes the display control method described above. Similarly, a program for executing the display control method described above and a computer-readable recording medium storing the program are also included in this embodiment. Based on the display control device 1, the program, and the recording medium, in the corrected image Im2, the distortion of objects located in the area that becomes the driver's blind spot, specifically in the second area Ral and Rar of actual space, is reduced. Therefore, objects located in the area that becomes the driver's blind spot in actual space can be correctly identified.

[0074] As described above, embodiments of the present invention have been presented, but it should not be construed that the discussion and drawings, which form part of the present invention, limit the present invention. Based on the present invention, those skilled in the art will be able to identify various alternative embodiments, examples, and techniques.

[0075] Explanation of the label

[0076] 1 Display control device

[0077] 10 First Camera

[0078] 11. Second Camera

[0079] 12 Vehicle Sensors

[0080] 13 GPS receivers

[0081] 14 Map Database

[0082] 15 display units

[0083] 16 Main Displays

[0084] 17. Instrument Display

[0085] 20 Controllers

Claims

1. A display control method, performed by a display control device that displays images on a display screen mounted in a vehicle. The display control method includes the following steps: Acquire images captured by a camera mounted at the front or rear of the vehicle. A corrected image is generated by performing a correction process that corrects the distortion of the captured image. Based on the corrected image, a display image is generated. The image is displayed on the monitor. The correction process corrects the distortion so that the distortion of the image at a position offset laterally from the image center is less than the distortion of the image at the image center.

2. The display control method according to claim 1, wherein, When considering a hypothetical straight line orthogonal to the vehicle's longitudinal direction, located at a reference distance from the camera on the ground where the vehicle is situated... The correction process corrects the distortion so that the virtual line is located on the upper side of the image, the further it is from the center of the image towards the horizontal direction of the image.

3. The display control method according to claim 2, wherein, The virtual line extends in a straight line within a certain range of regions located at a position offset laterally from the center of the image.

4. The display control method according to claim 2, wherein, The virtual line has a shape that curves downwards in a region encompassing the center of the image.

5. The display control method according to any one of claims 1 to 4, wherein, In the case where the area of ​​actual space relative to the vehicle, defined by the vehicle's lateral width, is designated as the first area, and the area of ​​actual space further to the right and left than the first area is designated as the second area, In the displayed image, the horizontal width of the image corresponding to the first region is smaller than the horizontal width of the image corresponding to the second region.

6. The display control method according to claim 5, wherein, The correction process corrects the distortion so that the image shrinks more horizontally as it gets closer to the image center.

7. The display control method according to any one of claims 5 or 6, wherein, In the corrected image, the second region contains the actual size ratio region in which the aspect ratio of the object in the corrected image is most similar to the aspect ratio of the object in actual space.

8. The display control method according to any one of claims 5 to 7, wherein, The camera's field of view includes the first region and the second region. The displayed image is a single image where the first region and the second region are consecutive.

9. The display control method according to any one of claims 1 to 8, wherein, It also includes the step of generating a cut-out image, which is cut from the corrected image by a size larger than the vertical dimension of the display.

10. The display control method according to claim 9, wherein, It also includes the following steps: cropping the cut-out image in accordance with the shape of the display by removing the lower region of the cut-out image, thereby generating the display image.

11. The display control method according to any one of claims 1 to 10, wherein, It also includes the following steps: when a moving object is detected within the shooting range of the camera, an attention-grabbing image is overlaid on the display image based on the state of the vehicle's direction indicator.

12. The display control method according to claim 7, wherein, It also includes the following steps: identifying intersecting roads extending in directions that intersect the vehicle's direction of travel through image processing. The actual size ratio area is set on the intersection road in the second area.

13. The display control method according to claim 7, wherein, When the camera is set as the first camera... The vehicle also has a second camera, which is positioned further towards the center of the vehicle than the first camera in the longitudinal direction, and has a smaller field of view than the first camera. The actual size ratio area is set in the second region, in a region that is further outward than the field of view of the second camera.

14. The display control method according to any one of claims 1 to 13, wherein, It also includes the following steps: identifying intersecting roads extending in directions that intersect the vehicle's direction of travel through image processing. Determine whether the vehicle has entered the intersection.

15. A display control device, comprising: A camera, mounted at the front or rear of a vehicle; and The controller displays images captured by the camera on a display mounted on the vehicle. The controller performs the following steps: A corrected image is generated by performing a correction process that corrects the distortion of the captured image. Based on the corrected image, a display image is generated. The image is displayed on the monitor. The correction process corrects the distortion so that the distortion of the image at a position offset laterally from the image center is less than the distortion of the image at the image center.

16. A computer-readable recording medium storing a program that causes a computer to perform processing, the processing comprising the following steps: Acquire images captured by a camera mounted at the front or rear of the vehicle. A corrected image is generated by performing a correction process that corrects the distortion of the captured image. Based on the corrected image, a display image is generated. The image is displayed on a monitor mounted on the vehicle. The correction process corrects the distortion so that the distortion of the image at a position offset laterally from the image center is less than the distortion of the image at the image center.

Images