Display control method, display control device, and recording medium

Abstract

A display control device 100 according to the present invention acquires an image captured by a camera 2, executes correction processing including distortion correction processing for correcting distortion of the captured image, executes sharpening processing on a corrected image obtained by the correction processing, and outputs a display image obtained by the sharpening processing to a display 3. The correction amount of the correction processing is larger at an edge portion of the captured image than at a central portion of the captured image, and the sharpening processing is executed in a predetermined region including the edge portion.

Description

Display control method, display control device, and recording medium

[0001] The present invention relates to a display control method, a display control device, and a recording medium.

[0002] Conventionally, in-vehicle camera devices that display images of the outside of the vehicle captured by a fisheye camera on an in-vehicle monitor have been known. For example, the in-vehicle camera device described in Patent Document 1 converts the distorted image signal derived from the fisheye camera into a normal image signal, extracts an arbitrary area from the distortion-corrected image according to the vehicle's driving state, expands the extracted image, and displays it across the entire screen of the monitor.

[0003] Japanese Patent Publication No. 2021-64034

[0004] Image processing such as distortion correction has the problem that the corrected image becomes blurry because the amount of correction varies depending on the position in the image.

[0005] The problem that this invention aims to solve is to provide a display control method, a display control device, and a recording medium that sharpen images corrected by distortion correction processing.

[0006] The present invention solves the above problem by performing a correction process that includes a distortion correction process to correct the distortion of the captured image, performing a sharpening process on the corrected image corrected by the correction process, outputting the display image processed by the sharpening process to a display, and making the amount of correction in the correction process greater at the edges of the captured image than at the center of the captured image, and performing the sharpening process in a predetermined region including the edges.

[0007] According to the present invention, images corrected by distortion correction processing can be made sharper.

[0008] Figure 1 is a schematic diagram of the display system according to this embodiment. Figure 2 is a plan view showing a part of the instrument panel. Figure 3 is a flowchart of the control flow of the display control method executed by the display control device. Figure 4 is a diagram illustrating the image processing steps by the image processing unit. Figure 5 is a diagram illustrating the magnification map in the corrected image. Figure 6 is a conceptual diagram of the image displayed on the display.

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In this embodiment, the case in which the display control device according to the present invention is applied to a display system 1 mounted on a vehicle will be described as an example. The display system 1 of this embodiment displays images for understanding the situation around the vehicle on a display that is viewed by the vehicle's occupants.

[0010] Figure 1 is a block diagram of a display system 1 including a display control device 100 according to this embodiment. As shown in Figure 1, the display system 1 of this embodiment includes a camera 2, a display 3, and a display control device 100. The camera 2, the display 3, and the display control device 100 are connected by an in-vehicle communication network such as CAN or LIN, and can exchange information with each other.

[0011] The camera 2 in this embodiment is installed at a predetermined position on the vehicle. The number of cameras 2 and their installation positions are not particularly limited, but it is preferable that the number and installation positions be necessary to image the area around the vehicle. The camera 2 mounted on the vehicle images the vehicle and / or the area around the vehicle and sends the captured images to the controller 10. The captured images in this embodiment include images of a part of the vehicle and the area around the vehicle. The data of the captured images is used for generating images of the vehicle or the area around the vehicle, and for detecting objects from the generated images. The camera 2 includes at least a front camera. The front camera is installed on the bumper of the vehicle and images the area in front of the vehicle. The front camera has a wide-angle lens and captures a wide area along the width direction of the vehicle. For example, when a vehicle enters a T-junction with poor visibility, it is difficult for the driver to grasp the condition of the road ahead. In such cases, the image captured by the front camera is displayed on the display 3, and the driver can grasp the condition of the road ahead from the display screen of the display 3. Furthermore, cameras with wide-angle lenses may be used not only as front cameras but also as rear cameras. The rear camera is installed around the rear window of the vehicle and captures the area behind the vehicle.

[0012] Display 3 is an in-vehicle display included in an IVI (in-vehicle infotainment) system, and includes displays 31 and 32. Displays 31 and 32 are provided on the vehicle's instrument panel. Figure 2 is a plan view showing a part of the instrument panel. As shown in Figure 2, display 31 is provided in the center of the instrument panel. Display 32 is provided on the instrument panel in front of the driver. Display 31 displays, for example, maps in the navigation system, menu screens (such as application selection screens) and content videos for the entertainment system. Display 32 displays, for example, meters, a range indicator showing the current range of the shift position, icons indicating the seat belt status, vehicle images, images of the vehicle's surroundings, predicted route, driving route, etc. Display 31 and / or display 32 also display display images generated by the controller 10.

[0013] Displays 31 and 32 have horizontally elongated display screens along the vehicle width direction (left-right direction in the paper of Figure 2). As shown in Figure 2, displays 31 and 32 are arranged side by side along the vehicle width direction, with their display screens close together. Note that the vehicle does not need to have both displays 31 and 32; it may have only one of the displays.

[0014] The display control device 100 comprises a controller 10 and a recording medium 19. The controller 10 has functions for acquiring various data such as captured images and image processing functions. The controller 10 has an image acquisition unit 11 and an image processing unit 12 as functional blocks. The controller 10 stores programs for realizing various functions in the recording medium 19, and executes the processing flow indicated by the program by having the processor execute the control program. The recording medium 19 that records the control program may be stored as memory within the controller 10. The location of the recording medium 19 may be inside or outside the controller 10. The recording medium 19 records a display control program, which is a program that causes the controller 10 to execute processing including steps S1 to S6 described later.

[0015] The image acquisition unit 11 acquires the captured image taken by the camera 2.

[0016] The image processing unit 12 performs image processing on the captured image in order to display an image of the vehicle's surroundings and / or an overhead view image of the vehicle on the displays 31 and 32. The overhead view image of the vehicle is an image of the vehicle and its surroundings as seen from a virtual viewpoint, with a predetermined position outside the vehicle as the virtual viewpoint. For example, when parking a vehicle, such as in a garage or parallel parking, an overhead view image is an image of the vehicle viewed from directly above, and the driver can check the relationship between their vehicle and the parking position, as well as obstacles around the vehicle, by looking at the overhead view image displayed on the display 3 (this corresponds to a so-called around-view system). The image processing unit 12 generates a display image to be displayed on the displays 31 and 32 in a display mode specified by the user or system, and outputs the display image to the displays 31 and 32. In the following description, displays 31 and 32 will be collectively referred to as display 3.

[0017] There are multiple display modes, such as a front view or front wide view showing the front of the vehicle, a rear view or rear wide view showing the rear of the vehicle, a top view showing an overhead view of the vehicle, a 3D view showing the vehicle or its surroundings from a virtual viewpoint, and a skeleton view. In the front view or front wide view, an image is displayed showing the exterior of the vehicle, from in front of the vehicle's grille. In the rear view or rear wide view, an image is displayed showing the exterior of the vehicle, from behind the vehicle's tailgate. In the top view (around view), a position directly above the vehicle is used as a virtual viewpoint, and an overhead view image is displayed as if the vehicle were leaning over if there are tall three-dimensional objects such as walls around the vehicle. In the top view, a two-dimensional image is displayed, and the vehicle image is represented as a plan view of the vehicle. A virtual viewpoint is a perspective from which an object is viewed from outside the object, simulating its appearance. The virtual viewpoint is movable. For example, with a display image (3D image) of a vehicle viewed from the virtual viewpoint displayed on display 3, the 3D vehicle image can be rotated by rotating the virtual viewpoint around the vehicle along a plane. In addition, display 3 may display the area around the vehicle in other display modes, such as skeleton view, front wide view, and rear wide view. In skeleton view, in addition to the current captured image (real image) representing the area around the vehicle, a virtual image representing the underside of the vehicle is displayed. Front wide view can display an image of the area in front of the vehicle that is the driver's blind spot, for example, when the vehicle is turning left or right at an intersection with poor visibility, such as a T-junction, using an image from a front camera with a wide-angle lens.

[0018] The display mode can be specified by the user or changed as appropriate by the vehicle system according to the vehicle's driving conditions. For example, when the gear is shifted to reverse in order to park the vehicle, the display mode may automatically switch to top view, rear view, or rear wide view. Alternatively, the display mode may be switched by the driver operating a switch or similar device while the vehicle is in motion.

[0019] The image processing unit 12 can display images in multiple display modes on the display 31 on multiple screens. The image processing unit 12 may display images in different display modes on the display 31 and the display 32, or it may display the same image in the same display mode on both the display 31 and the display 32. The image processing unit 12 may also treat both the display 31 and the display 32 as a single display screen and display images in multiple display modes, or it may display an image in a single display mode. In the following description, the image processing performed by the image processing unit 12 when displaying an image on the display 31 will be described, but when displaying on the display 32, the image processing for the display 31 may be applied to the image display on the display 32.

[0020] The image processing unit 12 generates a display image according to the display mode based on a plurality of captured images acquired by the image acquisition unit 11, and outputs the display image to the display 31. When displaying an image in front wide view or rear wide view, the image processing unit 12 generates a display image showing the front or rear of the vehicle based on the front camera or rear camera included in the camera 2.

[0021] When displaying an image in top view, the image processing unit 12 generates an overhead view image of the vehicle based on multiple captured images taken by the camera 2. The virtual viewpoint of the overhead view image is set at a predetermined position, vertically separated from the vehicle's roof, and is located outside the vehicle. For example, the image processing unit 12 performs viewpoint transformation processing on images captured by multiple cameras 2 that capture images around the vehicle, generating viewpoint-transformed images that appear to be viewed from the virtual viewpoint, showing the front, rear, and left and right regions of the vehicle. That is, the image processing unit 12 transforms captured images corresponding to the view from the camera's position into viewpoint-transformed images viewed from the virtual viewpoint. The viewpoint transformation from the camera's position to the virtual viewpoint is performed by calculation processing using viewpoint parameters, which are stored in the recording medium 19. The image processing unit 12 then applies each image after the viewpoint transformation processing to the surrounding area of ​​the vehicle. The image processing unit 12 performs blending processing on the overlapping portions of each captured image from the camera 2. Blending processing is a process to maintain continuity between two overlapping images, and methods known at the time of filing this application can be applied. The resulting blended image becomes the surrounding image showing the area around the vehicle. The image processing unit 12 then composites the vehicle image located in the center of the screen with the surrounding image. The resulting composite image becomes an overhead view image, and the image processing unit 12 outputs this overhead view image to the display 31 as the display image.

[0022] When displaying an image in 3D view, the image processing unit 12 generates a display image of the vehicle as seen from a virtual viewpoint, based on multiple captured images taken by the camera 2. First, the image processing unit 12 sets the virtual viewpoint according to user specifications or driving conditions. The position and direction of the line of sight of the virtual viewpoint can be arbitrarily set based on commands from the user. The user can specify the position and direction of the line of sight of the virtual viewpoint, for example, by touching the display 31. The image processing unit 12 can also set the virtual viewpoint according to driving conditions. For example, when moving forward, the virtual viewpoint can be set behind the vehicle to view the image in front of the vehicle, and when reversing, the virtual viewpoint can be set in front of the vehicle to view the image behind the vehicle. Specifically, the image processing unit 12 sets the position of the virtual viewpoint when it receives shift information indicating that the shift position has entered reverse. Furthermore, the image processing unit 12 can also set the virtual viewpoint according to the vehicle speed. For example, depending on the vehicle speed input from the vehicle speed sensor 50, the virtual viewpoint may be set to move away from the vehicle when the vehicle speed is above a predetermined value, and to move closer to the vehicle when the vehicle speed is below a predetermined value.

[0023] Furthermore, the image processing methods for the display images shown in the front / rear view, top view, and 3D view are not limited to those described above; other image processing methods may be used. Also, for the image processing methods for the display images shown in other display modes, such as the skeleton view, methods known at the time of filing may be used as appropriate.

[0024] Next, we will explain the display control by the display control device 100, along with a specific example of the display screen of the display 31. Figure 3 is a flowchart showing the control flow of the display control method executed by the display control device 100. Figure 4 is a diagram illustrating the image processing steps by the image processing unit 12. In the example in Figure 4, the vehicle is stopped in preparation for entering a T-junction, and the display 3 displays a front wide-view image. The display mode is set to front wide-view.

[0025] In step S1, the image acquisition unit 11 acquires an image of the area in front of the vehicle from the front camera. The front camera has a wide-angle lens, such as a fisheye lens, and the image captured by the front camera is distorted. The image shown in Figure 4(a) is the image captured by the front camera (camera-through image).

[0026] The image processing unit 12 performs a correction process to correct the captured image according to the control flow of steps S2 to S4 below. In step S2, the image processing unit 12 performs a distortion correction process to correct the distortion of the captured image. The correction process includes coordinate transformation processing using distortion correction parameters, interpolation calculation processing, etc. The image processing unit 12 generates an image in which the entire field of view in the horizontal direction has been corrected by the distortion correction process. The method of distortion correction is not particularly limited, and any known method may be applied. A characteristic of images captured by a front camera is that the amount of distortion is small in the center of the captured image, but the amount of distortion is large at the edges of the captured image (see Figure 4(a)). Therefore, the correction amount (distortion correction amount) used in the distortion correction process is larger at the edges of the captured image than at the center of the captured image. For example, in coordinate transformation processing using distortion correction parameters, the correction parameters are set so that the amount of movement of points before and after coordinate transformation is larger at the edges than at the center. The image shown in Figure 4(b) is a corrected image corrected by the distortion correction process. When distortion correction is applied to an captured image (camera-through image), the size (area) per unit pixel is enlarged, and the size per unit pixel after correction becomes larger than the size per unit pixel before correction. Another characteristic of distortion correction is that the magnification rate per unit pixel differs depending on the pixel's position in the image; pixels located at both ends of the corrected image in the left-right direction have a higher magnification rate, while pixels located in the center of the corrected image have a lower magnification rate.

[0027] Figure 5 is a diagram illustrating the magnification map in the corrected image. In the example in Figure 5, the magnification of the left half of the corrected image is shown. The magnification of the top and bottom edges of the corrected image is omitted because they are cropped later. Note that the magnification values ​​are just examples. Region A １This indicates the pixel range at the left edge of the corrected image, region A. ２ This indicates the pixel range of the central part of the corrected image. Region A １ Then the magnification becomes 3.7, and region A ２ The magnification is greater than (2.4). Also, the image quality deteriorates as the magnification increases, so region A ２ The image quality in area A is good, but １ Image quality deteriorates in this case. In other words, images captured with a camera that has a wide-angle lens become more distorted from the center outwards. At the edges of the captured image, the magnification increases to compensate for the large distortion, resulting in poor image quality.

[0028] In step S3, the image processing unit 12 crops the image after distortion correction by cutting off the top and bottom edges. The image processing unit 12 crops the image after distortion correction so that the aspect ratio of the cropped image corresponds to the aspect ratio of the displays 31 and 32. For example, if the aspect ratio of the corrected image after distortion correction is 4:3 and the aspect ratio of the display screen is 16:9, the image processing unit 12 cuts out the frame (P) shown in Figure 4(b) from the corrected image with an aspect ratio (4:3) to form an image with an aspect ratio (16:9) shown in Figure 4(c). If there are unnecessary parts around the image after distortion correction, the image processing unit 12 may cut off the top and bottom edges and the left and right edges of the image, respectively. When imaging is performed with a camera 2 equipped with a wide-angle lens, vignetting may appear around the edges of the captured image. Even after distortion correction processing is performed on an image containing vignetting, vignetting may remain around the edges of the image. The image processing unit 12 may cut off the edges around the image to remove vignetting.

[0029] In step S4, the image processing unit 12 enlarges the clipped image to fit the size of the display screen of display 3. Figure 6(a) shows the display screen of display 31, and Figure 6(b) shows the display screens of displays 31 and 32. As shown in Figure 6(a), when the image is not displayed on display 32 but is displayed on display 31, the image processing unit 12 enlarges the clipped image (the image after the correction process in step S3) so that it becomes an image for display on display 31. Note that if the aspect ratio of the clipped image and the aspect ratio of the display screen of display 31 are different, the magnification ratio of the vertical length and the magnification ratio of the horizontal length of the image may be different values. Also, as shown in Figure 6(b), when the image is displayed across the entirety of displays 31 and 32, the image processing unit 12 enlarges the clipped image (the image after the correction process in step S3) so that it becomes an image for display on displays 31 and 32. Note that when displaying the image on displays 31 and 32, the magnification ratio of the horizontal length of the image is approximately twice that when it is displayed only on display 31.

[0030] Here, we will explain the magnification ratio of the size (area) per unit pixel. As described above, in the image after distortion correction, the magnification ratio of the size per unit pixel differs depending on the position of the pixel in the image. The magnification ratio of pixels located at both ends in the left-right direction (hereinafter also referred to as "edge magnification ratio") is greater than the magnification ratio of pixels located in the center of the corrected image (hereinafter also referred to as "center magnification ratio"). The edge magnification ratio indicates the magnification ratio of the size per unit pixel at pixels located at both ends in the left-right direction of the image. The center magnification ratio indicates the magnification ratio of the size per unit pixel at pixels located in the center of the image. The magnification process in step S4 further widens the difference between the edge magnification ratio and the center magnification ratio, and the edge magnification ratio becomes even larger. In other words, comparing the edge magnification ratio and the center magnification ratio, the edge magnification ratio becomes larger than the center magnification ratio in the distortion correction process in step S2, and becomes even larger in the magnification process in step S4. Furthermore, the magnification ratio in the magnification process is greater when displaying the image on both displays 31 and 32 than when displaying the image on display 31. As the edge magnification ratio increases, the image quality at both ends in the left-right direction deteriorates.

[0031] In step S5, the image processing unit 12 performs sharpening processing on the corrected image corrected by the correction process. The corrected image is the image after the enlargement process in step S4. The image processing unit 12 performs sharpening processing using a technique that sharpens degraded images using deep learning. Specifically, a ground truth image (an image without degradation) without blurring, etc., is prepared in advance for each vehicle driving scene. A degraded image is created by applying blurring, noise, or motion blur, etc., to the original ground truth image. The image processing unit 12 is trained to learn what kind of image processing can remove degradation factors such as blurring by comparing the ground truth image and the degraded image. The sharpening processing model obtained from the training results is recorded on the recording medium 19. The image processing unit 12 performs image processing to remove blurring, etc., contained in the corrected image using the trained sharpening processing model.

[0032] Further, the image processing unit 12 performs sharpening processing on the images located at the ends of the corrected image. The sharpening processing is executed in units of a predetermined region. That is, the image processing unit 12 performs sharpening processing in a predetermined region including the ends of the corrected image. The ends of the corrected image are located at both ends in the left-right direction of the corrected image, and correspond to the regions Q １ , Q ２ shown in FIG. 6. Also, the predetermined region indicates a processing unit when performing sharpening processing, and is represented by a square patch. Although FIG. 6 shows the image after the sharpening processing, in the image before the sharpening processing, the sharpening processing is performed in units of the patch (B) shown in FIG. 6. The image processing unit 12 adjusts the position of the patch (B) to the positions at both ends of the corrected image, and performs sharpening processing on the images within the regions Q １ , Q ２ . Also, the width of the patch (the length in the left-right direction in FIG. 6) is longer than the length (the length in the left-right direction in FIG. 6) of the regions Q １ , Q ２ . On the other hand, the length of the patch in the height direction (the length in the up-down direction in FIG. 6) is shorter than the length in the height direction (the length in the up-down direction in FIG. 6) of the regions Q １ , Q ２ . The image processing unit 12 may move the patch up and down from the position shown in FIG. 6 and perform sharpening processing on the images within the patch. On the other hand, the image processing unit 12 does not perform sharpening processing on the images located at the center of the corrected image. As a result, the images located at both ends of the corrected image become sharp, and the resolution of the images at both ends increases. That is, the sharpening processing includes a step of processing the corrected image so that the resolution of the images within the predetermined region (patch B) approaches the resolution of the images outside the predetermined region and not processed by the sharpening processing. As a result, the images located at both ends of the corrected image become sharp. Note that the resolution of the images within the predetermined region after the sharpening processing does not necessarily have to be greater than the resolution of the images not subjected to the sharpening processing.

[0033] Furthermore, when displaying an image across the entirety of displays 31 and 32, the image magnification is greater compared to when displaying an image on only one of the displays 31 or 32. Therefore, the resolution of the image at both ends may be higher when displaying an image across the entirety of displays 31 and 32 than when displaying an image on only one of the displays 31 or 32.

[0034] In step S6, the image processing unit 12 outputs the display image processed by the sharpening process to the display 3. Specifically, the image processing unit 12 outputs a control signal including the sharpened display image to the display 3. The display 3 displays the image on the screen based on the received control signal. As a result, the image shown in Figure 6 is displayed on the display 3. Then, the image processing unit 12 terminates the control flow shown in Figure 3.

[0035] As described above, the display control method or display control device according to this embodiment acquires an image captured by the camera 2, performs a correction process that includes a distortion correction process to correct the distortion of the captured image, performs a sharpening process on the corrected image corrected by the correction process, and controls the display 3 so that the display image processed by the sharpening process is displayed. Furthermore, the amount of correction in the correction process is greater at the edges of the captured image than at the center of the captured image, and the sharpening process is performed in a predetermined area including the edges. When distortion correction is performed on an image captured by a camera with a wide-angle lens, the amount of correction is large in the area at the edges of the captured image, making the blurriness noticeable. In this embodiment, sharpening is performed in the area where the blurriness is noticeable, making the blurriness less noticeable in the central area. In addition, the amount of sharpening can be reduced by performing sharpening on a part of the image rather than the entire captured image. As a result, the image corrected by the distortion correction process can be made sharper.

[0036] Also, in the display control method or display control device according to the present embodiment, a predetermined area (patch) is located at both ends in the left-right direction of the corrected image. Since a front wide view or a rear wide view displays a wide-angle image, the blurriness is more likely to be noticeable than in the front view or the rear view. Furthermore, blurriness is likely to occur at both ends of the image. In the present embodiment, a predetermined area is arranged at both ends in the left-right direction of the image, and sharpness processing is performed on the images included in both end portions. On the other hand, it is not necessary to arrange a predetermined area at the upper and lower ends of the captured image. For example, in a T-junction view (front wide view at a T-junction), since the captured image is taken at an angle such that the road runs along the left-right direction, the driver can check the situation of the intersection with a clear image.

[0037] Also, in the display control method or display control device according to the present embodiment, the shape of the predetermined area (patch) is square. Since the width and height of the image within the predetermined area (patch) are the same length, many image processing algorithms are easier to handle. For example, when performing geometric transformations such as rotation or scaling of an image, there is no need to worry about the aspect ratio, and the calculation can be simplified. Thereby, simplification of the algorithm can be achieved. Also, in machine learning or deep learning as the technology used in the sharpness processing, it is important that the size of the input data is unified. By using a square image as in the present embodiment, images of different sizes and shapes can be treated as data of the same dimension, and it becomes possible to maintain the consistency of the entire dataset.

[0038] Also, in the display control method or display control device according to the present embodiment, in the sharpness processing, the corrected image is processed so that the resolution of the image within the predetermined area (patch) approaches the resolution of the image outside the predetermined area and not processed by the sharpness processing. Thereby, image processing can be performed so that the sharpened portion is not conspicuous in the display image.

[0039] Also, in the display control method or display control device according to the present embodiment, the correction process includes an enlargement process step (corresponding to the control flow of step S4) of enlarging the image after correcting the distortion of the captured image. When the image after distortion correction is enlarged, the blurriness becomes more prominent. In the present embodiment, since the sharpness process is performed on the enlarged image, even if the image is enlarged to match the display of the display 3, a sharp display image can be obtained.

[0040] Also, in the present embodiment, the screen displayed on the display 3 includes a first display screen and a second display screen larger in size than the first display screen, and the enlargement ratio of the enlargement process is larger when displaying the display image on the second display screen than when displaying the display image on the first display screen. The first display screen corresponds to the display screen of either one of the displays 31 or 32, and the second display screen corresponds to the screen obtained by displaying the two screens of the displays 31 and 32 as one screen. In the present embodiment, since the sharpness process is performed on the corrected image, a sharp image can be displayed on the display 3 with a large screen size.

[0041] In the present embodiment, the sharpness process may include a step of processing the corrected image so that the resolution when the moving object is included in a predetermined area (patch) is larger than the resolution when the moving object is not included in the predetermined area. The image processing unit 12 detects a moving object between the control flow of step S1 and the control flow of step S5 shown in FIG. 3. The image processing unit 12 detects a moving object around the vehicle based on the captured image of the camera 2 or the corrected image after distortion correction. The moving object is another vehicle, a pedestrian, a bicycle, or the like. When a moving object is detected in the image, it is determined whether the position of the moving object is at both ends in the left-right direction of the corrected image. When the detected moving object is located at both ends in the left-right direction of the corrected image, the resolution is increased from the normal time, and the sharpness process is performed in a predetermined area (patch). The normal resolution is the set value set in the sharpness process when the moving object is not included in the predetermined area (patch). In the example of FIG. 6, the area (Q １ , Q ２A vehicle is located within the area (patch). In other words, the designated area (patch) on the left includes the moving object, while the designated area (patch) on the right does not. Therefore, sharpening processing is performed so that the resolution of the image in the designated area (patch) on the left is greater than the resolution of the image in the designated area (patch) on the right. As a result, the image of the moving object becomes sharper, making it easier for the driver to recognize the moving object from the display screen of display 3.

[0042] In this embodiment, the ECU control method executed by the controller 10 does not need to include all of the control flows in steps S1 to S6 described above; some steps may be omitted, and the processing order of each step may be changed. For example, the correction process only needs to include the distortion correction step in S2.

[0043] 1 Display system 2 Camera 3 Display 10 Controller 11 Image acquisition unit 12 Image processing unit 19 Recording medium 100 Display control device

Claims

1. A display control method performed by a display control device mounted on a vehicle, comprising: acquiring an image captured by a camera mounted on the vehicle; performing a correction process including a distortion correction process to correct distortion in the image; performing a sharpening process on the corrected image corrected by the correction process; outputting a display image processed by the sharpening process to a display, wherein the amount of correction in the correction process is greater at the edges of the image than at the center of the image, and the sharpening process is performed in a predetermined region including the edges.

2. The display control method according to claim 1, wherein the predetermined region is located at both ends in the left-right direction of the corrected image.

3. A display control method according to claim 1 or 2, wherein the shape of the predetermined area is a square.

4. A display control method according to any one of claims 1 to 3, wherein the sharpening process processes the corrected image such that the resolution of the image within the predetermined area approaches the resolution of the image outside the predetermined area that is not processed by the sharpening process.

5. A display control method according to any one of claims 1 to 4, wherein the sharpening process includes the step of processing the corrected image such that the resolution when a moving object is included in the predetermined area is greater than the resolution when a moving object is not included in the predetermined area.

6. A display control method according to any one of claims 1 to 4, wherein the correction process includes an enlargement process for enlarging the image after correcting the distortion of the captured image.

7. A display control method according to claim 6, wherein the screen displayed on the display includes a first display screen and a second display screen that is larger in size than the first display screen, and the magnification ratio of the magnification process is greater when the display image is displayed on the second display screen than when the display image is displayed on the first display screen.

8. A display control device comprising a controller for controlling a display mounted on a vehicle, wherein the controller acquires an image captured by a camera mounted on the vehicle, performs a correction process to correct distortion in the image, performs a sharpening process on the corrected image corrected by the correction process, outputs the display image processed by the sharpening process to the display, wherein the amount of correction in the correction process is greater at the edges of the image than at the center of the image, and the sharpening process is performed in a predetermined region including the edges.

9. A recording medium on which a display control program executed by a controller is recorded, wherein the display control program causes the controller to perform a process including: acquiring an image captured by a camera mounted on a vehicle; performing a correction process to correct distortion of the image; performing a sharpening process on the corrected image corrected by the correction process; and outputting the display image processed by the sharpening process to a display, wherein the amount of correction in the correction process is greater at the edges of the image than at the center of the image, and the sharpening process is performed in a predetermined region including the edges.

Images