Display control device and display control program

Abstract

To provide a technique capable of satisfactorily suppressing deviations in display position from the actual position and discomfort felt by a user caused by the deviations when an image of a portion corresponding to a current portion directly below the bottom of own vehicle.SOLUTION: A history image acquisition unit (532) acquires a history image storage from a history image storage unit (54) that stores a captured image including a first area image corresponding to a first area (R1) to which a host vehicle is traveling and a second area image corresponding to a second area (R2) outside the first area in chronological order as a history image. An image matching processing unit (533) collates a current image corresponding to a current captured image of the surrounding of the own vehicle and the second region image in the history image. A display processing unit (534) displays the first area image included in the history image that is identified as matching the current image through the collation as an underfloor image that is an image of a portion directly below the bottom of the own vehicle on a vehicle display device (2).SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a display control device and a display control program for controlling the display of an in-vehicle display device. [Background technology]

[0002] Patent Document 1 discloses a vehicle periphery display device and a vehicle periphery display program that display an image in which an image of the host vehicle and a captured image of the surroundings of the host vehicle are combined. The vehicle periphery display device and the vehicle periphery display program utilize the movement of the vehicle to generate and display an image of the area directly below the current bottom of the host vehicle from past images captured by an on-board camera that captures the surroundings of the vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2004-336613 Summary of the Invention [Problem to be solved by the invention]

[0004] In this type of technology, the position or movement amount of the vehicle is estimated using driving state information acquired from on-board sensors such as a wheel speed sensor in order to generate and display an image of the area directly below the bottom of the vehicle. At this time, estimation errors occur depending on sensor errors, slippage between the wheels and the ground, and the like. Accumulating such estimation errors over time can cause the displayed image of the area directly below the bottom of the vehicle to deviate from reality, which can cause a user to feel uncomfortable or misidentify the position of a target object such as a wheel chock. The present invention has been made in consideration of the above-mentioned circumstances. That is, the present invention provides a technology that can effectively suppress deviations from reality and the resulting discomfort felt by the user when displaying an image of the area directly below the bottom of the vehicle, for example. [Means for solving the problem]

[0005] The display control device (5) for controlling the display of the in-vehicle display device (2) according to claim 1 comprises: a historical image acquisition unit (532) that acquires the historical images from a historical image storage unit (54) that stores, in chronological order, photographed images including a first area image corresponding to a first area (R1) around the host vehicle on which the in-vehicle display device is mounted and a second area image corresponding to a second area (R2) outside the first area in a vehicle width direction as historical images; an image matching processing unit (533) that matches a current image corresponding to a currently captured image of the surroundings of the host vehicle with the second area image in the history image acquired by the history image acquisition unit; a display processing unit (534) that displays, on the in-vehicle display device, the first area image included in the history image that has been identified as matching the current image through matching in the image matching processing unit, as an underfloor image that is an image of a portion directly below the bottom of the vehicle; Equipped with picture, The image matching processing unit matches the current image with the second region image in the history image using template matching. . Claim to 5 The display control program is a program executed by a display control device (5) that controls the display of an in-vehicle display device (2), The process executed by the display control device is a historical image acquisition process for acquiring the historical images from a historical image storage unit (54) that stores, in chronological order, photographed images including a first area image corresponding to a first area (R1) around the host vehicle equipped with the on-vehicle display device and a second area image corresponding to a second area (R2) outside the first area in the vehicle width direction as historical images; an image matching process for matching a current image corresponding to a currently captured image of the surroundings of the host vehicle with the second area image in the history image acquired by the history image acquisition process; a display process for displaying, on the in-vehicle display device, the first area image included in the history image identified as matching the current image through the comparison in the image comparison process, as an underfloor image that is an image of a portion directly below the bottom of the vehicle; of Including, The image matching process is a process of matching the current image with the second region image in the history image using template matching. .

[0006] In addition, in each section of the application documents, each element may be assigned a reference symbol in parentheses. In this case, the reference symbol merely indicates an example of the correspondence between the element and the specific configuration described in the embodiment described below. Therefore, the present invention is not limited in any way by the description of the reference symbol. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a block diagram showing a schematic configuration of an in-vehicle system including a display control device according to an embodiment of the present invention; [Figure 2] 2 is a plan view showing a state in which a host vehicle equipped with the in-vehicle system shown in FIG. 1 is traveling forward. [Figure 3A] 2 is a schematic diagram showing an example of an underfloor image displayed by the vehicle-mounted display device shown in FIG. 1. FIG. [Figure 3B] FIG. 10 is a schematic diagram showing an example of a display of an underfloor image in a comparative example. [Figure 4] 2 is a flowchart showing a specific example of a display control program executed by the display control device shown in FIG. DETAILED DESCRIPTION OF THE INVENTION

[0008] (Embodiment) Hereinafter, embodiments of the present invention will be described with reference to the drawings. It should be noted that if a description of various modifications applicable to one embodiment is inserted in the middle of a series of descriptions relating to the embodiment, it may hinder understanding of the embodiment. Therefore, the modifications will be described together after the series of descriptions relating to the embodiment, rather than in the middle of the series of descriptions relating to the embodiment.

[0009] (composition) Fig. 1 shows a schematic functional block configuration of an in-vehicle system 1 mounted on a host vehicle C shown in Fig. 2. As shown in Fig. 2, the host vehicle C is a so-called four-wheeled standard automobile, and has a box-shaped body CB, with one wheel CT provided at each of the four corners of the body CB. Below, the schematic configuration of the in-vehicle system 1 will be described with reference to Figs. 1 and 2. As shown in Fig. 1, the in-vehicle system 1 includes an in-vehicle display device 2, a peripheral camera 3, an in-vehicle communication bus 4, and a display control device 5.

[0010] The in-vehicle display device 2 is a flat display device or a curved display device (i.e., for example, a CID device) provided on the dashboard of the vehicle C, and is configured with a surface display device such as a liquid crystal display device or an organic EL display device. CID stands for center information display. EL stands for electroluminescence. In this embodiment, the in-vehicle display device 2 is configured to be able to switch between displaying an image corresponding to an image of the surroundings of the vehicle C captured by the surrounding camera 3 and displaying other information, in response to an operation by an occupant of the vehicle C.

[0011] The peripheral camera 3 is attached to the host vehicle C so as to capture images of the periphery of the host vehicle C. Specifically, in this embodiment, the in-vehicle system 1 is equipped with a front camera 31, a rear camera 32, a left-side camera 33, and a right-side camera 34 as the peripheral cameras 3. The front camera 31 is provided so as to capture images of the front of the host vehicle C (i.e., the first forward region R1 shown in FIG. 2) and the front-side of the host vehicle C (i.e., the second forward region R2 shown in FIG. 2). As shown in FIG. 2, the first forward region R1, which corresponds to the "first region" of the present invention, is the region ahead of the host vehicle C while it is traveling forward, and is typically a region of a predetermined width on an extension of the vehicle center line CL of the host vehicle C. The "predetermined width" is, for example, αW, where W is the vehicle width dimension of the host vehicle C. α=0.9 to 1.5, and typically α=1. The second forward region R2, which corresponds to the "second region" of the present invention, is a region further outward in the vehicle width direction than the first forward region R1. The rear camera 32 is provided to capture images behind and to the rear sides of the host vehicle C. The left side camera 33 is provided to capture images on the left side of the host vehicle C. The right side camera 34 is provided to capture images on the right side of the host vehicle C.

[0012] (Display control device) The display control device 5 is connected to the peripheral camera 3 so as to be able to communicate signals with the peripheral camera 3, so as to receive, from the peripheral camera 3, image signals corresponding to images captured by the peripheral camera 3. The display control device 5 is also connected to the in-vehicle communication bus 4 so as to be able to communicate information or signals with the in-vehicle communication bus 4, so as to receive, via the bus 4, information or signals corresponding to the driving state and driving environment of the host vehicle C. The "driving state" includes, for example, quantities related to the driving operation state of the host vehicle C by an occupant or a driving automation system (i.e., an advanced driving assistance system or an autonomous driving system), such as accelerator opening, braking amount, shift position, and steering angle. The "driving state" also includes, for example, physical quantities related to the driving behavior of the host vehicle C, such as vehicle speed, longitudinal acceleration, lateral acceleration, and angular velocity. The "driving environment" includes, for example, physical quantities related to the natural environment, such as outside temperature and rainfall. The "driving environment" also includes, for example, the presence state of moving and stationary targets within a predetermined detection range around the host vehicle C. That is, the display control device 5 is connected via the in-vehicle communication bus 4 to well-known sensors necessary for vehicle driving control, such as an accelerator opening sensor, a steering angle sensor, a wheel speed sensor, an acceleration sensor, an angular velocity sensor, an outside air temperature sensor, a raindrop sensor, an ultrasonic sensor, and a radar sensor.

[0013] The display control device 5 has a configuration as a so-called HCU that controls the display of the in-vehicle display device 2. HCU is an abbreviation for HMI Control Unit. HMI is an abbreviation for Human Machine Interface. In this embodiment, the display control device 5 has a configuration as a so-called in-vehicle microcomputer that includes a CPU, ROM, non-volatile rewritable memory, RAM, an input / output interface, etc. CPU is an abbreviation for Central Processing Unit and is also called a "processor." ROM is an abbreviation for Read Only Memory. Non-volatile rewritable memory is, for example, a hard disk, EEPROM, flash ROM, etc. EEPROM is an abbreviation for Electronically Erasable and Programmable ROM. RAM is an abbreviation for Random Access Memory. ROM, non-volatile rewritable memory, and RAM correspond to so-called non-transient physical storage media and are sometimes simply called "memory." That is, the display control device 5 is configured so that the CPU reads and executes a computer program stored in advance in the ROM and / or nonvolatile rewritable memory, thereby controlling the display of the in-vehicle display device 2. In other words, the display control device 5 includes at least one processor and at least one memory, and is configured so that the processor reads and executes a computer program from the memory, thereby performing predetermined display control.

[0014] Specifically, the display control device 5 includes an image signal input unit 51, a communication interface 52, an image processing unit 53, a history image storage unit 54, and an image signal output unit 55. The image signal input unit 51 is an input interface for image signals from the peripheral cameras 3, and is connected to the peripheral cameras 3 so as to be able to communicate signals with them. The communication interface 52 is an input interface for information or signals corresponding to the driving state and driving environment of the host vehicle C, and is connected to the in-vehicle communication bus 4 so as to be able to communicate information or signals with them.

[0015] The image processing unit 53 performs various image processing on the image signals or image information acquired from the peripheral camera 3 and generates display image signals for displaying images on the in-vehicle display device 2. Specifically, in this embodiment, the image processing unit 53 generates a historical image for image display corresponding to an image captured by the front camera 31 or the rear camera 32 at a time prior to the current time, and a current image for image display corresponding to an image captured by the left side camera 33 or the right side camera 34 at the current time. In this manner, the historical image corresponds to an image captured in the past, i.e., before the current time, by the front camera 31 or the rear camera 32, and the current image corresponds to an image captured currently by the left side camera 33 or the right side camera 34. The historical image includes a first area image corresponding to a first area ahead of the host vehicle C and a second area image corresponding to a second area outward of the first area in the vehicle width direction. Specifically, the first area image includes an image of the road surface or ground ahead of the host vehicle C. The "road surface" refers to the surface facing the bottom of the host vehicle C when the host vehicle C is traveling on a paved road (i.e., during on-road driving). In contrast, the "ground surface" refers to the surface opposite the bottom of the vehicle C when the vehicle C is traveling on a road other than a paved road (i.e., during off-road driving). "Other than a paved road" includes, for example, unpaved roads such as gravel roads, as well as grass, sand, wasteland, rivers, riverbeds, mountains, etc. The second region image includes an image of the road surface or ground surface that is further outward in the vehicle width direction than the first region image. The configuration of the image processing unit 53 will be described in detail later.

[0016] The historical image storage unit 54 stores captured images including the first area image and the second area image in chronological order as historical images. Specifically, the historical image storage unit 54 is provided as a part of the total storage capacity of the nonvolatile rewritable memory and / or RAM and as a CPU function for storing, erasing, and rewriting information therein. In this embodiment, the historical image storage unit 54 stores, in chronological order, historical images corresponding to images captured by at least the front camera 31 of the peripheral cameras 3 during forward driving, and stores, in chronological order, historical images corresponding to images captured by at least the rear camera 32 during reverse driving. The image signal output unit 55 is an output interface for display image signals for displaying images on the in-vehicle display device 2 and is connected to the in-vehicle display device 2 so as to be capable of signal communication.

[0017] (Image processing unit) The image processing unit 53 has a viewpoint conversion processing unit 531, a history image acquisition unit 532, an image matching processing unit 533, and a display processing unit 534 as functional components or units realized on a microcomputer.

[0018] The viewpoint conversion processing unit 531 converts the image captured by the peripheral camera 3 into a bird's-eye view image. Specifically, this conversion to a bird's-eye view image is an inverse conversion of the conversion in which position data of an image on the ground is projected onto a screen plane T at a focal distance f from a camera position R at a height H above the ground. In other words, the conversion to a bird's-eye view image is a coordinate conversion of the image captured by the peripheral camera 3 into data in a ground plane coordinate system projected from the peripheral camera 3 as the viewpoint. Such viewpoint conversion techniques are disclosed, for example, in Japanese Patent Application Laid-Open Nos. 14-087160 and 10-211849. The historical images are images after viewpoint conversion by the viewpoint conversion processing unit 531 and are stored in chronological order in the historical image storage unit 54 so as to be readable at any time.

[0019] The historical image acquisition unit 532 is configured to acquire historical images from the historical image storage unit 54. That is, the historical image acquisition unit 532 is configured to read out historical images to be used as targets for matching by the image matching processing unit 533 from the historical image storage unit 54. In this embodiment, the historical image acquisition unit 532 is configured to read out historical images within a predetermined traveling time range or a predetermined traveling distance range prior to the current time from the historical image storage unit 54. Specifically, when traveling forward, the historical image acquisition unit 532 extracts historical images within a range of X±X0 as targets for matching by the image matching processing unit 533, taking into consideration sensor errors, slippage between the wheels CT and the road surface or ground, and the like, where X is the traveling distance or traveling time from the front wheel position, i.e., the midpoint of a pair of wheels CT provided at the front of the vehicle C, to the first front region R1 and the second front region R2.

[0020] The image matching processing unit 533 is configured to match a current image corresponding to a captured image of the periphery of the vehicle C with the second area image in the history image acquired by the history image acquisition unit 532. Specifically, the image matching processing unit 533 is configured to match the current image with the second area image in the history image using template matching.

[0021] The display processing unit 534 is configured to display, on the in-vehicle display device 2, a first region image included in the history image identified as matching the current image through matching by the image matching processing unit 533, as an underfloor image. The underfloor image is an image of the area directly below the bottom of the vehicle C, and in this embodiment, is an image that is estimated to represent the current state of the road surface or ground around the wheels CT of the vehicle C. Specifically, the display processing unit 534 is configured to generate a display image signal for displaying, on the in-vehicle display device 2, a composite image in which the underfloor image and the current image are arranged side by side, as shown in FIG. 3A. That is, the composite image shown in FIG. 3A includes an underfloor image displayed on a central screen V1 located in the central region in the width direction of the in-vehicle display device 2, a left side image displayed on a left side screen V2 located to the left of the central screen V1, and a right side image displayed on a right side screen V3 located to the right of the central screen V1. In addition, a wheel image TG is compositely displayed on the underfloor image. The wheel image TG is an image virtually showing the wheel CT of the host vehicle C, and is formed by a frame line or a semi-transparent pattern showing the outer shape of the wheel CT. The left side image is a display image corresponding to an image of the road surface or ground around the current host vehicle C taken by the left side camera 33. The right side image is a display image corresponding to an image of the road surface or ground around the current host vehicle C taken by the right side camera 34.

[0022] (Operation overview) Below, we will explain the operation of the in-vehicle system 1 and display control device 5 having the above configuration, as well as an overview of the display control method and display control program executed thereby, along with the effects achieved by such configuration. Note that the display control device 5 according to this embodiment and the display control method and display control program executed thereby will be collectively referred to simply as "this embodiment."

[0023] The peripheral cameras 3, i.e., the front camera 31, rear camera 32, left side camera 33, and right side camera 34, capture images of the surroundings of the vehicle C. Image signals generated as a result of capturing images by the peripheral cameras 3 are input to the display control device 5 by an image signal input unit 51. In addition, information or signals corresponding to the driving state and driving environment of the vehicle C are input to the display control device 5 by a communication interface 52 via the in-vehicle communication bus 4. The display control device 5 controls the display of the in-vehicle display device 2 based on the acquired image signals, etc.

[0024] Specifically, for example, when a predetermined underfloor view start condition is met, the display control device 5 executes an underfloor view operation. The underfloor view operation is an operation that causes the in-vehicle display device 2 to display an underfloor image, which is an image of the area directly below the bottom of the current host vehicle C, i.e., the area around the wheels CT, alongside a current image corresponding to the current images captured by the left side camera 33 and the right side camera 34. This underfloor view operation is effective, for example, when checking wheel chocks or rails in a mechanical parking lot during parking maneuvers, or when checking the situation directly below the host vehicle C or near the wheels CT while traveling on narrow roads or off-road. FIG. 3A shows, as an example, a screen display example when the host vehicle C passes through a ditch ST that is partially blocked by a grating cover GC while traveling forward.

[0025] In conventional technology, an underfloor image is selected using odometry based on driving state information acquired from on-board sensors such as wheel speed sensors. However, with this method, estimation errors occur in the odometry-based estimation of the position or movement amount of the host vehicle C, depending on sensor errors and the slip state between the wheels CT and the road or ground surface. These estimation errors accumulate over time, causing a deviation from the actual image displayed in the image directly below the bottom of the host vehicle C. FIG. 3B shows an example of a display in which such a deviation occurs. In the display example shown in FIG. 3B, a deviation occurs in the traveling direction of the host vehicle C between the underfloor image displayed on the center screen V1 and the images of the sides of the current host vehicle C displayed on the left side screen V2 and the right side screen V3. A screen display with such a deviation may cause discomfort to the user, i.e., the occupants, or may cause the user to misidentify the position of a target object such as a groove ST. Furthermore, for example, if the front wheels of the vehicle C are actually in front of the ditch ST or grating cover GC as shown in Figure 3A, but the underfloor image displays the front wheels as being positioned above the ditch ST or grating cover GC, the impact and sound of the front wheels actually running over the ditch ST or grating cover GC will reach the occupants later than displayed on the screen, which may cause discomfort or discomfort to the occupants.

[0026] Therefore, this embodiment compares a current image corresponding to a captured image of the side of the current host vehicle C with an area in a historical image captured in the past as an image of the host vehicle C's future that overlaps with the current image. Specifically, this comparison can be performed using well-known template matching. When the host vehicle C is traveling forward, the template image can be a target that straddles both the first forward region R1 and the second forward region R2 shown in FIG. 2, such as the upper or lower end of the grating cover GC in the example shown in FIG. 3A. Alternatively, a white line WL, for example, can be used. Then, this embodiment displays an underfloor image corresponding to the historical image identified as matching the current image through comparison. This makes it possible to effectively suppress deviations from reality and the resulting discomfort felt by the user when displaying an image of the area directly below the bottom of the current host vehicle C.

[0027] (Example of operation) 4 is a flowchart showing an example of a display control operation in this embodiment. In the figure, "S" is an abbreviation for "step." In addition, the processor and memory provided in the display control device 5 will be simply referred to as the "processor" and "memory" hereinafter.

[0028] In step 401, the processor inputs an image signal from the peripheral camera 3. In step 402, the processor executes image processing by the viewpoint conversion processing unit 531, i.e., viewpoint conversion processing. In step 403, the processor determines whether the current shift position is R-shift. If the current shift position is R-shift (i.e., step 403 = YES), the processor proceeds to step 404. In step 404, the processor extracts image areas behind and on the rear sides of the host vehicle C from the image captured by the rear camera 32. On the other hand, if the current shift position is not R-shift (i.e., step 403 = NO), the processor proceeds to step 405. In step 405, the processor extracts image areas in front and on the front sides of the host vehicle C from the image captured by the front camera 31. Then, in step 406, the processor stores the image extracted in step 404 or step 405 in memory as a history image.

[0029] In step 407, the processor reads from memory a historical image for a predetermined driving time range or a predetermined driving distance range prior to the current time, and compares it with the current image using template matching. In step 408, the processor extracts an underfloor image from the matched historical image based on the comparison result from template matching. In step 409, the processor generates a display image signal for displaying a composite image in which the underfloor image and the current image are arranged side by side on the in-vehicle display device 2. In step 410, the processor outputs the display image signal generated in step 409 to the in-vehicle display device 2.

[0030] (Variation) The present invention is not limited to the above-described embodiment. Therefore, the above-described embodiment can be modified as appropriate. Representative modifications will be described below. In the following description of the modifications, differences from the above-described embodiment will be mainly described. Furthermore, identical or equivalent parts between the above-described embodiment and the modifications are given the same reference numerals. Therefore, in the following description of the modifications, the description of the above-described embodiment can be used as appropriate for components having the same reference numerals as those in the above-described embodiment, unless there is a technical contradiction or special additional explanation.

[0031] The present invention is not limited to the specific device configurations shown in the above embodiments. For example, the host vehicle C is not limited to a standard automobile, but may be a large vehicle such as a bus or a cargo truck. There is no particular limitation on the number of wheels, and the host vehicle C may be a three-wheeled vehicle, or a six- or eight-wheeled vehicle such as a cargo truck. The type of the host vehicle C may be a conventional vehicle equipped with only an internal combustion engine, an electric vehicle or a fuel cell vehicle without an internal combustion engine, or a so-called hybrid vehicle. There are also no particular limitations on the use of the host vehicle C, the number of occupants, etc.

[0032] The in-vehicle display device 2 may be provided exclusively for displaying images corresponding to images of the surroundings of the vehicle C captured by the surrounding camera 3. Furthermore, the in-vehicle display device 2 is not limited to a CID device and may be, for example, part of a meter panel, or all or part of a display screen of an in-vehicle navigation device provided separately from the CID device. Furthermore, one or both of the left side screen V2 and the right side screen V3 of the in-vehicle display device 2 may be omitted. In other words, the present invention is not limited to a mode in which an underfloor image and a current image are displayed side by side.

[0033] The in-vehicle display device 2 is not limited to a flat display device or a curved display device provided on the dashboard of the host vehicle C. That is, for example, the in-vehicle display device 2 may be a so-called head-up display. Alternatively, the in-vehicle display device 2 may be a portable terminal such as a smartphone or a tablet terminal brought by an occupant of the host vehicle C. In this case, when the in-vehicle display device 2, which is a portable terminal, is brought into the host vehicle C, it is connected to the display control device 5 so as to be able to communicate information with it via short-range wireless communication such as Bluetooth (registered trademark) or TransferJet (registered trademark).

[0034] The display control device 5 may be connected to an ECU in a driving assistance system or an autonomous driving system via an in-vehicle communication bus 4, and may receive information or signals corresponding to the driving state and driving environment of the vehicle C from the ECU. ECU is an abbreviation for Electronic Control Unit.

[0035] The program according to the present invention, which enables the execution of the various operations, procedures, or processes described in the above embodiments, can be downloaded or upgraded via V2X communication. V2X stands for Vehicle to X. Alternatively, the program can be downloaded or upgraded via a terminal device installed in a vehicle manufacturing plant, a repair shop, a dealer, or the like. The program can be stored on a memory card, an optical disk, a magnetic disk, or the like.

[0036] In the above embodiment, the display control device 5 has a configuration as a so-called in-vehicle microcomputer including a CPU and the like. However, the present invention is not limited to such a configuration. For example, all or part of the display control device 5 may be configured to include a digital circuit, such as an ASIC or FPGA, configured to enable the above-described operations. ASIC stands for Application Specific Integrated Circuit. FPGA stands for Field Programmable Gate Array. In other words, the display control device 5 may have both an in-vehicle microcomputer portion and a digital circuit portion.

[0037] In this manner, each of the above functional configurations and methods may be implemented by a special-purpose computer provided by configuring a processor and memory programmed to perform one or more functions embodied in a computer program, or each of the above functional configurations and methods may be implemented by a special-purpose computer provided by configuring a processor with one or more dedicated hardware logic circuits.

[0038] The present invention is not limited to the specific functional configurations and operational examples shown in the above embodiment. For example, the operation in the above embodiment is described for forward driving, but the present invention can also be effectively applied to reverse driving. That is, the image capture area corresponding to the first forward area R1 and the second forward area R2 shown in FIG. 2 can be defined behind the host vehicle C when reversing.

[0039] The viewpoint conversion processing unit 531 may convert the image captured by the peripheral camera 3 into an around-view image looking down vertically instead of a bird's-eye view image looking down diagonally.

[0040] Similar expressions such as "obtain," "calculate," "estimate," "detect," "sensing," and "determine" may be substituted for each other as appropriate within the scope of technical inconsistency. "Detect" or "detection" and "extract" may also be substituted for each other as appropriate within the scope of technical inconsistency.

[0041] It goes without saying that the elements constituting the above-described embodiments are not necessarily essential unless expressly stated as essential or clearly considered essential in principle. Furthermore, when numerical values ​​such as the number, amount, range, etc. of components are mentioned, the present invention is not limited to those specific numerical values ​​unless expressly stated as essential or clearly limited to specific numerical values ​​in principle. Similarly, when the shape, direction, positional relationship, etc. of components are mentioned, the present invention is not limited to those shapes, directions, positional relationship, etc. unless expressly stated as essential or clearly limited to specific shapes, directions, positional relationship, etc. in principle.

[0042] The variations are not limited to the above examples. For example, all or part of one of the variations may be combined with all or part of another variation, provided that there is no technical inconsistency. Furthermore, all or part of the above-described embodiments may be combined with all or part of the above-described variations, provided that there is no technical inconsistency. In other words, the disclosure of this specification is not limited to the exemplified embodiments. The disclosure encompasses the exemplified embodiments and modifications made by those skilled in the art based on them. For example, the disclosure is not limited to the combinations of parts and elements shown in the embodiments, but can be implemented in various modifications. The disclosure can be implemented in various combinations. The disclosure can have additional parts that can be added to the embodiments. The disclosure encompasses omissions of parts and elements from the embodiments. The disclosure encompasses substitutions or combinations of parts and elements between one embodiment and another embodiment. The disclosed technical scope is not limited to the description of the embodiments. The disclosed technical scope is defined by the claims, and should be interpreted as including all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0043] 2 In-vehicle display device 31 Front camera 32 Rear camera 33 Left side camera 34 Right side camera 5 Display control device 532 History image acquisition unit 533 Image matching processing unit 534 Display processing section 54 Historical image storage section

Claims

1. A display control device (5) that controls the display of an in-vehicle display device (2), a historical image acquisition unit (532) that acquires the historical images from a historical image storage unit (54) that stores, in chronological order, photographed images including a first area image corresponding to a first area (R1) around the host vehicle equipped with the on-vehicle display device and a second area image corresponding to a second area (R2) outside the first area in the vehicle width direction as historical images; an image matching processing unit (533) that matches a current image corresponding to a currently captured image of the surroundings of the vehicle with the second area image in the history image acquired by the history image acquisition unit; a display processing unit (534) that displays, on the in-vehicle display device, the first area image included in the history image that has been identified as matching the current image through matching in the image matching processing unit, as an underfloor image that is an image of a portion directly below the bottom of the vehicle; Equipped with the image matching processing unit matches the current image with the second region image in the history image using template matching; Display control device.

2. the first area image and the second area image include an image of a road surface or a ground surface; The display control device according to claim 1 .

3. The history image corresponds to an image captured by a front camera (31) that captures images in front of and on the front sides of the vehicle or a rear camera (32) that captures images in the rear of and on the rear sides of the vehicle, The current image corresponds to an image captured by a left side camera (33) capturing an image on the left side of the vehicle or a right side camera (34) capturing an image on the right side of the vehicle. The display control device according to claim 1 or 2.

4. the display processing unit causes the in-vehicle display device to display the under-floor image and the current image side by side. The display control device according to any one of claims 1 to 3.

5. A display control program executed by a display control device (5) that controls the display of an in-vehicle display device (2), The process executed by the display control device is a historical image acquisition process for acquiring the historical images from a historical image storage unit (54) that stores, in chronological order, photographed images including a first area image corresponding to a first area (R1) around the host vehicle equipped with the on-vehicle display device and a second area image corresponding to a second area (R2) outside the first area in the vehicle width direction as historical images; an image matching process for matching a current image corresponding to a currently captured image of the surroundings of the host vehicle with the second area image in the history image acquired by the history image acquisition process; a display process for displaying, on the in-vehicle display device, the first area image included in the history image identified as matching the current image through the comparison in the image comparison process, as an underfloor image that is an image of a portion directly below the bottom of the vehicle; Including, the image matching process is a process of matching the current image with the second region image in the history image using template matching; Display control program.

6. the first area image and the second area image include an image of a road surface or a ground surface; The display control program according to claim 5 .

7. The history image corresponds to an image captured by a front camera (31) that captures images in front of and on the front sides of the vehicle or a rear camera (32) that captures images in the rear of and on the rear sides of the vehicle, The current image corresponds to an image captured by a left side camera (33) capturing an image on the left side of the vehicle or a right side camera (34) capturing an image on the right side of the vehicle.

7. The display control program according to claim 5.

8. the display process displays the underfloor image and the current image side by side on the in-vehicle display device. The display control program according to any one of claims 5 to 7.

Images