Periphery monitoring device

Abstract

To provide a periphery monitoring device applicable to a vehicle, which does not have map data, and capable of causing a driver to identify which position is a registered position without requiring a map expanding operation.SOLUTION: In a case where a driver performs a registering operation for registering a present position of a vehicle as a registered position, a control unit acquires as a registration-time image a periphery image which is generated based on an image captured by an imaging apparatus at the present position, stores registered position information associating the registration-time image with the present position and, in a case where a position condition that the vehicle is positioned within a predetermined first range with the registered position defined as a center is established, displays a periphery screen by establishing a display condition. Further, in a case where a predetermined erasure condition is established, the control unit displays on a display device a selection screen including registration-time images of the stored registered position information and erases registered position information of a registration-time image which is selected on the selection screen by the driver.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a surroundings monitoring device that displays a surroundings screen including a predetermined surroundings image on a display device mounted in a vehicle when a predetermined display condition is met. [Background technology]

[0002] Conventionally, there has been known a surroundings monitoring device that displays a surroundings screen including a surrounding image on a display device. For example, a surroundings monitoring device described in Patent Document 1 (hereinafter referred to as a "conventional device") automatically displays a surroundings screen including an image of a range including a vehicle body part that is most likely to come into contact with the vehicle when the vehicle is traveling on a narrow road or a narrow winding road. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2012-71833 A Summary of the Invention

[0004] The present inventors are considering a surroundings monitoring device that automatically displays a surroundings screen when the vehicle is located near a "registered position stored in advance by the driver." In such a surroundings monitoring device, the registered position needs to be erased when the storage capacity of the storage device that stores the registered position is reached or when the driver requests it.

[0005] When deleting a registered position, the surroundings monitoring device prompts the driver to select the registered position to be deleted. In this case, the surroundings monitoring device needs to display a screen that enables the driver to specify which registered position is.

[0006] A surroundings monitoring device may be considered that uses map data stored in a navigation device mounted on a vehicle and displays a registered position on a map of the map data, but has the following problems. - Cannot be applied to vehicles that do not have map data. - When multiple registered locations are located relatively close to each other, the driver must perform a zoom operation to enlarge the map.

[0007] The present invention has been made to address the above-mentioned problems. That is, one of the objects of the present invention is to provide a surroundings monitoring device that can be applied to a vehicle that does not have map data and that allows the driver to specify where the registered position is without the need to enlarge the map.

[0008] The perimeter monitoring device of the present invention (hereinafter also referred to as the "device of the present invention") is Imaging devices (22FR, 22RR, 22L, 22R) configured to image the surrounding area of ​​the vehicle; a control unit (20) configured to display, when a predetermined display condition is satisfied, a surrounding screen including a predetermined surrounding image generated based on an image captured by the imaging device on a display device (30) mounted on the vehicle; Equipped with The control unit When the driver of the vehicle performs a registration operation to register the current position of the vehicle as a registered position (step 705 "Yes"), the surrounding image generated based on the image captured by the imaging device at the current position is acquired as a registration time image (step 715), and registration time image that associates the registration time image with the current position is stored (step 725). When the location condition that the vehicle is located within a predetermined first range (AR1) centered on the registered location is satisfied (step 525 "Yes"), the display condition is satisfied (step 540), and the surrounding screen is displayed on the display device (step 605 "Yes", step 630, step 640). When a predetermined deletion condition is met (step 720 "No"), a selection screen (320) including the image at the time of registration of the stored registration position information is displayed on the display device (step 730); The registration position information of the registration image selected by the driver on the selection screen is erased (step 740). It is structured as follows.

[0009] The device of the present invention displays a selection screen including an image at registration, and erases the registration position information of the image at registration selected by the driver on the selection screen. This allows the driver to identify the location of the registration position information by visually checking the image at registration included in the selection screen. Therefore, the device of the present invention can be applied to vehicles that do not have map data, and the device of the present invention allows the driver to identify the location of the registration position without the need to zoom in on the map.

[0010] In one embodiment of the device of the present invention, The control unit is configured to, when the surrounding screen is displayed on the display device because the position condition is met (step 820 ``Yes''), end display of the surrounding screen (step 840) when the vehicle is located outside a second range that is set to be wider than the first range centered on the registered position (step 835 ``Yes'').

[0011] In this embodiment, the display of the peripheral screen is terminated not when the vehicle is located outside the first range, but when the vehicle is located outside the "second range set to be wider than the first range with the registered position as the center". This reduces the possibility that the display of the peripheral screen will be repeatedly started and ended due to the current position of the vehicle being determined to be within the first range or outside the first range, even if the accuracy of identifying the current position of the vehicle VA is poor. This reduces the possibility that the driver will find the repeated start and end of the display of the peripheral screen annoying.

[0012] In one embodiment of the device of the present invention, The control unit Identifying (25) a current position of the vehicle based on positioning signals received from a plurality of artificial positioning satellites; If the current position of the vehicle cannot be identified while the surrounding screen is being displayed on the display device because the position condition is satisfied (step 825 "No"), and if the distance traveled by the vehicle from the point when the current position of the vehicle cannot be identified reaches or exceeds a predetermined threshold distance (step 850 "Yes"), the display of the surrounding screen is terminated (step 840). It is structured as follows.

[0013] In this embodiment, when the current position of the vehicle cannot be identified, the display of the peripheral screen is terminated when the distance traveled from the point when the current position of the vehicle cannot be identified reaches or exceeds a threshold distance. This makes it possible to appropriately terminate the display of the peripheral screen even when the current position of the vehicle cannot be identified because the vehicle is located in an underground parking lot or the like and is therefore unable to receive a positioning signal.

[0014] In one embodiment of the device of the present invention, The control unit At least one of an image (302) in the traveling direction of the vehicle and an overhead image (304) of the surrounding area of ​​the vehicle is acquired as the surrounding image (step 615, step 715); displaying the peripheral screen including the peripheral image and the selection screen on the display device (steps 630, 640, and 730); It is structured as follows.

[0015] This allows the driver to grasp the situation of at least one of the area in the direction in which the vehicle is traveling and the area around the vehicle by viewing the surrounding screen.

[0016] In one embodiment of the device of the present invention, When the driver performs the registration operation (step 705 "Yes"), and when the number of stored registered location information is equal to or greater than a predetermined threshold (step 720 "No"), the control unit is configured to satisfy the deletion condition and display the selection screen on the display device (step 730).

[0017] In this embodiment, when the number of stored registered location information (number of registrations) is equal to or greater than a threshold, a selection screen is displayed to allow the driver to select the registered location information to be deleted. This makes it possible to prevent the storage capacity of the registered location information from becoming too large.

[0018] In one embodiment of the device of the present invention, The control unit is configured to display the surrounding screen on the display device by satisfying the display condition not only when the position condition is satisfied, but also when either an operation condition is satisfied that the driver has performed a request operation to request the display of the surrounding screen (step 520 ``Yes'') or an obstacle condition is satisfied that an obstacle has been detected around the vehicle (step 530 ``Yes'').

[0019] In this aspect, the peripheral screen is displayed when the operation condition is satisfied, so that the driver can display the peripheral screen by performing a request operation when he / she wishes to display the peripheral screen. Furthermore, in this aspect, the peripheral screen is displayed when the obstacle condition is satisfied, so that the peripheral screen can be displayed when an obstacle is present around the vehicle, and the possibility that the driver can recognize the obstacle can be increased.

[0020] In the above aspect, The display device is a touch panel type display device, The control unit When the peripheral screen is displayed on the display device, a registered switch image (306) is also displayed on the display device together with the peripheral screen; If the driver touches the registration switch image (step 705 "Yes"), it is assumed that the driver has performed the registration operation, and the registration time image is acquired and the registration position information is stored (step 725). It is structured as follows.

[0021] In this embodiment, the registration switch image is displayed together with the peripheral screen, which improves operability when the driver performs the registration operation.

[0022] In the above aspect, The control unit When the vehicle speed is equal to or lower than a predetermined threshold vehicle speed (step 620 "Yes"), it is determined that the driver has performed the registration operation when the driver touches the registration switch image. If the vehicle speed is greater than the threshold vehicle speed (step 620 “No”), even if the driver touches the registration switch image, it is not considered that the driver has performed the registration operation. It is structured as follows.

[0023] Since the driver is considered to have performed a registration operation only when the driver touches the registration switch image when the vehicle speed is equal to or lower than the threshold vehicle speed, the possibility that the driver will attempt to touch the registration switch image when the vehicle speed is higher than the threshold vehicle speed can be reduced.

[0024] In the above description, in order to facilitate understanding of the invention, the names and / or symbols used in the embodiments described below are added in parentheses to the configurations of the invention corresponding to those embodiments. However, each component of the invention is not limited to the embodiments defined by the names and / or symbols. Other objects, other features, and associated advantages of the present invention will be easily understood from the following description of the embodiments of the present invention, which will be described with reference to the drawings. [Brief description of the drawings]

[0025] [Figure 1] FIG. 1 is a schematic system configuration diagram of a surroundings monitoring device according to an embodiment of the present invention. [Diagram 2] FIG. 2 is an explanatory diagram illustrating an outline of the operation of the periphery monitoring device according to the embodiment of the present invention. [Diagram 3] FIG. 3 is an explanatory diagram of a selection screen displayed on the display device shown in FIG. [Figure 4] FIG. 4 is an explanatory diagram of the first range and the second range. [Diagram 5]FIG. 5 is a flowchart showing a display condition determination routine executed by the CPU of the PVMECU shown in FIG. [Figure 6] FIG. 6 is a flowchart showing a display control routine executed by the CPU of the PVMECU shown in FIG. [Figure 7] FIG. 7 is a flowchart showing a registration location information registration routine executed by the CPU of the PVMECU shown in FIG. [Figure 8] FIG. 8 is a flowchart showing a termination condition determination routine executed by the CPU of the PVMECU shown in FIG. [Figure 9] FIG. 9 is a flowchart showing a lost mid-distance acquisition subroutine executed by the CPU of the PVMECU shown in FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] <Configuration> As shown in FIG. 1, a surroundings monitoring device 10 according to an embodiment of the present invention is mounted on a vehicle VA.

[0027] The surroundings monitoring device 10 includes a panoramic view monitor ECU 20 (hereinafter, referred to as "PVMECU 20").

[0028] ECU is an abbreviation for Electronic Control Unit, and is an electronic control circuit whose main components are a microcomputer including a CPU, ROM, RAM, and interfaces. ECUs are also called "control units," "controllers," or "computers." The CPU performs various functions by executing instructions (routines, programs) stored in memory (ROM). The functions of an ECU may be distributed among multiple ECUs.

[0029] The surroundings monitoring device 10 includes a wheel speed sensor 21, a front camera 22FR, a rear camera 22RR, a left camera 22L, a right camera 22R, four front sonars 23FR, four rear sonars 23RR, a camera switch (SW) 24, and a GNSS (Global Navigation Satellite System) receiver 25. These are connected to the PVMECU 20 for data exchange.

[0030] A wheel speed sensor 21 is provided for each wheel of the vehicle VA. Each wheel speed sensor 21 generates one wheel pulse signal each time the corresponding wheel rotates a predetermined angle. The PVMECU 20 counts the number of pulses per unit time of the wheel pulse signal generated by each wheel speed sensor 21, and obtains the rotation speed of each wheel based on the number of pulses. Then, the PVMECU 20 obtains a vehicle speed Vs indicating the speed of the vehicle VA based on the rotation speed of each wheel. As an example, the PVMECU 20 obtains the average value of the wheel speeds of the four wheels as the vehicle speed Vs.

[0031] The front camera 22FR is disposed, for example, near the center of the front grill FG of the vehicle VA in the vehicle width direction. The front camera 22FR obtains a front image, which is an image of the area in front of the vehicle VA, and transmits the front image to the PVMECU20.

[0032] The rear camera 22RR is disposed, for example, near the center of a back door (not shown) of the vehicle VA in the vehicle width direction. The rear camera 22RR acquires a rear image, which is an image of the area behind the vehicle VA, and transmits the rear image to the PVMECU20.

[0033] The left camera 22L is disposed, for example, on the left side mirror LSM. The left camera 22L captures an image of the left side area of ​​the vehicle VA to obtain a left side image, which is an image of the area, and transmits the left side image to the PVMECU20.

[0034] The right camera 22R is disposed, for example, in the right side mirror RSM. The right camera 22R obtains a right side image, which is an image of the area on the right side of the vehicle VA, and transmits the right side image to the PVMECU20.

[0035] The images captured by the cameras 22FR to 22R are combined to form an image of the area around the vehicle VA (360 degrees).

[0036] The four front sonars 23FR are disposed at the left end, the left center, the right center, and the right end of the front bumper FB of the vehicle VA, respectively.

[0037] The front sonar 23FR emits sound waves and receives the sound waves reflected by a three-dimensional object (hereinafter, sometimes referred to as an “obstacle”). Based on the received sound waves, the front sonar 23FR acquires the distance from the vehicle VA to the three-dimensional object and the lateral position of the three-dimensional object relative to the vehicle VA as forward three-dimensional object information, and transmits the forward three-dimensional object information to the PVMECU20.

[0038] The PVMECU20 can detect three-dimensional objects existing in the area from the left front side to the right front side of the vehicle VA based on the forward three-dimensional object information received from each front sonar FR23.

[0039] The four rear sonars 23RR are disposed at the left end, the left center, the right center, and the right end of a rear bumper (not shown) of the vehicle VA, respectively.

[0040] The rear sonar 23RR emits sound waves, receives the sound waves reflected by the three-dimensional object, obtains the distance to the three-dimensional object and the lateral position of the three-dimensional object based on the received sound waves as rear three-dimensional object information, and transmits the rear three-dimensional object information to the PVMECU20.

[0041] The PVMECU 20 can detect three-dimensional objects existing in the area extending from the left rear side to the right rear side of the vehicle VA based on the rear three-dimensional object information received from each rear sonar 23RR.

[0042] The camera SW24 is disposed near the steering wheel (not shown) of the vehicle VA and at a position where the driver of the vehicle VA can operate it. When the driver operates the camera SW24, the PVMECU20 displays the front mode screen 300 shown in Fig. 2 on the display device 30. The front mode screen 300 may be referred to as the "periphery screen."

[0043] The GNSS receiver 25 is a device that receives positioning signals transmitted from a plurality of artificial positioning satellites. The PVMECU 20 identifies the current position (latitude and longitude) of the vehicle VA based on the positioning signals received by the GNSS receiver 25.

[0044] Furthermore, the surroundings monitoring device 10 includes a display device 30 and a storage device 31. These are connected to the PVMECU 20 so as to be able to exchange data with each other.

[0045] The display device 30 is disposed within the cabin of the vehicle VA at a position visible to the driver, and is a touch panel display that allows the driver to perform input by touching a screen displayed on the display device 30. For example, the display device 30 is a multimedia display provided near the center of the instrument panel of the vehicle VA.

[0046] The storage device 26 is a non-volatile storage device from which the PVMECU 20 can read and write data. For example, the storage device 26 is a hard disk drive, but is not limited to a hard disk drive, and may be any well-known storage device or storage medium from which the PVMECU 20 can read and write data.

[0047] The storage device 26 has an image storage unit 310 and a registration position information storage unit 312. A registration time image, which will be described later, is stored in the image storage unit 310. The registration position information storage unit 312 stores registration position information, which associates a registration position, which will be described later, with the registration time image.

[0048] (Overview of operation) An outline of the operation of the periphery monitoring device 10 will now be described with reference to FIGS. 2A, when the vehicle VA enters a T-junction, the driver wants to check the conditions on the left and right sides of the T-junction because visibility is poor at the T-junction, and operates the camera SW 24. If the vehicle speed Vs is equal to the first threshold vehicle speed Vsth1 when the camera SW 24 is operated, the surroundings monitoring device 10 displays the front mode screen 300 on the display device 30.

[0049] The front mode screen 300 includes a traveling direction image area 302 , a panoramic view (PV) image area 304 and a registration switch (SW) image 306 . In the traveling direction image area 302, a traveling direction image generated based on a forward image captured by the front camera 22FR is displayed. A PV image generated based on the front image, the rear image, the left side image, and the right side image is displayed in the PV image area 304. The PV image is an overhead image of the surrounding area of ​​the vehicle VA viewed from a viewpoint vertically above the vehicle VA.

[0050] The registration SW image 306 is an image that the driver touches when registering the current position as a registered position. When the vehicle speed Vs is equal to or lower than a "second threshold vehicle speed vsth that is lower than the first threshold vehicle speed Vsth1," the surroundings monitoring device 10 regards this touch operation as a registration operation and registers the current position of the vehicle VA as the registered position.

[0051] 2B, the vehicle VA stops before the stop line (i.e., the vehicle speed Vs becomes equal to or less than the second threshold vehicle speed Vsth2). The driver touches the registered SW image 306, wanting to display the front mode screen 300 whenever the vehicle VA passes through this T-junction in the future.

[0052] When the vehicle speed Vs is equal to or less than the second threshold vehicle speed Vsth2 and the driver touches the registered SW image 306, the surroundings monitoring device 10 determines whether the number N of registered position information stored in the registered position information memory unit 312 (hereinafter referred to as the "number of registrations") is less than the threshold Nth.

[0053] When the registration number N is less than the threshold value Nth, as shown in (C) of Fig. 2, the periphery monitoring device 10 acquires a forward image, a rearward image, a left side image, and a right side image, generates a PV image based on these images, generates a traveling direction image based on the forward image, and stores the traveling direction image and the PV image as the registration time image in the image storage unit 310. Furthermore, in this case, the periphery monitoring device 10 stores registration position information that associates the current position of the vehicle VA with the registration time image in the registration position information storage unit 312. In detail, the periphery monitoring device 10 stores registration position information that associates the current position of the vehicle VA with the "storage path indicating the location where the registration time image is stored" in the registration position information storage unit 312.

[0054] On the other hand, when the number of registrations N is equal to or greater than the threshold value Nth, as shown in Fig. 2(D), the surroundings monitoring device 10 displays the selection screen 320 shown in Fig. 3. The selection screen 320 is a screen for allowing the driver to select the registered position information to be deleted.

[0055] The selection screen 320 will be described with reference to FIG. On the selection screen 320, an image at the time of registration of the registered position information stored in the registered position information storage unit 312 is displayed. The selection screen 320 includes a check box area 322 , a registration image area 324 , an OK image 326 , and a cancel image 328 . A check box image corresponding to each registration image is displayed in check box area 322. When a registration image is not selected, the check box image corresponding to this registration image is white, and when a registration image is selected, the check box image corresponding to this registration image changes to black. In the registration time image area 324, the registration time images (travel direction image and PV image) are displayed.

[0056] When at least one registration image is selected and the OK image 326 is touched, The periphery monitoring device 10 deletes the registration position information of the selected registration-time image from the registration position information storage unit 312, and also deletes the registration-time image of the registration position information from the image storage unit 310. Then, the periphery monitoring device 10 stores the registration position information that associates the current position of the vehicle VA with the newly acquired registration-time image in the registration position information storage unit 312, and stores the registration-time image in the image storage unit 310.

[0057] If the cancel image 328 is touched when at least one registration image is selected, the periphery monitoring device 10 cancels the selection. Furthermore, if the cancel image 328 is touched when no registration image is selected, the periphery monitoring device 10 cancels storing new registration position information.

[0058] The surroundings monitoring device 10 according to this embodiment displays a selection screen 320 including an image at the time of registration of the registered position information stored in the registered position information storage unit 312. This makes it easier for the driver to identify the location of the registered position information, and therefore allows the driver to erase unnecessary registered position information.

[0059] A periphery monitoring device that displays the registered positions of the registered position information on a map to allow the driver to select the registered position information to be deleted cannot be applied to a vehicle VA that does not have map data. In contrast, the periphery monitoring device 10 according to the present embodiment can be applied to a vehicle VA that does not have map data because it is sufficient to store an image at the time of registration when storing (registering) the registered position information. Furthermore, according to the periphery monitoring device 10 according to the present embodiment, even if multiple registered positions are located relatively close to each other, the driver can identify which position the registered position is by visually checking the image at the time of registration, eliminating the need to zoom in on the map.

[0060] (Display conditions) When the display condition is met, the surroundings monitoring device 10 starts displaying the front mode screen 300 on the display device 30.

[0061] The display condition is met when the vehicle speed Vs is equal to or less than the first threshold vehicle speed Vsth1 and the shift position SP is a position other than the parking (P) position or the reverse (R) position, and any of the following operation conditions, obstacle conditions, and position conditions are met.

[0062] Operation condition: Camera SW24 is operated. Obstacle condition: At least one of the front sonar 23FR and rear sonar 23RR detects a three-dimensional object (obstacle). Location condition: the vehicle VA is located within a predetermined first range AR1 (see FIG. 4) from the registered location of the registered location information.

[0063] As shown in FIG. 4, for example, the first range AR1 has a circular shape having a predetermined radius r1 centered on the registration position.

[0064] (Ending conditions) When the termination condition is met, the surroundings monitoring device 10 terminates the display of the front mode screen 300 on the display device 30, and displays on the display device 30 the screen that was displayed on the display device 30 before the front mode screen 300 was displayed.

[0065] The termination condition is met when the following condition 1 is met. Condition 1: The vehicle speed Vs becomes greater than a first threshold vehicle speed Vsth1.

[0066] Furthermore, when the display condition is met because the obstacle condition is met, the end condition is met when the following condition 2 is met even if condition 1 is not met. Condition 2: Neither the front sonar 23FR nor the rear sonar 23RR detected any three-dimensional objects.

[0067] Furthermore, when the display condition is met because the position condition is met, the end condition is met when either of the following conditions 3 and 4 is met even if condition 1 is not met. Condition 3: The vehicle VA is located outside the second range AR2 shown in FIG.

[0068] 4, for example, the second range AR2 is a circle having a predetermined radius r2 centered on the registration position. The radius r2 is longer than the radius r1. Therefore, the second range AR2 is wider than the first range AR1.

[0069] If the display of the front mode screen 300 ends when the condition that the vehicle VA is located outside the first range AR1 is satisfied, the current position of the vehicle is determined to be within the first range or outside the first range, even if the accuracy of identifying the current position of the vehicle VA is poor. This may cause the display of the front mode screen 300 to repeatedly start and end, which may be annoying to the driver. In this embodiment, the second range AR2 is set to be wider than the first range AR1, which reduces the possibility that the display of the front mode screen 300 will repeatedly start and end, and thus reduces the possibility that the driver will feel annoyed.

[0070] Condition 4: When the GNSS receiver 25 is no longer able to receive positioning signals (i.e., when it is no longer able to determine the current position of the vehicle VA), the distance traveled by the vehicle VA from the point when it was no longer able to receive positioning signals (distance traveled while lost) D becomes greater than or equal to the threshold distance Dth.

[0071] If the display of the front mode screen 300 ends when the condition that the GNSS receiver 25 can no longer receive the positioning signal is met, the driver may feel annoyed that the display of the front mode screen 300 ends even though the driver wants to see the front mode screen 300. According to this embodiment, even if the GNSS receiver 25 can no longer receive the positioning signal, the display of the front mode screen 300 does not end unless the vehicle VA travels a threshold distance Dth or more. If the vehicle VA has traveled a threshold distance Dth or more, there is a high possibility that the driver has already lost the need to see the front mode screen 300. This reduces the possibility that the driver feels annoyed.

[0072] (Specific operation) <Display condition determination routine> The CPU of the PVMECU 20 (hereinafter, when the term "CPU" is used, it refers to the CPU of the PVMECU 20 unless otherwise specified) executes a display condition determination routine shown in the flowchart of FIG. 5 every time a predetermined time elapses.

[0073] 5, and proceeds to step 505. In step 505, the CPU determines whether the first display flag Xdp1, the second display flag Xdp2, and the third display flag Xdp3 all have a value of "0."

[0074] The value of the first display flag Xdp1 is set to "1" when the display condition is met by satisfying the above-mentioned operation condition, and is set to "0" when the above-mentioned termination condition is met. The CPU sets the value of the first display flag Xdp1 to "0" in an initial routine that is executed when an ignition key switch (not shown) of the vehicle VA is changed from the OFF position to the ON position.

[0075] The value of the second display flag Xdp2 is set to "1" when the display condition is met by satisfying the above position condition, and is set to "0" when the above end condition is met. The CPU sets the value of the second display flag Xdp2 to "0" in the initial routine.

[0076] The value of the third display flag Xdp3 is set to "1" when the display condition is met by satisfying the obstacle condition, and is set to "0" when the end condition is met. The CPU sets the value of the third display flag Xdp3 to "0" in the initial routine.

[0077] If the values ​​of the first display flag Xdp1, the second display flag Xdp2, and the third display flag Xdp3 are all “0”, the CPU determines “Yes” in step 505 and proceeds to step 510.

[0078] In step 510, the CPU determines whether the vehicle speed Vs is equal to or less than the first threshold vehicle speed Vsth1. If the vehicle speed Vs is greater than the first threshold vehicle speed Vsth1, the CPU determines "No" in step 510 and proceeds to step 595 to temporarily end this routine.

[0079] If the vehicle speed Vs is equal to or lower than the first threshold vehicle speed Vsth1, the CPU determines "Yes" in step 510 and proceeds to step 515. In step 515, the CPU determines whether the shift position SP is either the P position or the R position.

[0080] If the shift position SP is either the P position or the R position, the CPU determines "Yes" in step 515, proceeds to step 595, and temporarily ends this routine.

[0081] If the shift position SP is neither the P position nor the R position, the CPU determines "No" in step 515, and proceeds to step 520. In step 520, the CPU determines whether or not the camera switch 24 has been operated.

[0082] If the camera SW 24 has not been operated, the CPU determines "No" in step 520, and proceeds to step 525. In step 525, the CPU determines whether or not the vehicle VA is located within the first range AR1.

[0083] If the vehicle VA is not located within the first range AR1, the CPU determines "No" in step 525 and proceeds to step 530. In step 530, the CPU determines whether or not at least one of the front sonar 23FR and the rear sonar 23RR has detected an obstacle, based on the front solid object information and the rear solid object information.

[0084] If neither the front sonar 23FR nor the rear sonar 23RR has detected an obstacle, the CPU makes a "No" determination in step 530, proceeds to step 595, and temporarily ends this routine.

[0085] On the other hand, if the camera SW 24 is being operated when the CPU proceeds to step 520, the CPU determines "Yes" in step 520 and proceeds to step 535. In this case, since the display condition is satisfied because the operation condition is satisfied, the CPU sets the value of the first display flag Xdp1 to "1" in step 535 and proceeds to step 595 to temporarily end this routine.

[0086] On the other hand, if the vehicle VA is located within the first range AR1 when the CPU proceeds to step 525, the CPU determines "Yes" in step 525 and proceeds to step 540. In this case, since the display condition is satisfied because the position condition is satisfied, the CPU sets the value of the second display flag Xdp2 to "1" in step 540 and proceeds to step 595 to temporarily end this routine.

[0087] On the other hand, if at least one of the front sonar 23FR and the rear sonar 23RR has detected an obstacle when the CPU proceeds to step 530, the CPU determines "Yes" in step 530 and proceeds to step 545. In this case, since the display condition is met because the obstacle condition is met, the CPU sets the value of the third display flag Xdp3 to "1" in step 545 and proceeds to step 595 to temporarily end this routine.

[0088] On the other hand, if the value of any of the first display flag Xdp1, the second display flag Xdp2, and the third display flag Xdp3 is "1" when the CPU proceeds to step 505, the CPU judges the result to be "No" in step 505, proceeds to step 595, and temporarily terminates this routine.

[0089] <Display control routine> The CPU executes a display control routine shown in the flowchart of FIG. 6 every time a predetermined time elapses.

[0090] 6, and proceeds to step 605. In step 605, the CPU determines whether the value of any one of the first display flag Xdp1, the second display flag Xdp2, and the third display flag Xdp3 is "1."

[0091] If the values ​​of the first display flag Xdp1, the second display flag Xdp2, and the third display flag Xdp3 are all "0", the CPU determines "No" in step 605, proceeds to step 695, and temporarily ends this routine.

[0092] If the value of any one of the first display flag Xdp1, the second display flag Xdp2, and the third display flag Xdp3 is "1", the CPU determines "Yes" in step 605 and executes steps 610 to 620 in order.

[0093] Step 610: The CPU obtains a front image, a rear image, a left side image, and a right side image from each of the cameras 22FR to 22R. Step 615: The CPU generates a PV image based on the front image, the rear image, the left side image, and the right side image, and generates a traveling direction image based on the front image. Step 620: The CPU determines whether the vehicle speed Vs is equal to or lower than a second threshold vehicle speed Vsth2.

[0094] If the vehicle speed Vs is equal to or lower than the second threshold vehicle speed Vsth2, the CPU determines "Yes" in step 620 and executes steps 625 and 630 in this order. Step 625: The CPU sets the first registered SW image as the registered SW image 306. Step 630: The CPU transmits a first display command to the display device 30, including the travel direction image, the PV image, and the first registered SW image. Thereafter, the CPU proceeds to step 695 and ends this routine.

[0095] When the display device 30 receives the first display command, it displays the front mode screen 300 including the travel direction image, the PV image, and the first registered SW image included in the first display command. When the first registered SW image is touched, the display device 30 transmits touch information indicating that the registered SW image 306 has been touched to the PVMECU20.

[0096] On the other hand, if the vehicle speed Vs is greater than the second threshold vehicle speed Vsth2, the CPU determines "No" in step 620 and executes steps 635 and 640 in this order. Step 635: The CPU sets the second registered SW image as the registered SW image 306. It is preferable that the second registered SW image is an image of a registered SW in a manner that is less noticeable than the first registered SW image. For example, the first registered SW image is a solid line, whereas the second registered SW image is a dotted line. The color of the second registered SW image may be closer to the background color of the front mode screen 300 than the color of the first registered SW image.

[0097] Step 640: The CPU transmits a second display command including the travel direction image, the PV image, and the second registered SW image. Thereafter, the CPU proceeds to step 695 and ends this routine.

[0098] When the display device 30 receives the second display command, the display device 30 displays the front mode screen 300 including the travel direction image, the PV image, and the second registered SW image included in the second display command. Even if the second registered SW image is touched, the display device 30 does not transmit the touch information to the PVMECU20.

[0099] Since the registered SW image 306 is displayed on the front mode screen 300 together with the traveling direction image and the PV image, it becomes easy for the driver to register the current position of the vehicle VA as the registered position. Furthermore, when the vehicle speed Vs is greater than the second threshold vehicle speed Vsth2, even if the registered SW image 306 is touched, the CPU does not regard it as a registration operation of the registration position. On the other hand, when the vehicle speed Vs is equal to or less than the second threshold vehicle speed Vsth2, if the registered SW image 306 is touched, The CPU regards the registration operation as having been performed. This reduces the possibility that the driver will attempt to touch the registration SW image and engage in distracted driving when the vehicle VA is traveling at a vehicle speed Vs that is greater than the second threshold vehicle speed Vsth2.

[0100] <Registration location information registration routine> The CPU executes a registration location information registration routine shown in the flowchart of FIG. 7 every time a predetermined time elapses.

[0101] 7, the CPU starts the process from step 700 and proceeds to step 705. In step 705, the CPU determines whether or not touch information has been received (that is, whether or not the registered SW image 306 has been touched).

[0102] If the registered SW image 306 has not been touched, the CPU determines "No" in step 705, proceeds to step 795, and ends this routine for the time being.

[0103] If the registered SW image 306 has been touched, the CPU determines "Yes" in step 705 and executes steps 710 to 720 in order. Step 710: The CPU acquires the current position of the vehicle VA based on the positioning signal received by the GNSS receiver 25. Step 715: The CPU generates a PV image based on the forward image, rear image, left side image, and right side image acquired from each camera 22FR to 22R, and generates a traveling direction image based on the forward image, and acquires the traveling direction image and the PV image as registration images. Step 720: The CPU determines whether or not the number of registered location information items (number of registrations) N stored in the registration time location information is less than a threshold value Nth.

[0104] Step 725: The CPU stores in the registration position information storage unit 312 the registration position information that associates the current position of the vehicle VA with the registration-time image, and stores in the image storage unit 310 the registration-time image. Thereafter, the CPU proceeds to step 795 and ends this routine.

[0105] On the other hand, if the number of registrations N is equal to or greater than the threshold value Nth when the CPU proceeds to step 720, the CPU determines "No" in step 720 and executes steps 730 and 735 in this order.

[0106] Step 730 : The CPU displays the selection screen 320 on the display device 30 . Step 735: The CPU determines whether or not the OK image 326 has been touched in a state in which at least one registration image has been selected on the selection screen 320.

[0107] If OK image 326 is touched with at least one registration image selected, the CPU determines "Yes" in step 735 and proceeds to step 740. In step 740, the CPU erases the registration position information of the selected registration image from registration position information storage unit 312, and erases the registration image from image storage unit 310. Thereafter, the CPU proceeds to step 725 to newly store registration position information, and proceeds to step 795 to temporarily end this routine.

[0108] If the OK image 326 has not been touched when the CPU proceeds to step 735, the CPU determines "No" in step 735 and proceeds to step 745. In step 745, the CPU determines whether or not the cancel image 328 has been touched in a state in which the registration image has not been selected on the selection screen 320.

[0109] If the cancel image 328 has not been touched, the CPU determines “No” in step 745 and returns to step 735 .

[0110] If the cancel image 328 is touched while the registration image is not selected, the CPU determines "Yes" in step 745 and proceeds to step 750. In step 750, the CPU ends the display of the selection screen 320 and displays the front mode screen 300. After that, the CPU proceeds to step 795 and temporarily ends this routine.

[0111] <Termination condition determination routine> The CPU executes a termination condition determination routine shown in the flowchart of FIG. 8 every time a predetermined time period elapses.

[0112] 8, and proceeds to step 805. In step 805, the CPU determines whether the value of any one of the first display flag Xdp1, the second display flag Xdp2, and the third display flag Xdp3 is "1."

[0113] If the values ​​of the first display flag Xdp1, the second display flag Xdp2, and the third display flag Xdp3 are all "0", the CPU determines "No" in step 805, proceeds to step 895, and temporarily ends this routine.

[0114] If any one of the first display flag Xdp1, the second display flag Xdp2, and the third display flag Xdp3 has a value of "1", the CPU determines "Yes" in step 805 and proceeds to step 810. In step 810, the CPU determines whether the vehicle speed Vs is greater than a first threshold vehicle speed Vsth1.

[0115] If the vehicle speed Vs is greater than the first threshold vehicle speed Vsth1, the CPU determines "Yes" in step 810 and proceeds to step 815. In step 815, the CPU sets the values ​​of the first display flag Xdp1, the second display flag Xdp2, and the third display flag Xdp3 to "0." After that, the CPU proceeds to step 895 and temporarily ends this routine.

[0116] If the vehicle speed Vs is equal to or less than the first threshold vehicle speed Vsth1, the CPU determines "No" in step 810, and proceeds to step 820. In step 820, the CPU determines whether the value of the second display flag Xdp2 is "1" or not.

[0117] If the value of the second display flag Xdp2 is "1", the CPU determines "Yes" in step 820 and proceeds to step 825. In step 825, the CPU determines whether or not the current position of the vehicle VA has been identified (i.e., whether or not the GNSS receiver 25 is receiving a positioning signal).

[0118] If the current position of the vehicle VA has been identified (that is, if the GNSS receiver 25 is receiving a positioning signal), the CPU determines "Yes" in step 825 and executes steps 830 and 835 in this order.

[0119] Step 830: The CPU sets the value of the lost flag Xlost to "0" and sets the lost travel distance D to "0".

[0120] The value of the lost flag Xlost is set to "1" when the current position of the vehicle VA cannot be identified, and is set to "0" when the current position of the vehicle VA can be identified. The CPU sets the value of the lost flag Xlost to "0" in the initial routine.

[0121] The lost travel distance D is the distance traveled by the vehicle VA while the current position of the vehicle VA cannot be identified.

[0122] Step 835: The CPU determines whether the vehicle VA is located outside the second range AR2. This second range AR2 is a range centered on the registered position used to establish the position condition when the value of the second display flag Xdp2 is set to "1."

[0123] If the vehicle VA is located outside the second range AR2, the CPU determines "Yes" in step 835 and proceeds to step 840. In step 840, the CPU sets the values ​​of the second display flag Xdp2 and the lost flag Xlost to "0", and sets the lost travel distance D to "0". After that, the CPU proceeds to step 895 and temporarily ends this routine.

[0124] If the vehicle VA is not located outside the second range AR2 (i.e., if the vehicle VA is located within the second range AR2), the CPU determines "No" in step 835, proceeds to step 895, and temporarily ends this routine.

[0125] On the other hand, if the current position of the vehicle VA cannot be determined when the CPU proceeds to step 825 (i.e., the GNSS receiver 25 is not receiving a positioning signal), the CPU judges "No" in step 825 and executes steps 845 and 850 in sequence.

[0126] Step 845: The CPU executes a lost travel distance acquisition subroutine. In practice, when the CPU proceeds to step 845, it executes a subroutine shown by the flowchart in Fig. 9. In this subroutine, the CPU obtains the lost travel distance D described above.

[0127] Step 850: The CPU determines whether the lost traveling distance D is equal to or greater than the threshold distance Dth.

[0128] If the lost traveling distance D is equal to or greater than the threshold distance Dth, the CPU judges "Yes" in step 850, proceeds to step 840, and sets the value of the second display flag Xdp2 to "0." After that, the CPU proceeds to step 895 and temporarily ends this routine.

[0129] If the lost traveling distance D is less than the threshold distance Dth, the CPU makes a "No" determination in step 850, proceeds to step 895, and temporarily ends this routine.

[0130] On the other hand, if the value of the second display flag Xdp2 is “0” when the CPU proceeds to step 820, the CPU makes a “No” determination in step 820 and proceeds to step 855.

[0131] In step 855, the CPU determines whether the value of the third display flag Xdp3 is "1" or not.

[0132] If the value of the third display flag Xdp3 is "1", the CPU determines "Yes" in step 855 and proceeds to step 860. In step 860, the CPU determines whether or not at least one of the front sonar 23FR and the rear sonar 23RR has detected an obstacle.

[0133] If at least one of the front sonar 23FR and the rear sonar 23RR has detected an obstacle, the CPU judges "Yes" in step 860, proceeds to step 895, and temporarily ends this routine.

[0134] If neither the front sonar 23FR nor the rear sonar 23RR has detected an obstacle, the CPU determines "No" in step 860 and proceeds to step 865. In step 865, the CPU sets the value of the third display flag Xdp3 to "0." After that, the CPU proceeds to step 895 and temporarily ends this routine.

[0135] On the other hand, if the value of the third display flag Xdp3 is "0" when the CPU proceeds to step 855, the CPU makes a "No" determination in step 855, proceeds to step 895, and temporarily ends this routine.

[0136] <Lost mid-distance acquisition subroutine> 8, the CPU starts the process from step 900 shown in Fig. 9, and proceeds to step 905. In step 905, the CPU determines whether the value of the lost flag Xlost is "0" or not.

[0137] If the value of the lost flag Xlost is "0", the CPU determines "Yes" in step 905 and proceeds to step 910. In step 910, the CPU sets the value of the lost flag Xlost to "1" and sets the lost travel distance D to "0". After that, the CPU proceeds to step 995 to temporarily end this routine, and proceeds to step 850 shown in FIG. 8.

[0138] On the other hand, if the value of the lost flag Xlost is "1", the CPU determines "No" in step 905 and executes steps 915 and 920 in this order.

[0139] Step 915: The CPU obtains the travel distance Da of the vehicle VA during the period from the previous execution of the termination condition determination routine shown in FIG. 8 to the current time based on the number of wheel pulse signals generated by the wheel speed sensor 21 during that period. Step 920: The CPU adds the above-mentioned traveling distance Da to the traveling distance D during the lost state. Thereafter, the CPU proceeds to step 995 to temporarily end this routine, and then proceeds to step 850 shown in FIG.

[0140] As described above, when the deletion condition that the registration number N is equal to or greater than the threshold value Nth is met, the periphery monitoring device 10 displays "the selection screen 320 including the image at the time of registration of the registration position information stored in the registration position information storage unit 312." The periphery monitoring device 10 deletes the registration position information corresponding to the image at the time of registration selected on the selection screen 320 from the registration position information storage unit 312.

[0141] This makes it easier for the driver to identify the location of the registered position information by visually checking the selection screen 320. Therefore, the driver can accurately select unnecessary registered position information to be deleted. Furthermore, this surroundings monitoring device 10 can be applied to a vehicle VA that does not have map data.

[0142] Furthermore, when the vehicle VA is located outside a second range AR2 that is wider than a first range AR1 centered on the registered position, the surroundings monitoring device 10 ends the display of the front mode screen 300. This makes it possible to reduce the possibility that the display of the front mode screen 300 will be repeatedly started and ended.

[0143] Furthermore, when the current position of the vehicle VA can no longer be identified (i.e., when the positioning signal can no longer be received), the surroundings monitoring device 10 ends the display of the front mode screen 300 when the travel distance (travel distance during loss) D of the vehicle VA from that point in time becomes equal to or greater than the threshold distance Dth. For example, when the vehicle VA is located in an underground parking lot or the like, it may become impossible to identify the current position of the vehicle VA. Even in such a case, the display of the front mode screen 300 can be appropriately ended.

[0144] The present invention is not limited to the above-described embodiment, and various modifications can be adopted within the scope of the present invention.

[0145] (First Modification) The front mode screen 300 may include at least one of a traveling direction image and a PV image. Similarly, the registration image may include at least one of a traveling direction image and a PV image.

[0146] (Second Modification) A delete switch (SW) image (not shown) is displayed on the display device 30. The delete SW image is touched when the driver wishes to delete registered location information. If the number of registrations N is equal to or greater than a threshold value Nth when the delete SW image is touched, the PVECU 20 may satisfy the delete condition and display a selection screen 320. As a result, the driver can select the registered location information to be deleted by touching the delete SW image.

[0147] (Third Modification) The first range AR1 and the second range AR2 may have a shape other than a circle, so long as the second range AR2 is wider than the first range AR1 and has the registration position as the center.

[0148] (Fourth Modification) If the periphery monitoring device 10 is capable of recognizing obstacles in the periphery of the vehicle VA based on the images captured by the cameras 22FR to 22R, it does not need to be equipped with the front sonar 23FR and the rear sonar 23RR.

[0149] The front sonar 23FR and the rear sonar 23RR may be remote sensing devices that can detect objects by transmitting a wireless medium other than sound waves and receiving the reflected wireless medium. Furthermore, the number of the front sonar 23FR and the rear sonar 23RR is not limited to four.

[0150] As long as the cameras 22FR to 22R are capable of capturing images of the area (peripheral area) around the vehicle VA (360 degrees), the number of the cameras 22FR to 22R is not limited to four.

[0151] (Fifth Modification) When the PVMECU20 of the fifth variant displays the front mode screen 300, if the vehicle speed Vs is equal to or less than the second threshold vehicle speed Vsth2, it displays the registered SW image 306, and if the vehicle speed Vs is greater than the second threshold vehicle speed Vsth2, it does not display the registered SW image 306.

[0152] (Sixth Modification) The PVMECU 20 according to the sixth modified example displays the registered SW image 306 on the front mode screen 300 regardless of the magnitude relationship between the vehicle speed Vs and the second threshold vehicle speed Vsth2. However, the PVMECU 20 determines whether the registered SW image 306 has been touched only when the vehicle speed Vs is equal to or less than the second threshold vehicle speed Vsth2, and if the registered SW image 306 has been touched, executes the processing from step 710 shown in FIG.

[0153] (Seventh Modification) In the above embodiment, when the shift position SP is a position other than the parking (P) position and the reverse (R) position, the PVMECU20 displays the front mode screen 300 including a traveling direction image generated based on a forward image in the traveling direction of the vehicle VA. However, when the shift position SP is the reverse (R) position, the PVMECU02 may display a rear mode screen including a traveling direction image generated based on a rear image in the traveling direction of the vehicle VA. The rear mode screen has the same configuration as the front mode screen 300, except that the traveling direction image is an image generated based on a rearward image.

[0154] (Eighth Modification) The surroundings monitoring device 10 is applicable not only to the above-mentioned engine vehicles, but also to hybrid electric vehicles (HEVs), plug-in hybrid electric vehicles (PHEVs), fuel cell electric vehicles (FCEVs), and battery electric vehicles (BEVs). [Explanation of symbols]

[0155] 10...periphery monitoring device, 20...PVM assistance ECU, 22FR...front camera, 22RR...rear camera, 22L...left camera, 22R...right camera, 30...display device, 300...front mode screen, 320...selection screen.

Claims

1. an imaging device configured to image an area surrounding the vehicle; a control unit configured to display, when a predetermined display condition is satisfied, a surrounding screen including a predetermined surrounding image generated based on an image captured by the imaging device on a display device mounted on the vehicle; Equipped with The control unit When a driver of the vehicle performs a registration operation for registering the current position of the vehicle as a registered position, the surrounding image generated based on an image captured by the imaging device at the current position is acquired as a registration time image, and registration position information that associates the registration time image with the current position is stored; When a position condition that the vehicle is located within a predetermined first range centered on the registered position is satisfied, the display condition is satisfied to newly generate the peripheral image, and the peripheral screen including the peripheral image is displayed on the display device; When a predetermined deletion condition is satisfied, a selection screen including an image at the time of registration of the stored registration position information is displayed on the display device; erase the registration position information of the registration time image selected by the driver on the selection screen; It is configured as follows: Furthermore, the control unit is configured to satisfy the display condition and newly generate the peripheral image and display the peripheral screen including the peripheral image on the display device, also when an obstacle condition is satisfied that an obstacle is detected around the vehicle, in addition to when the position condition is satisfied, The display device is a touch panel type display device, The control unit When the peripheral screen is displayed on the display device, a registered switch image is also displayed on the display device together with the peripheral screen; When the driver touches the registration switch image, it is assumed that the driver has performed the registration operation, and the registration time image is acquired and the registration position information is stored. It was configured as follows: Perimeter monitoring devices.

2. The surroundings monitoring device according to claim 1, the control unit is configured to, when the peripheral screen is displayed on the display device because the position condition is satisfied, terminate the display of the peripheral screen when the vehicle is located outside a second range that is set to be wider than the first range around the registered position. Perimeter monitoring devices.

3. The surroundings monitoring device according to claim 1 or 2, The control unit Identifying a current position of the vehicle based on positioning signals received from a plurality of artificial positioning satellites; if the current position of the vehicle cannot be identified while the peripheral screen is being displayed on the display device due to the position condition being satisfied, when a distance traveled by the vehicle from the point in time when the current position of the vehicle cannot be identified reaches or exceeds a predetermined threshold distance, the display of the peripheral screen is terminated. It was configured as follows: Perimeter monitoring devices.

4. The surroundings monitoring device according to any one of claims 1 to 3, The control unit acquiring at least one of an image in a traveling direction of the vehicle and an overhead image of a surrounding area of ​​the vehicle as the surrounding image; displaying the peripheral screen including the peripheral image and the selection screen on the display device; configured to Perimeter monitoring devices.

5. The surroundings monitoring device according to any one of claims 1 to 4, The control unit is configured to, when the driver performs the registration operation, and when the number of the stored registered position information is equal to or greater than a predetermined threshold, to display the selection screen on the display device by satisfying the deletion condition. Perimeter monitoring devices.

6. The surroundings monitoring device according to any one of claims 1 to 5, the control unit is configured to display the peripheral screen on the display device by satisfying the display condition when an operation condition is satisfied that the driver has performed a request operation to request display of the peripheral screen, in addition to when the position condition and the obstacle condition are satisfied. Perimeter monitoring devices.

7. In the peripheral monitoring device according to claim 1, The control unit When the vehicle speed is equal to or lower than a predetermined threshold vehicle speed, the driver is deemed to have performed the registration operation when the driver touches the registration switch image. When the vehicle speed is greater than the threshold vehicle speed, even if the driver touches the registration switch image, it is not considered that the driver has performed the registration operation. It was configured as follows: Perimeter monitoring devices.

Images