Parking assistance system
The parking assistance system addresses the issue of rear cargo protrusion by using a wide-angle camera to adjust guidelines and provide alerts, ensuring safe parking maneuvers.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- CANON KK
- Filing Date
- 2022-06-22
- Publication Date
- 2026-06-29
Smart Images
Figure 0007881389000001 
Figure 0007881389000002 
Figure 0007881389000003
Abstract
Description
Technical Field
[0003]
[0001] The present invention relates to a parking assistance system for a vehicle, and more particularly to a parking assistance system that presents a parking position.
Background Art
[0002] As a device for assisting a driver's reverse operation when parking a vehicle, there are a rear camera that images the rear of the vehicle and a video display device such as a rear monitor that presents the imaged rear video to the driver. By using this parking assistance device, when the driver switches the shift lever to the reverse position, the driver can visually recognize the video in the reverse direction through the rear monitor, and can perform a safer parking operation. In addition, there are parking assistance devices that not only display the video imaged by the rear camera as it is, but also have various functions added for safety reasons. For example, in order for the driver to easily grasp the distance between the rear end of the vehicle and the rear obstacle, there is a function of drawing guideline lines related to the vehicle width and the reverse distance on the rear monitor. In particular, the guideline line drawn at the position closest to the rear end of the vehicle body suggests that if the driver further performs a reverse operation beyond the guideline line, there is a possibility of contacting an obstacle, and it serves as a guide for parking. Further, in Patent Document 1, a parking assistance device is proposed that draws a guideline line at a position corresponding to the tip of the back door so that the driver can open and close it without difficulty when it is determined that there is a possibility of opening and closing the back door of the vehicle after parking. Thereby, it is possible to easily show the driver whether there is an obstacle that may come into contact within the range where the back door of the vehicle can be opened and closed.
Prior Art Documents
Patent Documents
[0003]
Patent Document 1
Summary of the Invention
Problems to be Solved by the Invention
[0005] It is conceivable that parking assist systems could be used for vehicles with cargo protruding from the rear, as described above. However, since the rear camera used for parking assist has a downward-facing field of view, it cannot capture images of cargo protruding from the rear. Therefore, if the driver reverses according to the guidelines displayed on the rear monitor, and the cargo protrudes from the rear, there is a risk that the vehicle may not be able to park in the correct position and may come into contact with an obstacle.
[0006] Therefore, the present invention aims to provide a parking assistance system that can perform a proper reversing maneuver when the load on the vehicle is protruding to the rear. [Means for solving the problem]
[0007] The parking assistance system according to the present invention is A rear imaging means having the function of imaging above and behind the vehicle, A detection means for detecting information about the cargo on the vehicle using images captured by the rear imaging means, A video display means that displays the video captured by the rear imaging means and presents parking assistance guidelines within the video, A guideline display control means controls the display position of the guideline in the video based on the information of the load detected by the detection means, It is characterized by having the following features. [Effects of the Invention]
[0008] According to the present invention, a parking assist system can be provided that can perform a proper reversing operation even when the load on the vehicle is protruding to the rear. [Brief explanation of the drawing]
[0009] [Figure 1] This figure shows an example of a vehicle, its cargo, and the installation configuration of a wide-angle rear camera. [Figure 2] This is a diagram showing an example configuration of a parking assistance system. [Figure 3] This figure shows an example of the rear monitor display during typical reverse operation when there is no cargo. [Figure 4] This figure shows an example of how to control the display position of the guideline when reversing with a load present. [Figure 5] This figure shows an example of how the loaded items are displayed during reverse operation. [Figure 6] This is a flowchart showing an example of parking assistance processing. [Figure 7] This flowchart shows an example of parking assistance processing when the load extends beyond the vehicle's surface. [Figure 8] This figure shows an example of a vehicle configuration used when calculating the amount of overhang of cargo on the vehicle. [Figure 9] This figure shows an example of the full-angle video format captured by a wide-angle rear camera. [Modes for carrying out the invention]
[0010] Hereinafter, embodiments of the parking assistance system of the present invention will be described in detail with reference to the drawings. In each figure, the same reference numeral is used for identical components, and redundant descriptions are omitted. Furthermore, the parking assistance system of the present invention can be embodied in various forms and is not limited to the embodiments described in this specification.
[0011] (1) Outline of the Embodiment When the engine of the vehicle is started, the parking support system activates a wide-angle rear camera to detect the presence of a load. When a load protruding beyond the rear of the vehicle is detected, the amount of protrusion is calculated. When the reverse operation of the vehicle is detected, the captured image of the load is displayed on the rear monitor for parking support, and the display position of the guideline is controlled based on the calculated amount of protrusion of the load.
[0012] Regarding the detection of the presence or absence of a load in this embodiment, during vehicle travel, when an object has been reflected at a predetermined position within the imaging angle of view of the wide-angle rear camera for a predetermined time or more, it is determined that a load is present. Also, the amount of protrusion of the load beyond the rear is calculated from the height distance between the rear camera and the load, and the angle-of-view information in which the load is imaged.
[0013] In this way, when the parking support system displays an image of the load on the vehicle when parking and controls the display position of the guideline according to the amount of protrusion of the load beyond the rear, the driver can grasp the sense of distance between the load and the rear obstacle and can park appropriately when reversing.
[0014] (2) Details of the embodiment FIG. 1 is a diagram showing an example of a vehicle according to an embodiment of the present invention, a load loaded on the vehicle, and an installation mode of a wide-angle rear camera. The wide-angle rear camera 100 is attached to the rear of the vehicle 10, for example, to the upper part of the rear bumper 12, and images the rear of the vehicle with a wide angle of view after the engine is started. The wide-angle rear camera 100 shown here uses, for example, one that can image a range of 150 degrees of vertical angle of view. Also, the vehicle 10 refers to, for example, a vehicle traveling on a public road such as a passenger car or a commercial vehicle (truck, bus, etc.), in which a driver 500 rides and can move to an arbitrary place. The rear monitor 101 is a video display device that presents the video imaged by the rear camera 100 to the driver when parking. Also, the load 11 refers to a long object that can be loaded on the vehicle roof or the loading platform and may protrude beyond the vehicle length. For example, wood, iron pipes, surfboards, etc. are included.
[0015] FIG. 2 is a diagram showing a configuration example of a parking support system. The processing unit 200 includes a camera imaging processing unit 210, a driving environment determination unit 220, and a reverse support control unit 230.
[0016] The processing unit 200 includes a CPU (Central Processing Unit) that performs operations and controls, and ROM (Read Only Memory) and RAM (Random Access Memory) which are main storage devices (both not shown in the figure). The ROM stores basic setting data and a series of parking support processing programs according to this embodiment. The CPU calls a program corresponding to the processing content from the ROM and expands it into the RAM to execute the operations of each block.
[0017] The camera imaging processing unit 210 includes an image processing unit 211 and a video recognition processing unit 212. The image processing unit 211 develops the image signal transmitted from the wide-angle rear camera 100 into a video, and performs processes such as WDR (Wide Dynamic Range) correction, gamma correction, distortion correction, and demosaicing. The video recognition processing unit 212 uses the image signal output from the image processing unit 211 to perform detection processing on the objects reflected in the wide-angle rear camera 100. Deep learning is used for object detection. For example, as deep learning, it is preferable to use YOLO (You Only Look Once) which is easy to learn and fast in detection. Also, as other deep learning, SSD (Single Shot MultiBox Detector) may be used. Additionally, Faster R-CNN (Regional Convolution Neural Network), Fast R-CNN, R-CNN, etc. may also be used. Object detection processing is performed using the above recognition technologies, and the recognition result is output to the reverse support control unit 230.
[0018] The driving environment determination unit 220 communicates with the in-vehicle ECU (Electronic Control Unit) via an interface such as CAN (Controller Area Network). This allows it to acquire driving information, such as whether the vehicle 10 is moving forward or backward. Means for acquiring the vehicle 10's driving information include, for example, the input status of the shift gear 103, the vehicle 10's speed, and the direction of tire rotation. If the driving environment determination unit 220 determines from the acquired driving information that the vehicle 10 is about to perform a reverse maneuver, it outputs the determination result to the reverse support control unit 230.
[0019] The reverse support control unit 230 consists of a load detection unit 231, an overhang calculation unit 232, a video display control unit 233, a guideline display control unit 234, and a warning unit 235. The load detection unit 231 determines whether or not there is any load that overhangs the rear of the vehicle among the detected objects, based on the object recognition results of the rear video output from the video recognition processing unit 212. The method for determining the load is described in detail in the flowchart of step S603 in Figure 6. The overhang calculation unit 232 calculates how far the load 11 overhangs the rear if the load detection unit 231 determines that there is a load 11 overhanging the rear of the vehicle. The method for calculating the load overhang is described in detail in the flowchart of step S605 in Figure 6. The presence or absence of the load 11 and the amount of rear overhang may also be determined based on manual operation by the driver 500. For example, if it is difficult to capture images of the upper rear due to constraints on the mounting position of the wide-angle rear camera 100, it is preferable to set the cargo information manually.
[0020] When the driving environment determination unit 220 determines that the vehicle 10 is about to reverse, the video display control unit 233 crops the lower part of the field of view of the image captured by the wide-angle rear camera 100 and displays the rear view image 101a for parking assistance on the rear monitor 101. The video display control unit 233 also draws guidelines 234a regarding the vehicle width and reversing distance of the vehicle 10 on the rear view image.
[0021] Figure 3 shows an example of guideline display during a typical reversing operation when there is no cargo. When the guideline display control unit 234 determines that cargo 11 is present, it controls the display position of the guideline 234a in the rear view image on the rear monitor 101 based on the calculated overhang amount.
[0022] Figure 4 shows an example of guideline display position control during reverse operation when there is cargo present. For example, if it is calculated that the cargo 11 on the vehicle extends 50 cm behind the rear end of the vehicle, the guideline 234a shown in Figure 3 is shifted 50 cm behind the normal display position and drawn as shown in guideline 234b in Figure 4.
[0023] By performing a reverse maneuver so that the distance guideline 234c does not come into contact with the rear obstacle 50, the vehicle 10 and its cargo 11 can be prevented from coming into contact with the rear obstacle 50. The guidelines 234a and 234b may be displayed in the predicted path direction of the vehicle 10 according to the steering angle, or they may be displayed in the straight-ahead direction of the vehicle 10 at the moment.
[0024] Figure 5 also shows an example of the display format of the wide-angle rear view image 101b including the cargo. As shown in Figure 5, the purpose of the video display control unit 233 is to make it easier for the driver 500 to identify the cargo 11 on the vehicle. For example, the wide-angle view image 101b including the cargo is displayed in parallel with the parking assistance image 101a, which is a portion of the image captured by the wide-angle rear camera 100 that is cropped downwards and shows the area behind the vehicle 10. If cargo is detected on the vehicle by the cargo detection unit 231, the above-described video display format is displayed as the default. Furthermore, the above-described video display format can be switched on or off at will by the driver 500 using the cargo image display switch 104, which will be described in detail in step S704 of Figure 7.
[0025] The alarm unit 235, when it is determined that there is cargo 11 on the vehicle and that the vehicle 10 is performing a reverse maneuver, will notify the driver 500 of the possibility of contact with the cargo via the voice speaker 102, such as "The cargo is protruding 50 cm to the rear. Please reverse carefully." The above-described alarm mode can be switched on or off at the driver's discretion using the alarm control switch 105, which is pressed by the driver 500, and will be described in detail in step S706 of Figure 7. The cargo image display switch 104 and the alarm control switch 105 may be soft switches that display an image on the rear monitor or physical switches installed near the rear monitor. Alternatively, text information such as "Please reverse carefully and be careful of contact with the cargo" may be displayed on the rear monitor 101 to notify the driver of the possibility of contact with the cargo.
[0026] The CPU inside the processing unit 200 reads and executes the parking assistance processing program stored in ROM, thereby performing computer processing for each block of the camera imaging processing unit 210, the driving environment determination unit 220, and the reverse assistance control unit 230. The procedure for the series of parking assistance processing steps will be described in detail according to the flowcharts in Figures 6 and 7.
[0027] Figure 6 is a flowchart showing an example of a parking assistance process S600 executed by the CPU inside the processing unit 200. In step S601, when the power source of the vehicle 10 (e.g., the engine) is started, the wide-angle rear camera 100 starts imaging the area behind the vehicle.
[0028] In step S602, the driving environment determination unit 220 determines whether the vehicle 10 is moving forward or not. Means for determining whether the vehicle is moving forward include whether the shift gear 103 is input to drive and whether the tires are rotating toward the front of the vehicle. If it is determined that the vehicle 10 is moving forward (branching to Yes in step S602), the process proceeds to step S603. On the other hand, if it is determined that the vehicle is not moving forward (branching to No in step S602), the process proceeds to step S606.
[0029] In step S603, the image recognition processing unit 212 outputs the recognition result of objects captured within the field of view of the wide-angle rear camera 100 to the cargo detection unit 231 and determines whether or not there is cargo protruding from the rear of the vehicle 10 among the detected group of objects. Generally, when the vehicle 10 is moving forward, the image captured by the wide-angle rear camera 100 is constantly changing in accordance with the surrounding scenery. Therefore, if frame differences are continuously taken for a certain period of time (for example, 30 seconds) or more while driving, and it is recognized that some object remains unchanged in the upper part of the field of view of the wide-angle rear camera 100, that object is determined to be cargo 11 on the vehicle. One example of recognition of cargo 11 is semantic segmentation using deep learning. By performing semantic segmentation, the cargo detection unit 231 can appropriately detect not only the presence or absence of cargo 11, but also the cargo region (shape) in the image. Furthermore, the blue sky can be a potential source of false detection of cargo, but this is determined by deep learning-based sky detection technology and object recognition processing for traffic lights, buildings, and other objects that obstruct the blue sky while driving. If it is determined using the above method that an object has been visible above the field of view of the wide-angle rear camera 100 for a certain period of time or longer (branching to Yes in step S603), the process proceeds to step S604. On the other hand, if no object is detected above the field of view (branching to No in step S603), the process proceeds to step S610. However, the above method for determining the presence or absence of cargo is only effective when driving forward, where the surrounding scenery is constantly changing. The method for detecting cargo when the vehicle 10 reverses without driving forward after the engine is started will be described in detail in step S608.
[0030] In step S604, the load detection unit 231 determines that the load 11 is protruding from the rear of the vehicle and outputs information of the field of view in which the load is being captured by the wide-angle camera 100 to the overhang amount calculation unit 232.
[0031] In step S605, the overhang calculation unit 232 calculates how much the load 11 protrudes from the rear of the vehicle. Figure 8 shows an example of the vehicle configuration when calculating the load overhang in this embodiment, and Equation 1 shows the calculation formula for the load overhang X in this embodiment. X = Y / tanθ ... Equation (1) X: Amount of overhang of the load 11 from the rear end of the vehicle Y: Height distance between wide-angle rear camera 100 and cargo 11 θ: Field of view information 100b in which the load 11 is being imaged. As shown in Equation 1, the amount X of the rear overhang of the load 11 is calculated from the height distance Y between the wide-angle rear camera 100 and the load 11, and the field of view information θ100b in which the load is imaged (the angle between the horizontal direction and the imaginary line connecting the rear imaging means and the load). As shown in Figure 8, when the load 11 overhangs approximately horizontally from the vehicle roof (when the rear end of the load is at the same height as the top of the vehicle roof), the height of the load 11 can be considered equivalent to the vehicle height. In this case, the height distance Y between the vehicle height and the wide-angle rear camera 100 is a value that has been registered in advance as vehicle information.
[0032] Figure 9 also shows an example of the full-angle image captured by the wide-angle rear camera 100. The angle of view information θ100b in which the cargo 11 is captured is obtained from the intersection of the grid line 100a of the captured angle of view and the edge position of the cargo 11 recognized by the image recognition processing unit 212, as shown in Figure 9. For example, if the height distance between the wide-angle rear camera 100 and the vehicle roof is Y=70cm, and the angle of view grid line 100b in which the cargo 11 is captured is θ=+60 degrees in the vertical direction, the overhang of the cargo 11 can be calculated to be approximately 40cm using Equation 1.
[0033] On the other hand, if the cargo 11 does not protrude approximately horizontally from the vehicle roof (for example, if a long item is loaded at an upward angle, or if the rear of the cargo is curved upward or downward), the height of the cargo 11 cannot be considered equivalent to the vehicle height. In other words, the amount of cargo 11 overhang is not calculated based on the image captured by the wide-angle rear camera 100. Therefore, in cases where it is determined that the amount of cargo 11 overhang should not be calculated based on the image captured by the wide-angle rear camera 100, it is necessary to detect the height of the cargo 11 from the ground and calculate the amount of overhang. Methods for detecting the height of the cargo 11 from the ground include distance measuring devices such as 3D-LiDAR (Light Detection And Ranging), ultrasonic sensors, monocular cameras, and stereo cameras (none of which are shown) installed at the rear of the vehicle. One method is to install these devices to measure distance and determine the height from the road surface to the cargo. Based on the amount of cargo overhang calculated by the above method, the system proceeds to the reverse support process S700 when the cargo overhangs. Whether the cargo 11 protrudes from the vehicle roof in a nearly horizontal manner or not can be determined, for example, by photographing the cargo with a wide-angle rear camera 100 and recognizing the captured image with a cargo detection unit 231. Alternatively, a 3D-LiDAR or ultrasonic sensor may be mounted on the rear of the vehicle 10 to measure the loading situation in three dimensions and determine whether the cargo 11 protrudes in a nearly horizontal manner. Furthermore, the user may input the loading situation from the rear monitor 11 to determine whether it is nearly horizontal or not.
[0034] In the above embodiment, the degree to which the cargo protruded was determined based on the height of the vehicle roof, but it may also be determined based on the height of the cargo bed on the vehicle roof.
[0035] In step S606, the driving environment determination unit 220 determines whether the vehicle 10 is attempting to reverse. Means for determining whether the vehicle is attempting to reverse include whether the shift gear 103 is input to reverse and whether the tires are rotating toward the rear of the vehicle. If the vehicle 10 is detected to be attempting to reverse (branching to Yes in step S606), the process proceeds to step S607. On the other hand, if the vehicle 10 is not detected to be attempting to reverse (branching to No in step S506), the process returns to step S502.
[0036] In step S607, the video display control unit 233 displays a rear view image 101a for parking assistance, with a downward cropped field of view, on the rear monitor 101, as shown in Figure 3. At the same time, as shown in Figure 4, a wide-angle rear view image 101b that allows for viewing upwards is also displayed in parallel.
[0037] In step S608, the cargo detection unit 231 requests information from the driver 500 regarding whether or not there is cargo on the vehicle. Methods for requesting cargo information include displaying a wide-angle rear view image 101b on the rear monitor 103 and asking the driver 500 to touch the area corresponding to the cargo, or requesting the driver to input an estimated amount of overhang of the cargo 11 as numerical information. When requesting cargo information from the driver 500, the request may be made via voice notification from the voice speaker 102 or displayed as text information on the rear monitor 101.
[0038] In step S609, the load detection unit 231 determines whether or not load information has been entered by the driver 500. If load information has been entered (branching to Yes in step S609), the process proceeds to step S604. On the other hand, if load information has not been entered (branching to No in step S609), the process proceeds to step S610.
[0039] In step S610, the load detection unit 231 determines that the load does not protrude beyond the rear of the vehicle, and when parking, it performs a general reverse assistance process as shown in Figure 3.
[0040] The above describes in detail the process of determining whether the cargo will protrude in steps S601 to S610. However, during this process, there is a possibility that the cargo 11 may be moved or shifted due to vibrations during driving. Therefore, the wide-angle rear camera 100 is kept constantly active during this process to calculate the amount of cargo 11 protruding while the vehicle is in motion. If there is a difference between the calculated value and the value calculated in the past, multi-threaded processing may be performed to update the amount of protrusion each time.
[0041] Next, the reverse support process when the load overhangs in step S700, shown in Figure 6, will be described in detail using Figure 7. In step S701, the driving environment determination unit 220 determines whether or not the vehicle 10 is attempting to reverse. The means for determining reverse movement is described in detail in step S506. If a reverse movement of the vehicle 10 is detected (branching to Yes in step S701), the process proceeds to step S702. On the other hand, if a reverse movement of the vehicle 10 is not detected (branching to No in step S701), the process in step S701 is repeated until a reverse movement is detected.
[0042] In step S702, the video display control unit 233 displays a rear view image 101a for parking assistance, with the field of view cropped downwards, on the rear monitor 101, as shown in Figure 3.
[0043] In step S703, the guideline display control unit 234 determines the amount of movement corresponding to the amount of rearward overhang X of the load 11 calculated by the overhang amount calculation unit 232. As a result, the display position of the guideline 234a drawn on the rear monitor 101 is shifted backward, as shown in Figures 3 and 4. The larger the rearward overhang X, the greater the amount of movement of the guideline 234a. This allows the driver 500 to determine whether or not there is a rearward obstacle 50 within the rearward overhang range of the load 11.
[0044] In step S704, the video display control unit 233 determines whether or not the cargo video display switch 104 is activated. If an input from the cargo video display switch 104 is detected (branching to Yes in step S704), the process proceeds to step S705. On the other hand, if no input from the cargo video display switch 104 is detected (branching to No in step S704), the process proceeds to step S706.
[0045] In step S705, the video display control unit 233 displays a wide-angle video 101b including the cargo 11 in parallel with the normal parking assistance video 101a, as shown in Figure 4. This allows the driver 500 to perform the reversing maneuver while visually confirming the cargo 11 on the vehicle, enabling safe parking.
[0046] In step S706, the alarm unit 235 determines whether or not the alarm control switch 105 is activated. If an input from the alarm control switch 105 is detected (branching to Yes in step S706), the process proceeds to step S707. On the other hand, if no input from the alarm control switch 105 is detected (branching to No in step S706), the process proceeds to step S708.
[0047] In step S707, the alarm unit 235 uses the voice speaker 102 to inform the driver 500 of the possibility of contact with the cargo. This allows the driver 500 to recognize the presence of the cargo 11 through voice information, even if the driver is reversing without checking the rear monitor 101, and enables safe parking.
[0048] In step S708, the driving environment determination unit 220 determines whether the vehicle 10 has finished its reverse operation. If the end of the reverse operation is detected (branching to Yes in step S708), the process proceeds to step S709. On the other hand, if the end of the reverse operation is not detected (branching to No in step S708), the process returns to step S704.
[0049] In step S709, the video display control unit 233 terminates the video display on the rear monitor 101. After step S709, the series of reverse support processes for when the load overhangs are completed, and the process is executed again from step S601 in preparation for the next reverse operation.
[0050] As described above, the parking assist system displays an image of the cargo 11 on the vehicle when parking and controls the position of the guideline display 234a according to the amount the cargo 11 overhangs from the rear. Therefore, the driver 500 can grasp the distance between the cargo 11 and the rear obstacle 50 and prevent contact with the rear obstacle 50 when reversing.
[0051] Although the present invention has been described in detail above based on its preferred embodiments, the present invention is not limited to these specific embodiments and can be modified in various ways without departing from the spirit of the invention.
[0052] For example, in this embodiment, a wide-angle rear camera 100 capable of capturing images above and behind the vehicle was used. However, in addition to the commonly used rear cameras with a fixed downward field of view, a camera with a fixed upward field of view may also be installed. In that case, when the engine is started, only the camera with the fixed upward field of view is activated (step S601 in Figure 6), and when reversing, both rear cameras are activated (steps S702 and S705 in Figure 7).
[0053] Furthermore, the parking assistance system of the present invention is not limited to the rear of the vehicle, but can also be applied to the sides and front of the vehicle. [Explanation of symbols]
[0054] 10 vehicles 11. Cargo 12 Rear bumper 50 Obstacles behind 100 Wide-angle rear camera 100a Wide-angle rear camera field of view grid lines 100b Field of view grid line θ where the edge of the load was detected 101 Rear Monitor 101a Rear view with the field of view cropped downwards. 101b Rear view with the field of view cropped upwards. 102 Voice Speakers 103 Shift Gear 104 Loaded object image display switch 105 Alarm control switch 200 Processing Units 210 Camera imaging processing unit 211 Image Processing Unit 212 Image Recognition Unit 220 Driving environment determination unit 230 Reverse Assist Control Unit 231 Load detection unit 232 Overhang calculation unit 233 Video Display Control Unit 234 Guideline Display Control Unit 234a Standard guideline display 234b Guideline display for when cargo overhangs 234c Distance guideline 235 Alarm section 500 drivers
Claims
1. A rear imaging means having the function of imaging above and behind the vehicle, A detection means for detecting information on how much the load on the vehicle protrudes from the vehicle to the rear using video footage captured by the rear imaging means, A video display means that displays the video captured by the rear imaging means and presents parking assistance guidelines within the video, A guideline display control means controls the display position of the guideline in the video based on the information of the load detected by the detection means, The system includes an overhang calculation means that calculates the angle between the horizontal direction and a virtual line connecting the rear imaging means and the cargo, based on the image captured by the rear imaging means, and calculates the amount of overhang of the cargo from the rear end of the vehicle to the rear based on the calculated angle. The guideline display control means determines the display position based on the overhang amount calculated by the overhang amount calculation means. The parking assistance system is characterized in that the overhang amount calculation means determines whether or not to calculate the overhang amount based on the image captured by the rear imaging means, according to the way the load overhangs from the vehicle to the rear as detected by the detection means.
2. It has distance measuring means for measuring the distance to an object behind the vehicle, The distance measuring means measures the distance from the distance measuring means to the load and the distance to the road surface when the overhang calculation means decides not to calculate the overhang amount based on the image captured by the rear imaging means. The overhang calculation means calculates the height distance from the road surface to the load based on the distance from the distance measuring means to the load and the distance to the road surface, and calculates the overhang amount based on the calculated height distance. The parking assistance system according to feature 1.
3. The detection means uses the image captured by the rear imaging means to detect how much the loaded material protrudes from the vehicle towards the rear. The parking assistance system according to feature 1 or 2.
4. The detection means detects how much the loaded material protrudes from the vehicle towards the rear based on input from the user. The parking assistance system according to feature 1 or 2.
5. The aforementioned video display means displays a parking assistance video, which is a portion of the video captured by the rear imaging means, cropped downwards to show the area behind the vehicle. The parking assistance system according to feature 1.
6. The parking assistance system according to claim 5, characterized in that the video display means displays, in addition to the display of the parking assistance video, a wide-angle video that includes at least a portion of the area of the parking assistance video and an area in which the load on the vehicle can be seen, in parallel.
7. The aforementioned overhang calculation means continuously calculates the overhang amount while the vehicle is in operation, and updates the overhang amount if there is a difference from the overhang amount calculated in the past. The parking assistance system according to feature 1.
8. Rear imaging means having the function of imaging above and behind the vehicle, A detection means for detecting information about the cargo on the vehicle using images captured by the rear imaging means, A video display means that displays the video captured by the rear imaging means and presents parking assistance guidelines within the video, A guideline display control means controls the display position of the guideline in the video based on the information of the load detected by the detection means, The vehicle has an environmental determination means for determining whether it is about to perform a reverse maneuver, If the detection means determines that the vehicle is about to perform a reverse maneuver, it determines whether there is any cargo on the vehicle based on user input to the video displayed by the video display means. A parking assistance system characterized by the following features.
9. A rear imaging process that captures images above and behind the vehicle, A detection step in which information on how much the load on the vehicle protrudes from the vehicle to the rear is detected using the image captured by the rear imaging step, A video display step which displays the video captured by the rear imaging step and presents parking assistance guidelines in the video, A guideline display control step controls the display position of the guideline in the video based on the information of the load detected in the detection step, Based on the image captured by the rear imaging step, the angle between the horizontal direction and a virtual line connecting the rear imaging step and the cargo is calculated, and based on the calculated angle, the amount of overhang of the cargo from the rear end of the vehicle to the rear is calculated in the overhang calculation step. It has, The guideline display control step determines the display position based on the overhang amount calculated in the overhang amount calculation step. The parking assistance method is characterized in that the overhang amount calculation step determines whether or not to calculate the overhang amount based on the image captured by the rear imaging step, according to the way the load overhangs from the vehicle to the rear as detected by the detection step.
10. A program that, when read and executed by a computer, causes the computer to perform each of the steps described in claim 9.