Control device, control method, and control program product

Abstract

The invention provides a control device, a control method, and a control program product capable of suppressing discomfort when a guide path image is displayed in a superimposed manner. The present invention is provided with: an external information acquisition unit (55) that acquires external information of a vehicle (10) that drags a trailer (60); and a display control unit (56) that displays a guide path image (71) indicating a guide path for the vehicle (10) and / or the trailer (60) in a superimposed manner on a peripheral image (5) of the vehicle (10) obtained on the basis of the outside information. The display control unit (56) varies the overlapping manner of the guide path image (71) between a first range (62) in the peripheral image (5) in which the trailer (60) is displayed and a second range (63) in the peripheral image (5) other than the first range (62).

Description

Technical Field

[0001] This invention relates to a control device, control method, and control program product. Background Technology

[0002] In recent years, efforts to promote the use of sustainable transportation systems that also take into account vulnerable members of the transportation population have become active. To achieve this goal, research and development related to autonomous driving technologies are being undertaken to further improve the safety and convenience of transportation.

[0003] Conventionally, techniques exist for overlaying guide lines indicating the predicted driving area of ​​a vehicle onto an image surrounding the vehicle, based on the vehicle's driving state, during driving / turning. For example, Patent Document 1 describes a parking assistance device comprising: an image processing unit that generates a surrounding image representing the surrounding conditions of the towing vehicle based on images captured by a camera mounted on a towing vehicle; a path acquisition unit that, when a target parking position for a towed vehicle connected to the towing vehicle is set, acquires a guide path extending from the target parking position to guide the towed vehicle to the target parking position; and a display processing unit that, upon acquiring the guide path, overlays the guide path onto the surrounding image, wherein the target parking position includes a pair of target positions corresponding to the width of the towed vehicle, and the display processing unit overlays a guide path extending from at least one of the target positions located inside the turning radius of the towing vehicle onto the surrounding image.

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 2019-069717 Summary of the Invention

[0007] The problem that the invention aims to solve

[0008] However, when a guide path is overlaid on the surrounding image, if the guide path is also overlaid on the image of the towed vehicle, it can reduce the visual distinguishability of the towed vehicle and the guide path for the driver, or cause discomfort due to the guide path being displayed on the towed vehicle. However, Patent Document 1 does not describe a guide path overlaid on the towed vehicle. Therefore, the method for displaying the guide path in the towing vehicle is open to improvement.

[0009] The purpose of this invention is to provide control devices, control methods, and control programs that can suppress discomfort when overlaying guideway images. This, in turn, contributes to the development of sustainable transportation systems.

[0010] Methods for solving problems

[0011] This invention is a control device, wherein,

[0012] The control device includes:

[0013] The external information acquisition unit acquires external information about the towing vehicle that is towing the towed vehicle; and

[0014] The display control unit overlays a guide path image showing the guide path of at least one of the tractor vehicle and the towed vehicle onto a surrounding image of the tractor vehicle obtained based on the external information.

[0015] The display control unit makes the overlapping method of the guide path image different in the first range of the surrounding image where the towed vehicle is displayed, and in the second range of the surrounding image other than the first range.

[0016] This invention is a control method, wherein,

[0017] The control method causes the processor of the control device to perform the following processes:

[0018] Obtain external information about the towing vehicle that is towing the towed vehicle;

[0019] A guide path image showing the guide path of at least one of the towing vehicle and the towed vehicle is overlaid on the surrounding image of the towing vehicle obtained based on the external information; and

[0020] The overlapping manner of the guide path images is different in the display of the first range of the towed vehicle in the surrounding images and in the second range of the surrounding images other than the first range.

[0021] This invention is a control program product, which includes a control program, wherein,

[0022] The control program causes the processor of the control device to perform the following processes:

[0023] Obtain external information about the towing vehicle that is towing the towed vehicle;

[0024] A guide path image showing the guide path of at least one of the towing vehicle and the towed vehicle is overlaid on the surrounding image of the towing vehicle obtained based on the external information; and

[0025] The overlapping manner of the guide path images is different in the display of the first range of the towed vehicle in the surrounding images and in the second range of the surrounding images other than the first range.

[0026] Invention Effects

[0027] According to the present invention, control devices, control methods, and control programs are available that can suppress discomfort when displaying overlaid guide path images. Attached Figure Description

[0028] Figure 1 This is a side view showing an example of a vehicle equipped with the control device of the present invention.

[0029] Figure 2 yes Figure 1 The vehicle shown is shown in a top view.

[0030] Figure 3 It means Figure 1 A block diagram illustrating an example of the internal structure of a vehicle.

[0031] Figure 4 This is a diagram showing an example of a guide path image displayed on the surrounding image of vehicle 10.

[0032] Figure 5 It means Figure 4 The image shown is an example of the generation of the subsequent image 5a.

[0033] Figure 6 This is a flowchart illustrating an example of the display processing of the rear image when vehicle 10 is reversing.

[0034] Figure 7 This is a flowchart illustrating a variation of the display processing of the rear image when vehicle 10 is reversing.

[0035] Figure 8 It means Figure 4 The diagram shows a deformed example of the generated rear image 5a.

[0036] Explanation of reference numerals in the attached figures

[0037] 5: Surrounding Images

[0038] 5a: Post-image

[0039] 5b: Overhead view

[0040] 10, 10a: Vehicles

[0041] 11L, 11R: Side rearview mirror

[0042] 12Fr: Front camera

[0043] 12Rr: Rear camera

[0044] 12L: Left-side camera

[0045] 12R: Right-side camera

[0046] 14: Operation Input Section

[0047] 16: Sensor group

[0048] 18: Navigation device

[0049] 20: Control ECU

[0050] 22: EPS System

[0051] 24: Ministry of Communications

[0052] 26: Drive force control system

[0053] 28: Braking force control system

[0054] 32a: Forward Sonar Group

[0055] 32b: Post-Sonar Group

[0056] 32c: Left sonar group

[0057] 32d: Right sonar group

[0058] 34a, 34b: Wheel sensors

[0059] 36: Vehicle speed sensor

[0060] 38: Operation and Inspection Department

[0061] 42: Touch panel

[0062] 44: Speaker

[0063] 50: Input / Output Section

[0064] 52: Arithmetic Unit

[0065] 54: Storage Department

[0066] 55: External Information Acquisition Department

[0067] 56: Display Control Unit

[0068] 60, 60a: Trailer

[0069] 61: Background

[0070] 62: First Range

[0071] 63: Second Range

[0072] 71, 71a, 71b: Guide path images

[0073] 81: Field of view

[0074] 82: Slider

[0075] 91: Post-external image

[0076] 92: Range Division Image

[0077] 93: Guide Image

[0078] 100: Steering angle sensor

[0079] 102: Torque sensor

[0080] 104: EPS motor

[0081] 106: Rotary Transformer

[0082] 108: EPS ECU

[0083] 110: Steering mechanism

[0084] 112: Steering column

[0085] 120: Communication device

[0086] 130: Drive ECU

[0087] 132: Braking ECU. Detailed Implementation

[0088] Hereinafter, one embodiment of the control device, control method, and control program product of the present invention will be described based on the accompanying drawings. Furthermore, the drawings are viewed along the directions indicated by the reference numerals. Additionally, in this specification and the like, for the sake of simplicity and clarity, the front-back, left-right, and up-down directions are arranged according to... Figure 1 and Figure 2 The directions observed from the driver's position of the vehicle 10 are recorded in the attached drawings. The front of the vehicle 10 is denoted as Fr, the rear as Rr, the left as L, the right as R, the top as U, and the bottom as D.

[0089] <Vehicle 10 equipped with the control device of the present invention>

[0090] Figure 1 This is a side view showing an example of a vehicle equipped with the control device of the present invention. Figure 2 yes Figure 1 The vehicle shown is a top view. Vehicle 10 is an example of the "tractor vehicle" of the present invention.

[0091] Vehicle 10 is an automobile having a drive source (not shown) and wheels including drive wheels driven by the power of the drive source and steering wheels capable of steering. In this embodiment, vehicle 10 is a four-wheeled automobile having a pair of front wheels on the left and right and a pair of rear wheels on the left and right. Alternatively, vehicle 10 may be an automobile capable of towing a trailer, for example. The drive source of vehicle 10 is, for example, an electric motor. Alternatively, the drive source of vehicle 10 may be an internal combustion engine such as a gasoline engine or a diesel engine, or a combination of an electric motor and an internal combustion engine. Furthermore, the drive source of vehicle 10 can drive the pair of front wheels on the left and right, the pair of rear wheels on the left and right, or all four wheels. The front wheels and rear wheels may both be steering wheels capable of steering, or only one of them may be a steering wheel capable of steering.

[0092] The vehicle 10 also includes side rearview mirrors 11L and 11R. The side rearview mirrors 11L and 11R are mirrors (rearview mirrors) located on the outside of the front doors of the vehicle 10, used to allow the driver to see behind and to the sides. The side rearview mirrors 11L and 11R are fixed to the main body of the vehicle 10 by a rotating shaft extending in a vertical direction, and can be opened and closed by rotating around the rotating shaft.

[0093] The vehicle 10 also includes a front camera 12Fr, a rear camera 12Rr, a left-side camera 12L, and a right-side camera 12R. The front camera 12Fr is a camera device (e.g., a digital camera) positioned in front of the vehicle 10 to capture images of the front of the vehicle 10. The rear camera 12Rr is a digital camera positioned behind the vehicle 10 to capture images of the rear of the vehicle 10. The left-side camera 12L is a digital camera positioned at the left-side rearview mirror 11L of the vehicle 10 to capture images of the left side of the vehicle 10. The right-side camera 12R is a digital camera positioned at the right-side rearview mirror 11R of the vehicle 10 to capture images of the right side of the vehicle 10.

[0094] <Internal structure of vehicle 10>

[0095] Figure 3 It means Figure 1 A block diagram illustrating an example of the internal structure of a vehicle. (As shown) Figure 3 As shown, vehicle 10 includes a sensor group 16, a navigation device 18, a control ECU (Electronic Control Unit) 20, an EPS (Electric Power Steering) system 22, and a communication unit 24. Additionally, vehicle 10 includes a drive force control system 26 and a braking force control system 28.

[0096] Sensor group 16 acquires various detection values ​​for controlling ECU 20. Sensor group 16 includes a front camera 12Fr, a rear camera 12Rr, a left-side camera 12L, and a right-side camera 12R. Furthermore, sensor group 16 includes a front sonar group 32a, a rear sonar group 32b, a left-side sonar group 32c, and a right-side sonar group 32d. Additionally, sensor group 16 includes wheel sensors 34a and 34b, a vehicle speed sensor 36, and an operation detection unit 38.

[0097] The front camera 12Fr, rear camera 12Rr, left camera 12L, and right camera 12R capture images of the external environment surrounding the vehicle 10, thereby obtaining external images used for identifying the vehicle 10 and its surroundings. The images of the vehicle 10's surroundings captured by the front camera 12Fr, rear camera 12Rr, left camera 12L, and right camera 12R are respectively referred to as the front image, rear image, left image, and right image. Alternatively, the image composed of the left and right images can be called a side image. The image of the vehicle 10 and its surroundings generated by combining the images captured by the front camera 12Fr, rear camera 12Rr, left camera 12L, and right camera 12R is called a top-down image of the vehicle 10.

[0098] The front sonar group 32a, rear sonar group 32b, left sonar group 32c, and right sonar group 32d emit sound waves to the periphery of the vehicle 10 and receive reflected sounds from other objects. The front sonar group 32a, for example, contains four sonars. The sonars constituting the front sonar group 32a are respectively positioned at the left diagonally forward, the left front, the right front, and the right diagonally forward of the vehicle 10. The rear sonar group 32b, for example, contains four sonars. The sonars constituting the rear sonar group 32b are respectively positioned at the left diagonally rear, the left rear, the right rear, and the right diagonally rear of the vehicle 10. The left sonar group 32c, for example, contains two sonars. The sonars constituting the left sonar group 32c are respectively positioned at the front left side and the rear left side of the vehicle 10. The right sonar group 32d, for example, contains two sonars. The sonars constituting the right sonar group 32d are respectively positioned at the front right side and the rear right side of the vehicle 10.

[0099] Wheel sensors 34a and 34b detect the rotation angle of the wheels of vehicle 10. Wheel sensors 34a and 34b can be composed of angle sensors or displacement sensors. Wheel sensors 34a and 34b output detection pulses every time the wheel rotates a predetermined angle. The detection pulses output from wheel sensors 34a and 34b are used to calculate the rotation angle and rotational speed of the wheels. Based on the rotation angle of the wheels, the distance traveled by vehicle 10 is calculated. Wheel sensor 34a, for example, detects the rotation angle θa of the left rear wheel. Wheel sensor 34b, for example, detects the rotation angle θb of the right rear wheel.

[0100] Vehicle speed sensor 36 detects the speed of the vehicle body 10, i.e., vehicle speed V, and outputs the detected vehicle speed V to control ECU 20. Vehicle speed sensor 36 detects vehicle speed V, for example, based on the rotation of the transmission countershaft. In addition, the sensor group 16 may also include, for example, object detection sensors capable of detecting objects around the vehicle 10. Objects include trailers connected to (towed by) the vehicle 10, other vehicles present in the vicinity of the vehicle, etc. Object detection sensors include, for example, radar, LiDAR (light detection and ranging or laser imaging detection and ranging), sonar, yaw rate sensor, magnetic sensor, and steering angle sensor.

[0101] The operation detection unit 38 detects the user's operation performed using the operation input unit 14 and outputs the detected operation to the control ECU 20. The operation input unit 14 includes various user interfaces such as a side mirror switch for switching the opening and closing states of the side mirrors 11L and 11R, and a gear shift lever (selector lever, selector).

[0102] The navigation device 18 uses, for example, GPS (Global Positioning System) to detect the current location of the vehicle 10 and guides the user to the destination. The navigation device 18 has a storage device (not shown) containing a map information database.

[0103] The navigation device 18 includes a touch panel 42 and a speaker 44. The touch panel 42 functions as an input device and display device for controlling the ECU 20. The speaker 44 outputs various guidance information to the user of the vehicle 10 through sound.

[0104] The touch panel 42 is configured to input various commands to the control ECU 20. For example, the user can input commands related to the mobility assistance of the vehicle 10 via the touch panel 42. Mobility assistance includes parking assistance and exit assistance for the vehicle 10. Furthermore, the touch panel 42 is configured to display various screens related to the control content of the control ECU 20. For example, screens related to the mobility assistance of the vehicle 10 are displayed on the touch panel 42. Specifically, the touch panel 42 displays a parking assistance button for requesting parking assistance for the vehicle 10 and an exit assistance button for requesting exit assistance. The parking assistance buttons include an automatic parking button for requesting parking based on the automatic steering of the control ECU 20 and an auxiliary parking button for requesting assistance when parking via user operation. The exit assistance buttons include an automatic exit button for requesting exiting based on the automatic steering of the control ECU 20 and an auxiliary exit button for requesting assistance when exiting via user operation. Additionally, for example, a top-down image generated by combining images captured by the front camera 12Fr, rear camera 12Rr, left camera 12L, and right camera 12R can be displayed on the touch panel 42. Alternatively, components other than the touch panel 42, such as smartphones or tablets, can be used as input or display devices.

[0105] The control ECU 20 includes an input / output unit 50, an arithmetic unit 52, and a storage unit 54. The arithmetic unit 52 is, for example, a CPU (Central Processing Unit). The arithmetic unit 52 controls each component based on a program stored in the storage unit 54, thereby performing various controls. Furthermore, the arithmetic unit 52 inputs and outputs signals to each component connected to the control ECU 20 via the input / output unit 50. The control ECU 20 is an example of the "control device" of the present invention.

[0106] The arithmetic unit 52 includes an external information acquisition unit 55 and a display control unit 56.

[0107] The external information acquisition unit 55 acquires external images of the vehicle 10 captured by the front camera 12Fr, rear camera 12Rr, left camera 12L, and right camera 12R. Based on the external images acquired from each camera, the external information acquisition unit 55 obtains external information about the vehicle 10 towing the trailer. This "external information about the vehicle 10 towing the trailer" also includes information about the trailer obtained from the images captured on the trailer.

[0108] The display control unit 56 displays the surrounding images of the vehicle 10 based on the external information acquired by the external information acquisition unit 55. "Surrounding images" include overhead views, rear views, left-side views, and right-side views. The surrounding images may also include not only surrounding images but also images of the vehicle 10 (e.g., an image of the vehicle 10 in an overhead view). Furthermore, the surrounding images can be generated by combining multiple pieces of external information.

[0109] The display control unit 56 overlays a guide path image showing the guide path of at least one of the vehicle 10 and the trailer towed by the vehicle 10 onto the surrounding image of the vehicle 10. The guide path is the path from the current position of at least one of the vehicle 10 and the trailer to the target parking position of at least one of the vehicle 10 and the trailer. The path from the current position to the target parking position can be, for example, from the current position of the vehicle 10 to the target parking position of the vehicle 10, or from the current position of the trailer to the target parking position of the trailer, or from the current position of both the vehicle 10 and the trailer to the target parking position of both the vehicle 10 and the trailer. The "target parking position" can be configured to be user-defined, for example, a parking position set by the user. Alternatively, the "target parking position" can also be a parking position set by the vehicle 10 automatically searching for an available parking position (target parking position).

[0110] The guide path is calculated based on the trailer's specifications and the relative angle between vehicle 10 and the trailer. Trailer specifications include, for example, the trailer's shape (including size) and the position of its wheels. The relative angle refers to the engagement angle, such as the trailer ball, that connects vehicle 10 and the trailer. The guide path can also be calculated taking into account the direction of travel of the vehicle 10 towing the trailer. The direction of travel refers to the desired direction in which the vehicle 10 towing the trailer is to travel, for example, a direction that the user can select within the range of -30 degrees to +30 degrees. For example, the guide path is the path when the vehicle 10 towing the trailer is in reverse gear and moving backward.

[0111] In the surrounding image of vehicle 10, the display control unit 56 displays the area containing the towed trailer as the first range of the surrounding image, and the area excluding the first range as the second range of the surrounding image. The display control unit 56 displays the guide path images in different overlapping ways in the first and second ranges. "Overlapping way" includes, for example, a non-overlapping way, an overlapping way with a transparency of 100% or less, etc. The display control unit 56 overlaps the guide path image in the first range with a way that the visual recognizability of the guide path image in the first range is lower than that of the guide path image in the second range. "With a way that is lower than..." means, for example, setting the guide path image in the first range to a non-overlapping way, or overlapping the guide path image displayed in the first range with a higher transparency compared to the guide path image displayed in the second range, etc. Alternatively, for example, the guide path image in the first range may be overlaid behind the surrounding image (an image with 0% transparency). The display control unit 56 may also not display the guide path image in the first range.

[0112] The display control unit 56 designates a first range based on the amount of change in each region of the surrounding image. "Amount of change in each region" can be, for example, the amount of change in each pixel, or the amount of change in multiple blocks (multiple pixels within a block) after dividing the surrounding image. "Amount of change" refers to the amount of change in a pixel value per unit time. The display control unit 56, for example, designates areas where the amount of change per unit time is below a predetermined value as the first range where the trailer is displayed. The display control unit 56 determines areas where the amount of change per unit time is greater than the predetermined value as displays other than the trailer, such as displays reflecting the background around the trailer, and designates them as a second range other than the first range. The display control unit 56 repeatedly calculates the "amount of change" as the vehicle 10 moves and updates the first range based on the calculation results.

[0113] Additionally, the display control unit 56 specifies a first range based on the relative angle between the vehicle 10 and the trailer. The "relative angle" can be detected, for example, by a sensor (various sensors such as mechanical or optical sensors) capable of detecting the engagement angle. The "relative angle" can also be detected, for example, by image recognition of the connecting part (trailer ball) located at a fixed position within the peripheral image.

[0114] In vehicle 10, there is corresponding information between the relative angle between vehicle 10 and trailer, and a first range of trailer display at that relative angle; that is, information about the first range can be derived based on the relative angle. The corresponding information can be prepared as a table, or as a trained model that takes the relative angle as input and the first range as output. The corresponding information is generated, for example, by using sensors to detect the engagement angle of vehicle 10 during travel, while also specifying the first range based on the changes in various regions of the surrounding image. The travel of vehicle 10 can be either actual travel or travel for calibration purposes. In addition to the relative angle, the input information for the corresponding information may also include trailer specification information.

[0115] The display control unit 56, for example, repeatedly specifies a first range displaying the trailer and a second range other than the first range while the vehicle 10 is in motion, and updates it to the new range.

[0116] In addition, the computing unit 52 performs automatic parking assistance and automatic exit assistance for the vehicle 10 based on automatic steering, which automatically operates the steering device 110 under the control of the computing unit 52. In automatic parking assistance and automatic exit assistance, the accelerator pedal (not shown), brake pedal (not shown), and operation input unit 14 are operated automatically. Furthermore, the computing unit 52 performs parking support assistance and exit support assistance when the user (driver) manually stops and exits the vehicle 10 by operating the accelerator pedal, brake pedal, and operation input unit 14. Moreover, during automatic parking assistance and automatic exit assistance, the user can be either riding in the vehicle 10 or exiting the vehicle 10 and being outside.

[0117] For example, the arithmetic unit 52 performs motion control to move the vehicle 10 based on the external images of the vehicle 10 obtained by the front camera 12Fr, rear camera 12Rr, left camera 12L, and right camera 12R, and the parking space specified by the user. Motion control includes parking control, which automatically parks the vehicle 10 to the designated parking space (target parking position), and exit control, which automatically moves the vehicle 10 from the designated parking space to the target moving position. The arithmetic unit 52 can execute parking control and exit control according to instruction signals input from the outside via the input / output unit 50. Inputs from the outside include wireless communication-based inputs from information terminals or the like that carried by the user of the vehicle 10. Furthermore, the arithmetic unit 52 can send information related to parking control and exit control to an external information terminal via the input / output unit 50.

[0118] The EPS system 22 includes a steering angle sensor 100, a torque sensor 102, an EPS motor 104, a rotary transformer 106, and an EPS ECU 108. The steering angle sensor 100 detects the steering angle θst of the steering unit 110. The torque sensor 102 detects the torque TQ applied to the steering unit 110.

[0119] The EPS motor 104 can assist the occupant in operating the steering device 110 and perform automatic steering during parking assistance by applying driving or reaction force to the steering column 112 connected to the steering device 110. The rotary transformer 106 detects the rotation angle θm of the EPS motor 104. The EPS ECU 108 is responsible for the overall control of the EPS system 22. The EPS ECU 108 includes an input / output unit (not shown), an arithmetic unit (not shown), and a storage unit (not shown).

[0120] The communication unit 24 is capable of wireless communication with other communication devices 120. These other communication devices 120 include base stations, communication devices in other vehicles, and information terminals such as smartphones or tablets that can be carried by the user of vehicle 10. For example, the communication unit 24 is equipped with a UWB interface for UWB (Ultra Wide Band) communication with information terminals.

[0121] The drive force control system 26 includes a drive ECU 130. The drive force control system 26 controls the drive force of the vehicle 10. The drive ECU 130 controls the engine (not shown) and other components based on user operations on the accelerator pedal (not shown), thereby controlling the drive force of the vehicle 10.

[0122] The braking force control system 28 includes a braking ECU 132. The braking force control system 28 controls the braking force of the vehicle 10. The braking ECU 132 controls the braking mechanism (not shown) and other components based on the user's operation of the brake pedal (not shown), thereby controlling the braking force of the vehicle 10.

[0123] <Example of displaying a guide path image>

[0124] Figure 4 This is an example of a guide path image displayed in the surrounding image of vehicle 10. For example... Figure 4 As shown, the surrounding images 5 of vehicle 10 are displayed side-by-side, for example, a rear image 5a and a top-down image 5b. When the gear of vehicle 10 is set to reverse, both the rear image 5a and the top-down image 5b are displayed. The rear image 5a and the top-down image 5b are displayed, for example, on the touch panel 42 of the navigation device 18. In this example, both the rear image 5a and the top-down image 5b are displayed, but only the rear image 5a may also be displayed.

[0125] The rear image 5a is, for example, an image captured by the rear camera 12Rr of vehicle 10. Overlaid on the rear image 5a is an image of the trailer 60 towed by vehicle 10 and a guide path image 71 showing the guide path of vehicle 10 and trailer 60. Trailer 60 is an example of the "towed vehicle" of the present invention. The guide path is, for example, a path that guides vehicle 10 and trailer 60 to a target parking position.

[0126] In the rear image 5a, the guide path image 71 is displayed without overlap in the area where the trailer 60 is displayed (first range), and overlaps in the area other than the image of the trailer 60 (second range). In this example, in the rear image 5a, the guide path image 71 overlaps in the area between the vehicle 10 and the trailer 60. Additionally, the rear image 5a displays message images for the user, such as "Please reverse the vehicle at low speed" or "Please check the surroundings directly."

[0127] The overhead view 5b is, for example, an image captured by the front camera 12Fr, the rear camera 12Rr, the left camera 12L, and the right camera 12R. The overhead view 5b shows a vehicle 10 towing a trailer 60. The vehicle displayed in the overhead view 5b is an "image of vehicle 10" generated by the display control unit 56. In the following description, the "image of vehicle 10" in the overhead view 5b will be simply referred to as vehicle 10a. Similarly, the trailer 60 displayed in the overhead view 5b is also an "image of trailer 60" generated by the display control unit 56. In the following description, the "image of trailer 60" in the overhead view 5b will be simply referred to as trailer 60a.

[0128] Additionally, a guide path image 71 showing the guide path of vehicle 10a and trailer 60a, and a field of view 81 of the rear image captured by the rear camera 12Rr of vehicle 10a are overlaid on the overhead view 5b. Furthermore, a slider 82 for indicating the reversing direction of vehicle 10a towing trailer 60a is displayed in the overhead view 5b. By sliding the slider 82 in the direction of the arrow, the reversing direction of vehicle 10a towing trailer 60a can be indicated within, for example, a range of 30 degrees to the left rearward and 30 degrees to the right rearward.

[0129] <Example of image generation>

[0130] Figure 5 It means Figure 4 The image shown is an example of the generation of the subsequent image 5a. Figure 5 As shown, the display control unit 56 of vehicle 10 generates a rear external image 91 of vehicle 10 based on the external image captured by the rear camera 12Rr. The rear external image 91 includes an image of the trailer 60 towed by vehicle 10, an image of the background 61 of vehicle 10, etc.

[0131] Additionally, the display control unit 56 generates a range division image 92 based on the rear external image 91, which divides a first range 62 and a second range 63. The first range 62 shows the area where the trailer 60 is displayed, and the second range 63 shows the area in the rear external image 91 other than the first range 62.

[0132] Additionally, the display control unit 56 generates a guidance image 93 representing a guidance path image 71 for the vehicle 10 towing the trailer 60. The guidance path image 71 is an image guiding the direction of reversing from the current position of the vehicle 10 to the target parking position.

[0133] The display control unit 56 generates a rear image 5a that overlaps with the guide path image 71 based on the generated rear external image 91, the range division image 92, and the guide image 93. Figure 5 The rear image 5a shown is... Figure 4 The rear image 5a shown is also called the guide path image 71, which is displayed only in the area other than the image of trailer 60 (second range) and not in the area where trailer 60 is displayed (first range).

[0134] <Post-image display processing>

[0135] Figure 6 This is a flowchart illustrating an example of the display processing of the rear image when vehicle 10 is reversing.

[0136] The display control unit 56 determines whether to display the rear image of the vehicle 10, acquired by the external information acquisition unit 55, on the touch panel 42 of the navigation device 18 (step S11). For example, the rear image is displayed when the vehicle 10 is in reverse gear. The display control unit 56 detects whether the gear is in reverse gear and determines whether to display the rear image. Alternatively, the display control unit 56 may also determine whether to display the rear image based on whether the navigation mode of the navigation device 18 is enabled.

[0137] In step S11, if the rear image is not displayed, i.e., the gear is not in reverse (step S11: No), the display control unit 56 repeatedly performs the processing of step S11. In step S11, if the rear image is displayed, i.e., the gear is in reverse (step S11: Yes), the display control unit 56 displays the rear image on the touch panel 42 (step S12).

[0138] Next, the display control unit 56 detects the amount of change in each region of the subsequent image and designates a first range in which the trailer 60 is displayed based on the detection results (step S13). For example, the display control unit 56 detects regions where the amount of change of each pixel per unit time is below a predetermined value and designates the detected regions as the first range in which the trailer 60 is displayed. In addition, the display control unit 56 designates regions in the subsequent image other than the first range as a second range.

[0139] Next, the display control unit 56 displays a guide path image showing the guide path of the vehicle 10 and the trailer 60 only in a second range other than the first range specified in step S13 (step S14).

[0140] The display control unit 56 determines whether to not display the rear image of the vehicle 10 displayed on the touch panel 42, i.e. whether the gear is in a non-reverse position (step S15).

[0141] In step S15, if the subsequent image is not not displayed (step S15: No), the display control unit 56 returns to step S13 and performs each process. In step S15, if the subsequent image is not displayed (step S15: Yes), the display control unit 56 does not display the subsequent image on the touch panel 42 (step S16).

[0142] In addition, the generation process of the overlapping guide path image is performed separately from this process.

[0143] Figure 7 This is a flowchart illustrating a variation of the display processing of the rear image when vehicle 10 is reversing. For example... Figure 7 As shown, the processing of steps S11 to S12 is similar to... Figure 6 The processes described in steps S11 to S12 are the same.

[0144] Next, the display control unit 56 designates a first range where the trailer 60 is displayed based on the attachment angle of the trailer ball connecting the vehicle 10 and the trailer 60, i.e., the relative angle between the vehicle 10 and the trailer 60 (step S13A). The display control unit 56 detects the attachment angle using a sensor, and for example, reads the first range corresponding to the detected attachment angle from a corresponding information table and designates the first range. In addition, the display control unit 56 designates the area in the subsequent image other than the first range as a second range.

[0145] Furthermore, the processing in steps S14 to S16 is similar to... Figure 6 The processes described in steps S14 to S16 are the same.

[0146] <Example of image generation>

[0147] Figure 8 It means Figure 4The diagram shows a deformed example of the generated rear image 5a. (See figure.) Figure 8 As shown, the control unit 56 generates the external image 91, the range division image 92, and the guide image 93, and controls them accordingly. Figure 5 The generation example of the subsequent image 5a is the same as that described in the text.

[0148] The display control unit 56 generates a rear image 5a that overlaps the generated rear external image 91, the range division image 92, and the guide image 93, displaying the guide path images 71a and 71b. Figure 8 In the rear image 5a shown, the guide path image 71a is displayed in the area other than the image of trailer 60 (the second range), which is consistent with the previous image. Figure 5 In the generation example, the guide path image 71 is displayed in the second range in the same way. On the other hand, in the area where the trailer 60 is displayed (the first range), a guide path image 71b with lower visual recognizability compared to the guide path image 71a in the second range is displayed, which is different from the case where no guide path image is displayed in the first range. Figure 5 The generated examples are different. The display control unit 56 displays, for example, a guide path image 71b with a dashed border in the first area where the trailer 60 is displayed.

[0149] Thus, in embodiments of the present invention, when the control device (control ECU 20) overlays a guide path image 71 guiding the reversing of the vehicle 10 and trailer 60 in the rear image 5a of the vehicle 10 towing the trailer 60, it displays the guide path image 71 overlaid on the first range showing the trailer 60 and the guide path image 71 overlaid on the second range other than the first range in the rear image 5a in different overlay patterns. According to this structure, for example... Figure 4 As shown, the user can easily grasp that the guide path image 71, which is superimposed on the area between the towing vehicle 10 and the towed trailer 60 (the area of ​​the second range), also continuously extends to the area below the trailer 60. Therefore, when the user observes the rear image 5a with the guide path image 71 superimposed, the rear image 5a can be observed without discomfort.

[0150] Furthermore, according to the control device, the guide path image 71(71b) is superimposed on the first range in such a way that it has lower visual recognizability than the guide path image 71(71a) superimposed on the first range where the trailer 60 is displayed. Thus, the user can distinguish and identify the guide path image 71(71a) superimposed on the second range and the guide path image 71(71b) that extends continuously from the guide path image 71(71a) to the area below the trailer 60 without discomfort.

[0151] Furthermore, according to the control device, a first range is specified based on the amount of change in pixel values ​​in each region of the subsequent image 5a. With this structure, regions with small changes can be specified as areas displaying the trailer 60 (first range), and regions with large changes can be specified as areas reflecting the background surrounding the trailer 60 (second range). Thus, the first range can be correctly specified, and the guide path image 71 can be appropriately overlaid and displayed.

[0152] Furthermore, according to the control device, a first range is specified based on the relative angle between the vehicle 10 and the trailer 60 towed by the vehicle 10. Based on this structure, by detecting the relative angle (attachment angle) of the trailer relative to the vehicle 10 and reading the corresponding information of the first range corresponding to that attachment angle from a table, the range in the subsequent image 5a showing the trailer 60 can be easily specified.

[0153] Furthermore, the control method described in the foregoing embodiments can be implemented by a computer executing a pre-prepared control program. This control program is recorded in a computer-readable storage medium and executed by reading it from the storage medium. Additionally, this control program can be provided in the form of storage on a non-transitory storage medium such as flash memory, or via a network such as the Internet. The computer executing this control program can be included in a control device, or in an electronic device such as a smartphone, tablet computer, or personal computer capable of communicating with the control device, or in a server device capable of communicating with these control devices and electronic devices.

[0154] The embodiments of the present invention have been described above, but the present invention is not limited to the above embodiments and can be appropriately modified or improved.

[0155] In addition, at least the following matters are described in this specification. Furthermore, although the corresponding components and the like are shown in parentheses in the above embodiments, the present invention is not limited thereto.

[0156] (1) A control device (control ECU 20), wherein,

[0157] The control device includes:

[0158] The external information acquisition unit (external information acquisition unit 55) acquires external information about the towing vehicle (vehicle 10) that is towing the towed vehicle (trailer 60); and

[0159] The display control unit (display control unit 56) overlays a guide path image (guide path image 71) showing the guide path of at least one of the tractor vehicle and the towed vehicle on the surrounding image (surrounding image 5) of the tractor vehicle obtained based on the external information.

[0160] The display control unit makes the overlapping method of the guide path image different in the first range (first range 62) of the surrounding image where the towed vehicle is displayed, and in the second range (second range 63) of the surrounding image other than the first range.

[0161] According to (1), by making the display mode of the guide path image that overlaps the first range where the towed vehicle is displayed different from the display mode of the guide path image that overlaps the other ranges, the discomfort when displaying the guide path image in an overlapping manner can be suppressed.

[0162] (2) The control device according to (1), wherein,

[0163] The display control unit overlaps the guide path image in the first range in such a way that the visual recognizability of the guide path image in the first range is lower than that of the guide path image in the second range.

[0164] As in (2), by reducing the visual recognizability of the guide path image that overlaps with the first range of the towed vehicle, the discomfort of the overlapping guide path image can be suppressed.

[0165] (3) The control device according to (2), wherein,

[0166] The display control unit does not display the guide path image within the first range.

[0167] As in (3), by not displaying the guide path image that overlaps with the first range of the towed vehicle, the discomfort caused by the overlapping guide path image can be suppressed.

[0168] (4) The control device according to any one of (1) to (3), wherein,

[0169] The guiding path is the path from the current position of at least one of the towing vehicle and the towed vehicle to the target parking position of at least one of the towing vehicle and the towed vehicle.

[0170] As in (4), the preferred guide path, which is displayed in conjunction with the surrounding images, is the path from the current position of the towing vehicle and the towed vehicle to the target parking position.

[0171] (5) The control device according to any one of (1) to (4), wherein,

[0172] The guiding path is a path obtained based on the specifications of the towed vehicle and the relative angle between the towing vehicle and the towed vehicle.

[0173] According to (5), by setting a guide path based on the specifications of the towed vehicle and the relative angle between the towing vehicle and the towed vehicle, an appropriate guide path can be generated.

[0174] (6) The control device according to any one of (1) to (5), wherein,

[0175] The display control unit specifies the first range based on the amount of change in each region of the surrounding image.

[0176] According to (6), by specifying the first range where the towed vehicle is displayed based on the amount of change in each region in the surrounding image, the correct first range can be specified.

[0177] (7) The control device according to any one of (1) to (5), wherein,

[0178] The display control unit specifies the first range based on the relative angle between the tractor vehicle and the towed vehicle.

[0179] According to (7), by specifying the first range in which the towing vehicle is displayed based on the relative angle between the towing vehicle and the towing vehicle, the correct first range can be specified.

[0180] (8) The control device according to any one of (1) to (7), wherein,

[0181] The display control unit repeatedly updates the first range and the second range.

[0182] As in (8), by repeatedly updating the first and second ranges, the guide path images can be appropriately overlapped even during traction movement.

[0183] (9) A control method, wherein,

[0184] The control method causes the processor of the control device to perform the following processes:

[0185] Obtain external information about the towing vehicle that is towing the towed vehicle;

[0186] A guide path image showing the guide path of at least one of the towing vehicle and the towed vehicle is overlaid on the surrounding image of the towing vehicle obtained based on the external information; and

[0187] The overlapping manner of the guide path images is different in the display of the first range of the towed vehicle in the surrounding images and in the second range of the surrounding images other than the first range.

[0188] According to (9), by making the display mode of the guide path image superimposed on the first range where the towed vehicle is displayed different from the display mode of the guide path image superimposed on other ranges, the discomfort when displaying the guide path image superimposed can be suppressed.

[0189] (10) A control program product comprising a control program, wherein,

[0190] The control program causes the processor of the control device to perform the following processes:

[0191] Obtain external information about the towing vehicle that is towing the towed vehicle;

[0192] A guide path image showing the guide path of at least one of the towing vehicle and the towed vehicle is overlaid on the surrounding image of the towing vehicle obtained based on the external information; and

[0193] The overlapping manner of the guide path images is different in the display of the first range of the towed vehicle in the surrounding images and in the second range of the surrounding images other than the first range.

[0194] According to (10), by making the display mode of the guide path image superimposed on the first range where the towed vehicle is displayed different from the display mode of the guide path image superimposed on other ranges, it is possible to suppress the discomfort when displaying the guide path image superimposed.

Claims

1. A control device, wherein the control device is provided with: an outside information acquisition section that acquires outside information of a towing vehicle that is towing a towed vehicle; and a display control section that superimposes and displays a guide path image that shows a guide path of at least either one of the towing vehicle and the towed vehicle on a surrounding image of the towing vehicle that is obtained based on the outside information, the display control section makes a manner of superimposition of the guide path image different in a first range in which the towed vehicle is displayed in the surrounding image, and a second range in which the first range is excluded in the surrounding image.

2. The control device according to claim 1, wherein the display control section superimposes the guide path image on the first range in a manner in which visual recognizability of the guide path image in the first range is lower than that in the second range.

3. The control device according to claim 2, wherein the display control section does not display the guide path image in the first range.

4. The control device according to claim 1, wherein the guide path is a path from a current position of at least either one of the towing vehicle and the towed vehicle to a target parking position of at least either one of the towing vehicle and the towed vehicle.

5. The control device according to claim 1, wherein the guide path is a path that is obtained based on specification information of the towed vehicle, and a relative angle of the towing vehicle to the towed vehicle.

6. The control device according to claim 1, wherein the display control section specifies the first range based on a variation amount of each region of the surrounding image.

7. The control device according to claim 1, wherein the display control section specifies the first range based on a relative angle of the towing vehicle to the towed vehicle.

8. The control device according to any one of claims 1 to 7, wherein the display control section repeatedly updates the first range and the second range.

9. A control method, wherein the control method causes a processor of a control device to execute the following processes: acquiring outside information of a towing vehicle that is towing a towed vehicle; superimposing and displaying a guide path image that shows a guide path of at least either one of the towing vehicle and the towed vehicle on a surrounding image of the towing vehicle that is obtained based on the outside information; and making a manner of superimposition of the guide path image different in a first range in which the towed vehicle is displayed in the surrounding image, and a second range in which the first range is excluded in the surrounding image.

10. A control program product including a control program, wherein the control program causes a processor of a control device to execute the following processes: acquiring outside information of a towing vehicle that is towing a towed vehicle; superimposing and displaying a guide path image that shows a guide path of at least either one of the towing vehicle and the towed vehicle on a surrounding image of the towing vehicle that is obtained based on the outside information; and The overlapping manner of the guide path image is made different in a first range in which the towed vehicle is displayed in the peripheral image, and a second range in which the peripheral image is other than the first range.

Images