Control device, control method, and control program

Abstract

A control device, a control method, and a control program are provided that can suppress the sense of incongruity that occurs when a guide route image is superimposed and displayed. [Solution] The system includes an external environment information acquisition unit 55 that acquires external environment information of a vehicle 10 towing a trailer 60, and a display control unit 56 that displays a guided route image 71 showing a guided route for at least one of the vehicle 10 and the trailer 60 superimposed on a peripheral image 5 of the vehicle 10 based on the external environment information. The display control unit 56 differentiates the superimposition of the guided route image 71 between a first range 62 of the peripheral image 5 in which the trailer 60 is displayed and a second range 63 of the peripheral image 5 excluding the first range 62.

Description

[Technical Field]

[0001] The present invention relates to a control device, a control method, and a control program. [Background technology]

[0002] In recent years, efforts to provide access to sustainable transportation systems that take into consideration vulnerable transport participants have been gaining momentum. To achieve this, we are focusing on research and development into autonomous driving technology to further improve traffic safety and convenience.

[0003] A conventional technique for superimposing guide lines indicating a predicted vehicle travel area based on the vehicle travel state on a vehicle surroundings image when the vehicle is traveling or turning is known. For example, Patent Document 1 describes a parking assistance device that includes an image processing unit that generates a surroundings image showing the surrounding conditions of the towing vehicle based on the image capture results of an imaging unit provided in the towing vehicle, a route acquisition unit that, when a target parking position for a towed vehicle coupled to the towing vehicle is set, acquires a guide route that extends from the target parking position as a starting point for guiding the towed vehicle to the target parking position, and a display processing unit that, when the guide route is acquired, superimposes and displays the guide route on the surroundings image. The target parking positions include a pair of target positions corresponding to the width of the towed vehicle, and the display processing unit superimposes and displays the guide route extending from at least one of the pair of target positions that is located on the inner side of the turning of the towing vehicle on the surroundings image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-069717 Summary of the Invention [Problem to be solved by the invention]

[0005] However, when a guide route is superimposed on a peripheral image, if the guide route is also superimposed on an image of the towed vehicle, there are drawbacks such as reduced visibility of the towed vehicle and the guide route for the driver, or the driver feeling uncomfortable seeing the guide route displayed on the towed vehicle. However, Patent Document 1 does not disclose a guide route that is superimposed on the towed vehicle. Therefore, there is room for improvement in how the guide route is displayed on the towing vehicle that tows the towed vehicle.

[0006] The present invention aims to provide a control device, a control method, and a control program that can suppress the sense of incongruity that occurs when a guide route image is superimposed and displayed, thereby contributing to the development of a sustainable transportation system. [Means for solving the problem]

[0007] The present invention provides an external environment information acquisition unit that acquires external environment information of a towing vehicle that tows a towed vehicle; a display control unit that displays a guide route image showing a guide route for at least one of the towing vehicle and the towed vehicle superimposed on an image of the surroundings of the towing vehicle based on the external environment information; Equipped with the display control unit changes the manner in which the guide route image is superimposed between a first range in the peripheral image in which the towed vehicle is displayed and a second range in the peripheral image excluding the first range. It is a control device.

[0008] The present invention provides a processor of the control device, Acquire external information about a towing vehicle towing a towed vehicle; a guided route image showing a guided route for at least one of the towing vehicle and the towed vehicle is superimposed on a surrounding image of the towing vehicle based on the external environment information; The superimposition manner of the guide route image is made different between a first range in the peripheral image in which the towed vehicle is displayed and a second range in the peripheral image excluding the first range. It is a control method.

[0009] The present invention provides A processor of the control device Acquire external information about a towing vehicle towing a towed vehicle; a guided route image showing a guided route for at least one of the towing vehicle and the towed vehicle is superimposed on a surrounding image of the towing vehicle based on the external environment information; The superimposition manner of the guide route image is made different between a first range in the peripheral image in which the towed vehicle is displayed and a second range in the peripheral image excluding the first range. It is a control program that executes the process. [Effects of the Invention]

[0010] According to the present invention, it is possible to provide a control device, a control method, and a control program that can suppress the sense of discomfort that occurs when a guide route image is superimposed and displayed. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a side view showing an example of a vehicle equipped with a control device of the present invention. [Figure 2] FIG. 2 is a top view of the vehicle shown in FIG. [Figure 3] 2 is a block diagram showing an example of the internal configuration of the vehicle shown in FIG. 1. FIG. [Figure 4] 1 is a diagram showing an example of a guide route image displayed on a surrounding image of a vehicle 10. FIG. [Figure 5] FIG. 5 is a diagram showing an example of generation of the rear image 5a shown in FIG. [Figure 6] 10 is a flowchart showing an example of a display process for a rear image when the vehicle 10 is moving backward. [Figure 7] 10 is a flowchart showing a modified example of the display process of the rear image when the vehicle 10 is moving backward. [Figure 8] 5 is a diagram showing a modified example of generating the rear image 5a shown in FIG. 4. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0012] An embodiment of a control device, a control method, and a control program according to the present invention will be described below with reference to the accompanying drawings. The drawings are to be viewed in accordance with the directions of the reference numerals. For simplicity and clarity, the front-rear, left-right, and up-down directions in this specification and elsewhere are described according to the directions as seen by the driver of the vehicle 10 shown in Figures 1 and 2. In the drawings, the front of the vehicle 10 is indicated as Fr, the rear as Rr, the left as L, the right as R, the top as U, and the bottom as D.

[0013] <Vehicle 10 equipped with the control device of the present invention> Fig. 1 is a side view showing an example of a vehicle equipped with a control device of the present invention. Fig. 2 is a top view of the vehicle shown in Fig. 1. Vehicle 10 is an example of a "towing vehicle" of the present invention.

[0014] The vehicle 10 is an automobile having a drive source (not shown) and wheels including drive wheels driven by the power of the drive source and steerable wheels. In this embodiment, the vehicle 10 is a four-wheeled automobile having a pair of left and right front wheels and rear wheels. The vehicle 10 is also an automobile capable of towing, for example, a trailer. The drive source of the vehicle 10 is, for example, an electric motor. The drive source of the vehicle 10 may be an internal combustion engine such as a gasoline engine or a diesel engine, or a combination of an electric motor and an internal combustion engine. The drive source of the vehicle 10 may drive a pair of left and right front wheels, a pair of left and right rear wheels, or all four wheels, i.e., a pair of left and right front wheels and rear wheels. Both the front wheels and the rear wheels may be steerable wheels, or either one of the front wheels may be steerable wheels.

[0015] The vehicle 10 further includes side mirrors 11L and 11R. The side mirrors 11L and 11R are mirrors (rearview mirrors) provided on the outside of the front doors of the vehicle 10 to enable the driver to check what is behind and to the rear sides. The side mirrors 11L and 11R are fixed to the body of the vehicle 10 by a rotation shaft that extends vertically, and can be opened and closed by rotating around this rotation shaft.

[0016] The vehicle 10 is further equipped with a front camera 12Fr, a rear camera 12Rr, a left side camera 12L, and a right side camera 12R. The front camera 12Fr is an imaging device (e.g., a digital camera) provided at the front of the vehicle 10 and captures images in the front direction of the vehicle 10. The rear camera 12Rr is a digital camera provided at the rear of the vehicle 10 and captures images in the rear direction of the vehicle 10. The left side camera 12L is a digital camera provided on the left side mirror 11L of the vehicle 10 and captures images in the left direction of the vehicle 10. The right side camera 12R is a digital camera provided on the right side mirror 11R of the vehicle 10 and captures images in the right direction of the vehicle 10.

[0017] <Internal configuration of vehicle 10> Fig. 3 is a block diagram showing an example of the internal configuration of the vehicle shown in Fig. 1. As shown in Fig. 3, the vehicle 10 has a sensor group 16, a navigation device 18, a control ECU (Electronic Control Unit) 20, an EPS (Electric Power Steering) system 22, and a communication unit 24. The vehicle 10 also has a driving force control system 26 and a braking force control system 28.

[0018] The sensor group 16 acquires various detection values ​​used for control by the control ECU 20. The sensor group 16 includes a front camera 12Fr, a rear camera 12Rr, a left side camera 12L, and a right side camera 12R. The sensor group 16 also includes a front sonar group 32a, a rear sonar group 32b, a left side sonar group 32c, and a right side sonar group 32d. The sensor group 16 also includes wheel sensors 34a and 34b, a vehicle speed sensor 36, and an operation detection unit 38.

[0019] The front camera 12Fr, rear camera 12Rr, left side camera 12L, and right side camera 12R capture images of the outside world of the vehicle 10 to obtain images of the outside world for recognizing the vehicle 10 and its surroundings. The images of the area around the vehicle 10 captured by the front camera 12Fr, rear camera 12Rr, left side camera 12L, and right side camera 12R are referred to as the front image, rear image, left side image, and right side image, respectively. An image composed of the left side image and the right side image may also be referred to as a lateral image. An image of the vehicle 10 and its surroundings generated by combining the images captured by the front camera 12Fr, rear camera 12Rr, left side camera 12L, and right side camera 12R is referred to as an overhead image of the vehicle 10.

[0020] The front sonar group 32a, rear sonar group 32b, left side sonar group 32c, and right side sonar group 32d emit sound waves around the vehicle 10 and receive reflected sound from other objects. The front sonar group 32a includes, for example, four sonars. The sonars that make up the front sonar group 32a are provided diagonally forward left, front left, front right, and diagonally forward right of the vehicle 10, respectively. The rear sonar group 32b includes, for example, four sonars. The sonars that make up the rear sonar group 32b are provided diagonally rear left, rear left, rear right, and diagonally rear right of the vehicle 10, respectively. The left side sonar group 32c includes, for example, two sonars. The sonars that make up the left side sonar group 32c are provided respectively on the front left side and rear left side of the vehicle 10. The right side sonar group 32d includes, for example, two sonars. The sonars that make up the right side sonar group 32d are provided at the front and rear of the right side of the vehicle 10, respectively.

[0021] The wheel sensors 34a, 34b detect the rotation angles of the wheels of the vehicle 10. The wheel sensors 34a, 34b may be configured as angle sensors or displacement sensors. The wheel sensors 34a, 34b output a detection pulse each time the wheel rotates a predetermined angle. The detection pulses output from the wheel sensors 34a, 34b are used to calculate the wheel rotation angle and wheel rotation speed. The travel distance of the vehicle 10 is calculated based on the wheel rotation angle. The wheel sensor 34a detects, for example, the rotation angle θa of the left rear wheel. The wheel sensor 34b detects, for example, the rotation angle θb of the right rear wheel.

[0022] The vehicle speed sensor 36 detects the speed of the body of the vehicle 10, i.e., the vehicle speed V, and outputs the detected vehicle speed V to the control ECU 20. The vehicle speed sensor 36 detects the vehicle speed V based on, for example, the rotation of a countershaft of the transmission. Note that the sensor group 16 may also include, for example, a target detection sensor capable of detecting targets around the vehicle 10. Targets include trailers connected to (towed by) the vehicle 10, other vehicles around the vehicle, and the like. The target detection sensor includes, for example, radar, lidar (light detection and ranging or laser imaging detection and ranging), sonar, a yaw rate sensor, a magnetic sensor, and a steering angle sensor.

[0023] The operation detection unit 38 detects the operation content performed by the user using the operation input unit 14, and outputs the detected operation content to the control ECU 20. The operation input unit 14 includes various user interfaces, such as a side mirror switch that switches the open / closed state of the side mirrors 11L, 11R, and a shift lever (selector lever or selector).

[0024] The navigation device 18 detects the current position of the vehicle 10 using, for example, a GPS (Global Positioning System), and provides the user with guidance on a route to the destination. The navigation device 18 has a storage device (not shown) that has a map information database.

[0025] The navigation device 18 is equipped with a touch panel 42 and a speaker 44. The touch panel 42 functions as an input device and a display device for the control ECU 20. The speaker 44 outputs various types of guidance information to the user of the vehicle 10 by voice.

[0026] The touch panel 42 is configured to allow various commands to be input to the control ECU 20. For example, a user can input commands related to movement assistance for the vehicle 10 via the touch panel 42. Movement assistance includes parking assistance and exit assistance for the vehicle 10. The touch panel 42 is also configured to display various screens related to the control content of the control ECU 20. For example, the touch panel 42 displays a screen related to movement assistance for the vehicle 10. Specifically, the touch panel 42 displays a parking assistance button for requesting parking assistance for the vehicle 10 and an exit assistance button for requesting exit assistance. The parking assistance buttons include an automatic parking button for requesting parking using automatic steering by the control ECU 20 and an assisted parking button for requesting assistance when parking via a user's operation. The exit assistance buttons include an automatic exit button for requesting exit using automatic steering by the control ECU 20 and an assisted exit button for requesting assistance when leaving via a user's operation. Furthermore, for example, an overhead image generated by combining images captured by the front camera 12Fr, rear camera 12Rr, left side camera 12L, and right side camera 12R is displayed on the touch panel 42. Note that a component other than the touch panel 42, for example, an information terminal such as a smartphone or tablet, may be used as the input device or the display device.

[0027] The control ECU 20 has an input / output unit 50, a calculation unit 52, and a memory unit 54. The calculation unit 52 is configured by, for example, a CPU (Central Processing Unit). The calculation unit 52 performs various controls by controlling each unit based on a program stored in the memory unit 54. The calculation unit 52 also inputs and outputs signals to and from each unit connected to the control ECU 20 via the input / output unit 50. The control ECU 20 is an example of a "control device" in the present invention.

[0028] The calculation unit 52 includes an outside world information acquisition unit 55 and a display control unit 56 .

[0029] The external environment information acquisition unit 55 acquires external environment images of the vehicle 10 captured by the front camera 12Fr, rear camera 12Rr, left side camera 12L, and right side camera 12R from each camera. The external environment information acquisition unit 55 acquires external environment information of the vehicle 10 towing a trailer based on the external environment images acquired from each camera. "External environment information of the vehicle 10 towing a trailer" also includes information about the trailer based on captured images of the trailer.

[0030] The display control unit 56 displays a peripheral image of the vehicle 10 based on the external environment information acquired by the external environment information acquisition unit 55. The "periphery image" may be an overhead image, a rear image, a left side image, a right side image, etc. The peripheral image may include not only an image of the surroundings but also an image of the vehicle 10 (for example, an image of the vehicle 10 in an overhead image). The peripheral image may also be generated by combining multiple pieces of external environment information.

[0031] The display control unit 56 displays a guided route image showing a guided route for at least one of the vehicle 10 and the trailer towed by the vehicle 10, superimposed on the image of the surroundings of the vehicle 10. The guided route is a route from the current position of at least one of the vehicle 10 and the trailer to a target parking position for at least one of the vehicle 10 and the trailer. The route from the current position to the target parking position may be, for example, a route from the current position of the vehicle 10 to the target parking position for the vehicle 10, a route from the current position of the trailer to the target parking position for the trailer, or a route from the current positions of the vehicle 10 and the trailer to the target parking position for the vehicle 10 and the trailer. The "target parking position" may be configured to be set by the user, for example, and may be a parking position set by the user. The "target parking position" may also be, for example, a parking position set by the vehicle 10 automatically searching for an available parking position (target parking position).

[0032] The guided route is a route calculated based on trailer specification information and the relative angle between the vehicle 10 and the trailer. The trailer specification information includes, for example, the shape (including size) of the trailer and the position of the trailer wheels. The relative angle is, for example, the hitch angle of the hitch ball that connects the vehicle 10 and the trailer. The guided route may be calculated including the traveling direction of the vehicle 10 towing the trailer. The traveling direction is the direction in which the vehicle 10 towing the trailer is desired to travel, and is a direction that the user can select from, for example, a range of -30 degrees to +30 degrees. The guided route is, for example, a guided route when the vehicle 10 towing the trailer is in reverse gear and moving backward.

[0033] The display control unit 56 displays the area in the peripheral image of the vehicle 10 where the towed trailer is displayed as a first area of ​​the peripheral image, and displays the area excluding the first area as a second area of ​​the peripheral image. The display control unit 56 displays the guide route image in different superimposition manners between the first area and the second area. The "superimposition manner" includes, for example, a non-superimposition manner and a superimposition manner with 100% transparency. The display control unit 56 superimposes the guide route image on the first area in a manner in which the visibility of the guide route image in the first area is lower than the visibility of the guide route image in the second area. "In a manner in which the visibility is lower" means, for example, a non-superimposition manner of the guide route image in the first area, or a manner in which the guide route image displayed in the first area is superimposed with a higher transparency than the guide route image displayed in the second area. Alternatively, for example, the guide route image in the first area may be superimposed behind the peripheral image (an image with 0% transparency). The display control unit 56 may not display the guide route image in the first area.

[0034] The display control unit 56 identifies the first range based on the amount of change in each region of the peripheral image. The "amount of change in each region" may be, for example, the amount of change in each pixel, or the amount of change in multiple blocks (multiple pixels within a block) into which the peripheral image is divided. The "amount of change" refers to the amount of change in pixel value per unit time. For example, the display control unit 56 identifies a region where the amount of change per unit time is equal to or less than a predetermined value as the first range in which the trailer is displayed. The display control unit 56 determines a region where the amount of change per unit time is greater than the predetermined value as a display other than the trailer, for example, a display of the background surrounding the trailer, and identifies the region as the second range excluding the first range. The display control unit 56 repeatedly calculates the "amount of change" while the vehicle 10 is traveling and updates the first range based on the calculation result.

[0035] The display control unit 56 also identifies the first range based on the relative angle between the vehicle 10 and the trailer. The "relative angle" is detected, for example, by a sensor (various types of sensors, such as mechanical and optical sensors) that can detect the hitch angle. The "relative angle" may also be detected, for example, by image recognition of a coupling part (hitch ball) that is located at a fixed position within the peripheral image.

[0036] The vehicle 10 is provided with correspondence information between the relative angle between the vehicle 10 and the trailer and a first range in which the trailer is displayed at that relative angle, i.e., information that allows the first range to be derived from the relative angle. The correspondence information may be provided, for example, as a table, or may be provided as a trained model that uses the relative angle as input and the first range as output. The correspondence information is created, for example, by detecting the hitch angle with a hitch angle sensor while the vehicle 10 is traveling, while also identifying the first range based on the amount of change in each area in the peripheral image. "While the vehicle 10 is traveling" may refer to when the vehicle 10 is actually traveling, or may refer to when a vehicle 10 for calibration is traveling. Input information for the correspondence information may include trailer specification information in addition to the relative angle.

[0037] The display control unit 56 repeatedly identifies the first range in which the trailer is displayed and the second range excluding the first range, for example, while the vehicle 10 is traveling, and updates the range to a new range.

[0038] The calculation unit 52 also performs automatic parking assistance and automatic leaving assistance for the vehicle 10 through automatic steering, in which the operation of the steering wheel 110 is performed automatically under the control of the calculation unit 52. In the automatic parking assistance and automatic leaving assistance, the operation of the accelerator pedal (not shown), the brake pedal (not shown), and the operation input unit 14 is performed automatically. The calculation unit 52 also performs auxiliary parking assistance and auxiliary leaving assistance when the user (driver) operates the accelerator pedal, the brake pedal, and the operation input unit 14 to manually park and manually leave the vehicle 10. During the automatic parking assistance and automatic leaving assistance, the user may be inside the vehicle 10 or may be outside the vehicle 10 after getting out of the vehicle 10.

[0039] For example, the calculation unit 52 performs movement control to move the vehicle 10 based on external images of the vehicle 10 acquired by the front camera 12Fr, the rear camera 12Rr, the left side camera 12L, and the right side camera 12R and a parking space designated by the user. The movement control includes parking control to automatically park the vehicle 10 in a predetermined parking space (target parking position) and exit control to automatically exit the vehicle 10 from the predetermined parking space to a target movement position. The calculation unit 52 can execute the parking control and the exit control in accordance with instruction signals input from the outside via the input / output unit 50. The input from the outside includes input via wireless communication from an information terminal or the like portable by the user of the vehicle 10. The calculation unit 52 can also transmit information related to the parking control and the exit control to an external information terminal via the input / output unit 50.

[0040] The EPS system 22 includes a steering angle sensor 100, a torque sensor 102, an EPS motor 104, a resolver 106, and an EPS ECU 108. The steering angle sensor 100 detects the steering angle θst of a steering wheel 110. The torque sensor 102 detects the torque TQ applied to the steering wheel 110.

[0041] The EPS motor 104 applies a driving force or a reaction force to a steering column 112 connected to a steering wheel 110, thereby enabling assistance to the occupant in operating the steering wheel 110 and automatic steering during parking assistance. A resolver 106 detects the rotation angle θm of the EPS motor 104. The EPS ECU 108 is responsible for overall control of the EPS system 22. The EPS ECU 108 is equipped with an input / output unit (not shown), a calculation unit (not shown), and a storage unit (not shown).

[0042] The communication unit 24 enables wireless communication with other communication devices 120. The other communication devices 120 include base stations, communication devices of other vehicles, and information terminals such as smartphones or tablet terminals that can be carried by the user of the vehicle 10. For example, the communication unit 24 includes a UWB (Ultra Wide Band: registered trademark) interface for performing UWB communication with the information terminal.

[0043] The driving force control system 26 includes a driving ECU 130. The driving force control system 26 controls the driving force of the vehicle 10. The driving ECU 130 controls the driving force of the vehicle 10 by controlling an engine (not shown) and the like based on a user's operation of an accelerator pedal (not shown).

[0044] The braking force control system 28 is provided in the braking ECU 132. The braking force control system 28 controls the braking force of the vehicle 10. The braking ECU 132 controls the braking force of the vehicle 10 by controlling a brake mechanism (not shown) and the like based on a user's operation of a brake pedal (not shown).

[0045] <Example of guidance route image display> 4 is a diagram showing an example of a guide route image displayed on the surrounding image of the vehicle 10. As shown in FIG. 4, the surrounding image 5 of the vehicle 10 displays, for example, a rearward image 5a and an overhead image 5b side by side. The rearward image 5a and the overhead image 5b are displayed when the gear position of the vehicle 10 is shifted to reverse. The rearward image 5a and the overhead image 5b are displayed on, for example, the touch panel 42 of the navigation device 18. Note that, although both the rearward image 5a and the overhead image 5b are displayed in this example, only the rearward image 5a may be displayed.

[0046] The rear image 5a is, for example, an image captured by the rear camera 12Rr of the vehicle 10. A captured image of the trailer 60 towed by the vehicle 10 and a guided route image 71 showing a guided route for the vehicle 10 and trailer 60 are superimposed on the rear image 5a. The trailer 60 is an example of a "towed vehicle" in the present invention. The guided route is, for example, a route that guides the vehicle 10 and trailer 60 to a target parking position.

[0047] In the rear image 5a, the guide route image 71 is not superimposed on the area (first range) where the trailer 60 is displayed, but is superimposed on the area (second range) excluding the image of the trailer 60. In the rear image 5a of this example, the guide route image 71 is superimposed on the area between the vehicle 10 and the trailer 60. In addition, a message image for the user, such as "Please back up the vehicle at a low speed" or "Please check your surroundings directly," is displayed on the rear image 5a.

[0048] The overhead-view image 5b is an image captured by, for example, the front camera 12Fr, the rear camera 12Rr, the left-side camera 12L, and the right-side camera 12R. The overhead-view image 5b displays a vehicle 10 towing a trailer 60. However, the vehicle displayed in the overhead-view image 5b is an "image of the vehicle 10" created by the display control unit 56. In the following description, the "image of the vehicle 10" in the overhead-view image 5b will be simply referred to as the vehicle 10a. The trailer 60 displayed in the overhead-view image 5b is also an "image of the trailer 60" created by the display control unit 56. In the following description, the "image of the trailer 60" in the overhead-view image 5b will be simply referred to as the trailer 60a.

[0049] Additionally, a guide route image 71 showing the guide route for vehicle 10a and trailer 60a and a field of view 81 of a rear image captured by rear camera 12Rr of vehicle 10 are superimposed on overhead image 5b. Furthermore, a slide bar 82 for indicating the direction in which vehicle 10a towing trailer 60a should reverse is displayed on overhead image 5b. By sliding slide bar 82 in the direction of the arrow, the user can indicate the direction in which vehicle 10a towing trailer 60a should reverse, for example, within a range of 30 degrees to the left rearward and 30 degrees to the right rearward.

[0050] <Example of rear image generation> Fig. 5 is a diagram showing an example of generation of the rear image 5a shown in Fig. 4. As shown in Fig. 5, the display control unit 56 of the vehicle 10 generates a rear external environment image 91 of the vehicle 10 based on an external environment image captured by the rear camera 12Rr. The rear external environment image 91 includes an image of the trailer 60 towed by the vehicle 10, an image of the background 61 of the vehicle 10, and the like.

[0051] In addition, the display control unit 56 generates a range division image 92 based on the rear external environment image 91, which divides the image into a first range 62 indicating the area in which the trailer 60 is displayed and a second range 63 indicating the area of ​​the rear external environment image 91 excluding the first range 62.

[0052] The display control unit 56 also generates a guide image 93 that displays a guide route image 71 for the vehicle 10 towing the trailer 60. The guide route image 71 is an image that guides the vehicle 10 towing the trailer 60 in the direction to back up from the current position to the target parking position.

[0053] The display control unit 56 generates a rearward image 5a on which the guide route image 71 is superimposed, based on the generated rearward external environment image 91, the range division image 92, and the guide image 93. In the rearward image 5a shown in Fig. 5, similar to the rearward image 5a shown in Fig. 4, the guide route image 71 is displayed only in an area excluding the image of the trailer 60 (second area), and is not displayed in an area where the trailer 60 is displayed (first area).

[0054] <Rear image display processing> FIG. 6 is a flowchart showing an example of a process for displaying a rear image when the vehicle 10 is moving backward.

[0055] The display control unit 56 determines whether or not to display a rear image of the vehicle 10 acquired by the external environment information acquisition unit 55 on the touch panel 42 of the navigation device 18 (step S11). The rear image is displayed, for example, when the gear position of the vehicle 10 is put into reverse. The display control unit 56 detects whether or not the gear position is in reverse and determines whether or not to display the rear image. The display control unit 56 may also determine whether or not to display the rear image based on, for example, whether or not the navigation mode of the navigation device 18 is turned on.

[0056] In step S11, if the rear image is not to be displayed, i.e., if the gear position is not in reverse (step S11: No), the display control unit 56 repeats the process of step S11. In step S11, if the rear image is to be displayed, i.e., if the gear position is in reverse (step S11: Yes), the display control unit 56 displays the rear image on the touch panel 42 (step S12).

[0057] Next, the display control unit 56 detects the amount of change in each area in the rear image and identifies a first range in which the trailer 60 is displayed based on the detection result (step S13). The display control unit 56 detects an area in which the amount of change per time of each pixel is equal to or less than a predetermined value, for example, and identifies the detected area as the first range in which the trailer 60 is displayed. The display control unit 56 also identifies an area in the rear image excluding the first range as a second range.

[0058] Next, the display control unit 56 superimposes and displays a guide route image showing the guide route for the vehicle 10 and the trailer 60 only on the second range excluding the first range specified in step S13 (step S14).

[0059] The display control unit 56 determines whether or not to hide the rear image of the vehicle 10 displayed on the touch panel 42, that is, whether or not the gear position is in a position other than reverse (step S15).

[0060] In step S15, if the rear image is not to be hidden (step S15: No), the display control unit 56 returns to step S13 and executes each process. In step S15, if the rear image is to be hidden (step S15: Yes), the display control unit 56 hides the rear image on the touch panel 42 (step S16).

[0061] The process of generating the guide route image to be superimposed and displayed is executed separately from this process.

[0062] 7 is a flowchart showing a modified example of the display process of a rear image when the vehicle 10 is reversing. As shown in FIG. 7, the processes from step S11 to step S12 are the same as the processes from step S11 to step S12 described in FIG.

[0063] Next, the display control unit 56 identifies a first range in which the trailer 60 is displayed based on the hitch angle of a hitch ball connecting the vehicle 10 and the trailer 60, i.e., the relative angle between the vehicle 10 and the trailer 60 (step S13A). The display control unit 56 detects the hitch angle using a sensor and identifies the first range by reading out the first range associated with the detected hitch angle, for example, from a correspondence information table. The display control unit 56 also identifies the area in the rear image excluding the first range as a second range.

[0064] The processes from step S14 to step S16 are the same as the processes from step S14 to step S16 described with reference to FIG.

[0065] <Example of rear image generation> Fig. 8 is a diagram showing a modified example of generation of the rearward image 5a shown in Fig. 4. As shown in Fig. 8, the control by the display control unit 56 to generate the rearward external scene image 91, the range division image 92, and the guide image 93 is similar to the example of generation of the rearward image 5a described in Fig. 5.

[0066] The display control unit 56 generates a rearward image 5a in which guide route images 71a and 71b are superimposed and displayed based on the generated rearward external environment image 91, range division image 92, and guide image 93. In the rearward image 5a shown in FIG. 8, the guide route image 71a is displayed in an area (second area) excluding the image of the trailer 60, similar to the manner in which the guide route image 71 is displayed in the second area in the generation example of FIG. 5. On the other hand, the guide route image 71b, which has lower visibility than the guide route image 71a in the second area (first area), is displayed in the area where the trailer 60 is displayed, which differs from the generation example of FIG. 5 in which the guide route image is not displayed in the first area. The display control unit 56 displays the guide route image 71b, for example, outlined with a dashed line, in the first area where the trailer 60 is displayed.

[0067] As described above, when the control device (control ECU 20) in the embodiment of the present invention superimposes and displays a guide route image 71 for guiding the vehicle 10 and the trailer 60 to reverse on the rear image 5a of the vehicle 10 towing the trailer 60, the control device displays the guide route image 71 superimposed on a first range of the rear image 5a in which the trailer 60 is displayed, and the guide route image 71 superimposed on a second range excluding the first range, in different superimposition modes. With this configuration, as shown in FIG. 4, for example, the user can easily see that the guide route image 71 superimposed on the area between the towing vehicle 10 and the towed trailer 60 (the second range) continues to extend into the area below the trailer 60. As a result, when the user looks at the rear image 5a on which the guide route image 71 is superimposed, the user can see the rear image 5a without feeling uncomfortable.

[0068] Furthermore, the control device superimposes the guide route image 71 (71b) on the first range in which the trailer 60 is displayed so that the guide route image 71 (71b) superimposed on the first range in which the trailer 60 is displayed is less visible than the guide route image 71 (71a) superimposed on the second range in which the trailer 60 is not displayed. This allows the user to naturally distinguish and recognize the guide route image 71 (71a) superimposed on the second range from the guide route image 71 (71b) that continues from the guide route image 71 (71a) and extends into the area below the trailer 60.

[0069] Furthermore, the control device identifies the first range based on the amount of change in pixel values ​​in each area of ​​the rear image 5a. With this configuration, it is possible to identify an area with a small amount of change as the area where the trailer 60 is displayed (first range), and to identify an area with a large amount of change as the background area (second range) that appears around the trailer 60. This makes it possible to accurately identify the first range, and to appropriately superimpose and display the guide route image 71.

[0070] Furthermore, the control device identifies the first range based on the relative angle between the vehicle 10 and the trailer 60 towed by the vehicle 10. With this configuration, the relative angle (hitch angle) of the trailer with respect to the vehicle 10 is detected, and the corresponding information of the first range associated with the hitch angle is read from the table, making it possible to easily identify the range in the rear image 5a in which the trailer 60 is displayed.

[0071] The control method described in the above-described embodiment can be realized by executing a prepared control program on a computer. The control program is recorded on a computer-readable storage medium and executed by being read from the storage medium. The control program may be provided in a form stored on a non-transitory storage medium such as a flash memory, or may be provided via a network such as the Internet. The computer that executes the control program may be included in the control device, or may be included in an electronic device such as a smartphone, tablet, or personal computer that can communicate with the control device, or may be included in a server device that can communicate with these control devices and electronic devices.

[0072] Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and modifications, improvements, etc. are possible as appropriate.

[0073] This specification also describes at least the following: Note that the components in parentheses correspond to those in the above-described embodiment, but are not limited to these.

[0074] (1) an external environment information acquisition unit (external environment information acquisition unit 55) that acquires external environment information of a towing vehicle (vehicle 10) towing a towed vehicle (trailer 60); a display control unit (display control unit 56) that displays a guided route image (guided route image 71) showing a guided route for at least one of the towing vehicle and the towed vehicle superimposed on a surrounding image (surrounding image 5) of the towing vehicle based on the external environment information; Equipped with The display control unit changes the superimposition manner of the guide route image between a first range (first range 62) in the peripheral image in which the towed vehicle is displayed and a second range (second range 63) in the peripheral image excluding the first range. Control device (control ECU 20).

[0075] According to (1), by making the display mode of the guide route image superimposed on the first range in which the towed vehicle is displayed different from the display mode of the guide route image superimposed on other ranges, it is possible to reduce the sense of incongruity when the guide route image is superimposed and displayed.

[0076] (2) The control device according to (1), the display control unit superimposes the guide route image on the first range in a manner such that visibility of the guide route image in the first range is lower than visibility of the guide route image in the second range. Control device.

[0077] As in (2), by displaying the guide route image superimposed on the first area in which the towed vehicle is displayed with reduced visibility, it is possible to reduce the sense of incongruity of the superimposed guide route image.

[0078] (3) The control device according to (2), The display control unit does not display the guide route image in the first range. Control device.

[0079] As in (3), by not displaying the guide route image that is superimposed on the first area in which the towed vehicle is displayed, it is possible to reduce the sense of incongruity caused by the superimposed guide route image.

[0080] (4) The control device according to any one of (1) to (3), the guided route is a route from a current position of at least one of the towing vehicle and the towed vehicle to a target parking position of at least one of the towing vehicle and the towed vehicle; Control device.

[0081] As in (4), the guide route displayed superimposed on the surrounding image is preferably a route from the current positions of the towing vehicle and the towed vehicle to the target parking position.

[0082] (5) The control device according to any one of (1) to (4), the guided route is a route based on specification information of the towed vehicle and a relative angle between the towing vehicle and the towed vehicle. Control device.

[0083] According to (5), an appropriate guide route can be generated by determining the guide route based on the specification information of the towed vehicle and the relative angle between the towing vehicle and the non-towing vehicle.

[0084] (6) The control device according to any one of (1) to (5), the display control unit specifies the first range based on an amount of change in each region of the peripheral image. Control device.

[0085] According to (6), by determining the first range in which the towed vehicle is displayed based on the amount of change in each area in the peripheral image, it is possible to determine the accurate first range.

[0086] (7) The control device according to any one of (1) to (5), the display control unit identifies the first range based on a relative angle between the towing vehicle and the towed vehicle. Control device.

[0087] According to (7), by determining the first range in which the towed vehicle is displayed based on the relative angle between the towing vehicle and the non-towing vehicle, it is possible to determine the accurate first range.

[0088] (8) The control device according to any one of (1) to (7), the display control unit repeatedly updates the first range and the second range. Control device.

[0089] As in (8), by repeatedly updating the first range and the second range, the guide route image can be appropriately superimposed and displayed even during towing.

[0090] (9) The processor of the control device Acquire external information about a towing vehicle towing a towed vehicle; a guided route image showing a guided route for at least one of the towing vehicle and the towed vehicle is superimposed on a surrounding image of the towing vehicle based on the external environment information; The superimposition manner of the guide route image is made different between a first range in the peripheral image in which the towed vehicle is displayed and a second range in the peripheral image excluding the first range. Control method.

[0091] According to (9), by making the display mode of the guide route image superimposed on the first range in which the towed vehicle is displayed different from the display mode of the guide route image superimposed on other ranges, the sense of incongruity when the guide route image is superimposed and displayed can be reduced.

[0092] (10) A processor of the control device, Acquire external information about a towing vehicle towing a towed vehicle; a guided route image showing a guided route for at least one of the towing vehicle and the towed vehicle is superimposed on a surrounding image of the towing vehicle based on the external environment information; The superimposition manner of the guide route image is made different between a first range in the peripheral image in which the towed vehicle is displayed and a second range in the peripheral image excluding the first range. A control program that executes processing.

[0093] According to (10), by making the display mode of the guide route image superimposed on the first range in which the towed vehicle is displayed different from the display mode of the guide route image superimposed on other ranges, it is possible to suppress the sense of incongruity when the guide route image is superimposed and displayed. [Explanation of symbols]

[0094] 5. Peripheral images 5a Rear view 5b Bird's-eye view 10,10a Vehicle 11L, 11R side mirrors 11R Side mirror 11L 12Fr forward camera 12Rr rear camera 12L Left side camera 12R Right side camera 14 Operation input section 16 sensors 18 Navigation devices 20 Control ECU 22 EPS system 24 Communications Department 26 Driving Force Control System 28 Braking force control system 32a Forward Sonar Group 32b Rear Sonar Group 32c Left Sonar Group 32d Right Sonar Group 34a, 34b Wheel sensors 36 Vehicle speed sensor 38 Operation detection unit 42 Touch Panel 44 speakers 50 Input / output section 52 Arithmetic section 54 Storage section 55 External world information acquisition department 56 Display control unit 60,60a trailer 61 Background 62 First Range 63 Second Range 71, 71a, 71b Guidance route images 81 Field of view 82 Slide bar 91 Rear external world image 92 Range division image 93 Guide Images 100 Steering angle sensor 102 Torque sensor 104 EPS motor 106 Resolver 108 ECU 110 Steering 112 Steering column 120 Communication equipment 130 Drive ECU 132 Braking ECU

Claims

1. an external environment information acquisition unit that acquires external environment information of a towing vehicle that tows a towed vehicle; a display control unit that displays a guide route image showing a guide route for at least one of the towing vehicle and the towed vehicle superimposed on an image of the surroundings of the towing vehicle based on the external environment information; Equipped with the display control unit changes the manner in which the guide route image is superimposed between a first range in the peripheral image in which the towed vehicle is displayed and a second range in the peripheral image excluding the first range. Control device.

2. The control device according to claim 1, the display control unit superimposes the guide route image on the first range in a manner such that visibility of the guide route image in the first range is lower than visibility of the guide route image in the second range. Control device.

3. The control device according to claim 2, the display control unit does not display the guide route image in the first range. Control device.

4. The control device according to claim 1, the guided route is a route from a current position of at least one of the towing vehicle and the towed vehicle to a target parking position of at least one of the towing vehicle and the towed vehicle; Control device.

5. The control device according to claim 1, the guided route is a route based on specification information of the towed vehicle and a relative angle between the towing vehicle and the towed vehicle. Control device.

6. The control device according to claim 1, the display control unit specifies the first range based on an amount of change in each region of the peripheral image. Control device.

7. The control device according to claim 1, the display control unit specifies the first range based on a relative angle between the towing vehicle and the towed vehicle. Control device.

8. The control device according to any one of claims 1 to 7, the display control unit repeatedly updates the first range and the second range. Control device.

9. a processor of the control device, Acquire external information about a towing vehicle towing a towed vehicle; a guided route image showing a guided route for at least one of the towing vehicle and the towed vehicle is superimposed on a surrounding image of the towing vehicle based on the external environment information; a first area of ​​the peripheral image in which the towed vehicle is displayed and a second area of ​​the peripheral image excluding the first area are overlaid with the guide route image in a different manner; Control method.

10. A processor of the control device Acquire external information about a towing vehicle towing a towed vehicle; a guided route image showing a guided route for at least one of the towing vehicle and the towed vehicle is superimposed on a surrounding image of the towing vehicle based on the external environment information; a first area of ​​the peripheral image in which the towed vehicle is displayed and a second area of ​​the peripheral image excluding the first area are overlaid with the guide route image in a different manner; A control program that executes processing.

Images