Control device, control method, and control program

Abstract

To provide a control device, a control method, and a control program capable of improving usability when performing automatic driving control of a vehicle to a target position.SOLUTION: This device is provided with an external environment recognition part 55 for acquiring the recognition data of the external environment of a vehicle 10, an automatic driving control part 56 for moving the vehicle 10 to a target position, a storage part 54 for storing target position information related to the target position, and a target position detection part 57 for detecting the target position based on the recognition data. In a case where the target position is detected, the automatic driving control unit 56 determines whether or not the self-position estimation of the vehicle 10 with respect to the target position can be performed on the basis of the comparison between the recognition data and the target position information, and in a case where the self-position estimation can be performed, determines whether or not the target object 82 not included in the target position information is present around the target position, and in a case where the target object 82 is present, notifies the user of an inquiry about whether or not to execute the automatic driving control to the target position, and in a case where the target object 82 is not present, executes the automatic driving control.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] The present invention relates to a control device, a control method, and a control program. [Background Art]

[0002] In recent years, efforts to provide access to sustainable transportation systems that take into account even vulnerable people among road users have become active. To achieve this goal, efforts are focused on research and development that further improves traffic safety and convenience through research and development related to autonomous driving technology.

[0003] Conventionally, automatic parking control that automatically moves a vehicle to a predetermined parking space for parking has been known. Patent Document 1 describes a vehicle driving support device that stores driving operations performed during past parking in a storage means in association with images captured by a camera, and if it is determined that there is similarity between an image captured by the camera when a new parking is to be performed and an image stored in the storage means, that is, if parking was performed in a similar state (same location) in the past, controls the vehicle to park in accordance with the stored driving operation. [Prior Art Documents] [Patent Documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2000-127851 [Summary of the Invention] [Problem to be Solved by the Invention]

[0005] However, when determining whether a parking spot has been used before based on the similarity of images captured by a camera, even if the parking spot has been used before, if an object that was not captured in the previous parking image is captured during the new parking, it may be determined that there is no similarity. In that case, automatic parking cannot be performed regardless of the object captured, which can be a burden on the user. Patent Document 1 does not describe the control processing when an object different from that captured in the previous parking image is captured. Therefore, there is room for improvement in this respect of the prior art.

[0006] The present invention aims to provide a control device, a control method, and a control program that can improve usability when automatically controlling a vehicle to a target position. Ultimately, this contributes to the development of sustainable transportation systems. [Means for solving the problem]

[0007] The present invention An external environment recognition unit that acquires recognition data of the external environment of a moving object, An automatic driving control unit that performs automatic driving control to move the moving body to a target position, A storage unit that stores target location information related to the aforementioned target location, The system includes a target position detection unit that detects the target position based on the recognition data, The automatic driving control unit, When the aforementioned target position is detected, Based on a comparison of the recognition data and the target position information, a first determination process is performed to determine whether or not the moving body can estimate its own position relative to the target position. If the self-position estimation fails in the first determination process, the user of the moving body selects the target position for the automatic driving control. It is a control device.

[0008] The present invention A control method using a control device that acquires recognition data of the external environment of a moving object, performs automatic driving control to move the moving object to a target position, stores target position information related to the target position, and detects the target position based on the recognition data, The processor of the control device When the aforementioned target position is detected, Based on a comparison of the recognition data and the target position information, a first determination process is performed to determine whether or not the moving body can estimate its own position relative to the target position. If the self-position estimation fails in the first determination process, the user of the moving body selects the target position for the automatic driving control. This is a control method.

[0009] The present invention A control program for a control device that acquires recognition data of the external environment of a moving object, performs automatic driving control to move the moving object to a target position, stores target position information related to the target position, and detects the target position based on the recognition data, The processor of the control device, When the aforementioned target position is detected, Based on a comparison of the recognition data and the target position information, a first determination process is performed to determine whether or not the moving body can estimate its own position relative to the target position. If the self-position estimation fails in the first determination process, the user of the moving body selects the target position for the automatic driving control. This is a control program for executing a process. [Effects of the Invention]

[0010] According to the control device, control method, and control program of the present invention, usability can be improved when automatically controlling a vehicle to a target position. [Brief explanation of the drawing]

[0011] [Figure 1]It is a side view showing an example of a vehicle equipped with the control device of the present embodiment. [Figure 2] It is a top view of the vehicle shown in FIG. 1. [Figure 3] It is a block diagram showing the internal configuration of the vehicle shown in FIG. 1. [Figure 4] It is a diagram showing an example of an image displayed on a navigation device when automatically parking the vehicle. [Figure 5] It is a flowchart showing an automatic parking process for parking the vehicle at a new target position. [Figure 6] It is a diagram in which a target position is set on a bird's-eye view image generated based on a captured image of a camera. [Figure 7] It is a diagram showing peripheral feature points with respect to the target position shown in FIG. 6. [Figure 8] It is a diagram showing an example of a map of target positions registered in a storage unit. [Figure 9] It is a flowchart showing a process for automatically parking the vehicle at a registered target position. [Figure 10] It is a diagram showing an example of an inquiry screen when a target that did not exist at the time of registration is detected. [Figure 11] It is a flowchart showing a first modification of the process for automatically parking the vehicle at a registered target position. [Figure 12] It is a flowchart showing a second modification of the process for automatically parking the vehicle at a registered target position. MODE FOR CARRYING OUT THE INVENTION

[0012] Hereinafter, an embodiment of the control device, control method, and control program of the present invention will be described with reference to the accompanying drawings. Note that the drawings are to be viewed in the orientation of reference signs. In addition, in order to make the description simple and clear in this specification and the like, the directions of front-back, left-right, and up-down are described in accordance with the directions viewed from the driver of the vehicle 10 shown in FIGS. 1 and 2. In the drawings, for the vehicle 10, the front is denoted as Fr, the rear as Rr, the left as L, the right as R, the upper as U, and the lower as D.

[0013] <Vehicle 10 equipped with the control device of the present invention> Figure 1 is a side view of a vehicle 10 equipped with the control device of the present invention. Figure 2 is a top view of the vehicle 10 shown in Figure 1. Vehicle 10 is an example of a mobile body of the present invention.

[0014] Vehicle 10 is an automobile having a drive source (not shown) and wheels including drive wheels and steerable wheels that are driven by the power of the drive source. In this embodiment, vehicle 10 is a four-wheeled automobile having a pair of left and right front wheels and rear wheels. The drive source of vehicle 10 is, for example, an electric motor. The drive source of vehicle 10 may be an internal combustion engine such as a gasoline engine or a diesel engine, or a combination of an electric motor and an internal combustion engine. Furthermore, the drive source of vehicle 10 may drive a pair of left and right front wheels, a pair of left and right rear wheels, or all four wheels including a pair of left and right front wheels and rear wheels. Both the front wheels and rear wheels may be steerable wheels, or either one of them may be a steerable wheel.

[0015] Vehicle 10 is further equipped with side mirrors 11L and 11R. Side mirrors 11L and 11R are mirrors (rearview mirrors) provided on the outside of the front passenger doors of vehicle 10 for the driver to check the area behind and to the rear side. Side mirrors 11L and 11R are each fixed to the body of vehicle 10 by a vertically extending pivot axis, and can be opened and closed by rotating around this pivot axis.

[0016] Vehicle 10 is further equipped with a front camera 12Fr, a rear camera 12Rr, a left-side camera 12L, and a right-side camera 12R. The front camera 12Fr is a digital camera located in front of vehicle 10 that captures images of the area in front of vehicle 10. The rear camera 12Rr is a digital camera located behind vehicle 10 that captures images of the area behind vehicle 10. The left-side camera 12L is a digital camera located on the left side mirror 11L of vehicle 10 that captures images of the left side of vehicle 10. The right-side camera 12R is a digital camera located on the right side mirror 11R of vehicle 10 that captures images of the right side of vehicle 10.

[0017] <Internal configuration of vehicle 10> Figure 3 is a block diagram showing an example of the internal configuration of the vehicle 10 shown in Figure 1. As shown in Figure 3, the vehicle 10 includes a sensor group 16, a navigation device 18, a control ECU (Electronic Control Unit) 20, an EPS (Electric Power Steering) system 22, and a communication unit 24. The vehicle 10 further includes a drive force control system 26 and a braking force control system 28.

[0018] The sensor group 16 acquires various detection values ​​used for control by the control ECU 20. The sensor group 16 includes a front camera 12Fr, a rear camera 12Rr, a left-side camera 12L, and a right-side camera 12R. The sensor group 16 also includes a front sonar group 32a, a rear sonar group 32b, a left-side sonar group 32c, and a right-side sonar group 32d. Furthermore, the sensor group 16 includes wheel sensors 34a and 34b, a vehicle speed sensor 36, and an operation detection unit 38.

[0019] The front camera 12Fr, rear camera 12Rr, left-side camera 12L, and right-side camera 12R acquire recognition data (e.g., surrounding images) for recognizing the outside world of the vehicle 10 by imaging the area around the vehicle 10. The surrounding images captured by the front camera 12Fr, rear camera 12Rr, left-side camera 12L, and right-side camera 12R are referred to as the front image, rear image, left-side image, and right-side image, respectively. The image composed of the left-side image and the right-side image may also be referred to as the side image.

[0020] The forward sonar group 32a, the rear sonar group 32b, the left-side sonar group 32c, and the right-side sonar group 32d emit sound waves around the vehicle 10 and receive reflected sound from other objects. The forward sonar group 32a includes, for example, four sonars. The sonars constituting the forward sonar group 32a are located on the left front, front left, front right, and right front of the vehicle 10, respectively. The rear sonar group 32b includes, for example, four sonars. The sonars constituting the rear sonar group 32b are located on the left rear, rear left, rear right, and right rear of the vehicle 10, respectively. The left-side sonar group 32c includes, for example, two sonars. The sonars constituting the left-side sonar group 32c are located on the left front and left rear of the vehicle 10, respectively. The right-side sonar group 32d includes, for example, two sonars. The sonars constituting the right-side sonar group 32d are located on the front right side and the rear right side of the vehicle 10, respectively.

[0021] Wheel sensors 34a and 34b detect the rotation angle of the wheels of the vehicle 10. Wheel sensors 34a and 34b may be composed of angle sensors or displacement sensors. Wheel sensors 34a and 34b output a detection pulse each time the wheel rotates by a predetermined angle. The detection pulses output from wheel sensors 34a and 34b are used to calculate the rotation angle and rotation speed of the wheels. Based on the rotation angle of the wheels, the distance traveled by the vehicle 10 is calculated. Wheel sensor 34a detects, for example, the rotation angle θa of the left rear wheel. Wheel sensor 34b detects, for example, the rotation angle θb of the right rear wheel.

[0022] The vehicle speed sensor 36 detects the speed of the vehicle body 10, i.e., the vehicle speed V, and outputs the detected vehicle speed V to the control ECU 20. The vehicle speed sensor 36 detects the vehicle speed V based, for example, on the rotation of the transmission countershaft.

[0023] The operation detection unit 38 detects the user's operation using the operation input unit 14 and outputs the detected operation to the control ECU 20. The operation input unit 14 includes various user interfaces, such as a side mirror switch for switching the open / closed state of the side mirrors 11L and 11R, and a shift lever (selector lever or selector).

[0024] The navigation device 18 detects the current location of the vehicle 10, for example, using GPS (Global Positioning System), and guides the user on the route to the destination. The navigation device 18 has a storage device (not shown) equipped with a map information database.

[0025] The navigation device 18 is equipped with a touch panel 42 and a speaker 44. The touch panel 42 functions as an input and display device for the control ECU 20. The speaker 44 outputs various guidance information to the user (passenger) of the vehicle 10 by voice.

[0026] The touch panel 42 is configured to allow input of various commands to the control ECU 20. For example, the user can input commands related to vehicle movement assistance for the vehicle 10 via the touch panel 42. This movement assistance includes parking assistance and exit assistance for the vehicle 10. The touch panel 42 is also configured to display various screens related to the control content of the control ECU 20. For example, the touch panel 42 displays screens related to vehicle movement assistance for the vehicle 10. Specifically, the touch panel 42 displays a parking assistance button to request parking assistance for the vehicle 10 and an exit assistance button to request exit assistance. The parking assistance buttons include an automatic parking button to request parking by automatic steering of the control ECU 20 and an auxiliary parking button to request assistance when parking by the driver. The exit assistance buttons include an automatic exit button to request exiting by automatic steering of the control ECU 20 and an auxiliary exit button to request assistance when exiting by the driver. Other components besides the touch panel 42, such as information terminals like smartphones or tablet devices, may be used as input or display devices. Alternatively, a Head-Up Display (HUD) may be used as the display device.

[0027] The control ECU 20 includes an input / output unit 50, an arithmetic unit 52, and a storage unit 54. The arithmetic unit 52 is configured, for example, by a CPU (Central Processing Unit). The arithmetic unit 52 performs various controls by controlling each unit based on a program stored in the storage unit 54. The arithmetic unit 52 also inputs and outputs signals to and from each unit connected to the control ECU 20 via the input / output unit 50. The control ECU 20 is an example of the control device of the present invention.

[0028] The calculation unit 52 includes an external environment recognition unit 55 that acquires recognition data of the external environment of the vehicle 10, an automatic driving control unit 56 that performs automatic driving control to move the vehicle 10 to a target position, and a target position detection unit 57 that detects the target position based on the recognition data.

[0029] The external environment recognition unit 55 acquires images of the vehicle 10's surroundings (external environment recognition data) captured by the front camera 12Fr, rear camera 12Rr, left-side camera 12L, and right-side camera 12R from each camera. The external environment recognition unit 55 can also acquire external environment recognition data of the vehicle 10 acquired by the sonar groups 32a to 32d or radar.

[0030] The automatic driving control unit 56 assists in the movement of the vehicle 10 by automatic steering, which automatically operates the steering wheel 110 under the control of the automatic driving control unit 56. In the automatic steering assistance, the accelerator pedal (not shown), brake pedal (not shown), and operation input unit 14 are operated automatically. The automatic driving control unit 56 also provides assistance when the driver manually moves the vehicle 10 by operating the accelerator pedal, brake pedal, and operation input unit 14.

[0031] For example, the automatic driving control unit 56 controls the automatic driving to move the vehicle 10 to a predetermined target location based on recognition data of the vehicle 10's external environment acquired by the front camera 12Fr, rear camera 12Rr, left-side camera 12L, and right-side camera 12R, and a predetermined target location specified by the user. The predetermined target location includes registered target locations registered by the user in the memory unit 54, and unregistered target locations that are not registered but where the vehicle 10 can be parked. A registered target location is a specific target location for the user, for example, a parking space where the user frequently parks their vehicle 10. Registered target locations include, for example, monthly parking lots, home parking lots, and frequently used coin-operated parking lots. Registered target locations are detected by comparing "target location information" related to the location where the vehicle 10 is parked. Target location information refers to "feature points" of the target location detected from the sensing data of the cameras, sonar, and radar.

[0032] The target position detection unit 57 detects the target position based on recognition data of the vehicle 10's external environment acquired by the front camera 12Fr, rear camera 12Rr, left-side camera 12L, and right-side camera 12R. The target position detection unit 57 detects the target position by comparing the feature points of the external environment recognition data for the currently detected target position with the feature points of the external environment recognition data for target positions previously registered by the user. For example, if the target position detection unit 57 determines that the agreement rate between the feature points of the current external environment recognition data and the feature points of the past external environment recognition data is above a threshold, it detects the currently detected target position as a registered target position previously registered by the user. Target position detection is performed, for example, when the vehicle 10 approaches the target position (for example, when it approaches within a radius of 100m) based on location information such as GPS. However, target position detection may be performed continuously while the vehicle 10 is moving.

[0033] Furthermore, the target location detection unit 57 registers the target location specified by the user in the storage unit 54. The target location detection unit 57 associates the target location to be registered with feature points (target location information) from the recognition data of the external environment of the vehicle 10. Feature points related to the target location include feature points of the target location itself and feature points of the area surrounding the target location. Feature points of the target location itself include, for example, feature points where the word "Parking" is displayed within the target location. Feature points of the area surrounding the target location include, for example, distinctive buildings or obstacles in the surrounding area. The target location is registered in the storage unit 54 as a map including feature points in the surrounding image of the vehicle 10.

[0034] Specifically, when the target position detection unit 57 detects a target position, the automatic driving control unit 56 performs a first determination process to determine whether or not the vehicle 10 can estimate its own position relative to the detected target position, based on a comparison between the vehicle 10's current external recognition data and the target position information registered in the storage unit 54. Self-position estimation refers to recognizing the vehicle 10's current relative position to the detected target position by comparing feature points detected in the current external recognition data with the registered target position information (feature points) of the target position.

[0035] If the automatic driving control unit 56 has been able to estimate its own position in the first determination process, it executes a second determination process to determine whether or not there are any objects not included in the target position information in the vicinity of the detected target position. The vicinity of the target position includes the area within the target position and the area outside the target position. An object is something that is detected by sensing using a camera, sonar, or radar. Basically, it is an object (e.g., a person, an object, a bicycle, an animal, etc.), but it may also be a hole (e.g., a crack in the ground, etc.).

[0036] If the automatic driving control unit 56 determines in the second determination process that a target exists, it notifies the user of the vehicle 10 to inquire whether or not to perform automatic driving control to the target position, and executes automatic driving control according to the user's response. If the automatic driving control unit 56 determines in the second determination process that no target exists, it executes automatic driving control without inquiring whether or not to perform automatic driving control.

[0037] If the automatic driving control unit 56 fails to estimate its own position in the first determination process, it will not execute automatic driving control. If the automatic driving control unit 56 fails to estimate its own position in the first determination process, it will accept a selection of a target position for automatic driving control from the user of the vehicle 10. Even if it is unable to estimate its own position, the automatic driving control unit 56 will not immediately discontinue automatic driving control, but will accept a specification from the user for the target position to which the vehicle 10 should move.

[0038] In the second determination process, the automatic driving control unit 56 determines whether or not a target exists based on a comparison between the user's gaze detection result (current gaze) and the user's gaze history information (past gaze). For example, if the current gaze deviates significantly from the representative value of the user's gaze history when the vehicle was automatically parked at the same target position in the past, it determines that "a target exists". The user's gaze history includes the directions and places the user often looks at when automatically parking (user habits). The representative value of the history may be, for example, the average value over multiple history events. In the second determination process, the automatic driving control unit 56 extracts gaze history information from the user's gaze history information for relative positions that are the same as the current relative position of the vehicle 10 to the target position of the vehicle 10, and determines whether or not a target exists based on a comparison between the user's gaze detection result and the extracted gaze history information.

[0039] If the automatic driving control unit 56 is able to estimate its own position in the first determination process and the history information of automatic driving control for the target position satisfies predetermined conditions, it will notify the user of the vehicle 10 to inquire whether or not to perform automatic driving control, even if no target exists. The predetermined conditions are that the user has little experience with automatic driving control for the target position, such as the number of times or frequency of use.

[0040] Note that "parking" is synonymous with "parking," for example. For example, "parking" refers to a stop involving passengers getting in and out of the vehicle, excluding temporary stops at traffic lights, etc. Also, "target position" refers to the parking position where a moving object is parked. "Parking position" is the position where a moving object is stopped, that is, the position where it is parked.

[0041] The EPS system 22 includes a steering angle sensor 100, a torque sensor 102, an EPS motor 104, a resolver 106, and an EPS ECU 108. The steering angle sensor 100 detects the steering angle θst of the steering 110. The torque sensor 102 detects the torque TQ applied to the steering 110.

[0042] The EPS motor 104 provides driving force or reaction force to the steering column 112 connected to the steering 110, thereby enabling assistance for the occupant's operation of the steering 110 and automatic steering during parking assistance. The resolver 106 detects the rotation angle θm of the EPS motor 104. The EPS ECU 108 controls the entire EPS system 22. The EPS ECU 108 is equipped with an input / output unit (not shown), a calculation unit (not shown), and a storage unit (not shown).

[0043] The communication unit 24 enables wireless communication with other communication devices 120. Other communication devices 120 include base stations, communication devices in other vehicles, and information terminals such as smartphones carried by passengers in vehicle 10.

[0044] The drive force control system 26 is equipped with a drive ECU 130. The drive force control system 26 performs drive force control of the vehicle 10. The drive ECU 130 controls the drive force of the vehicle 10 by controlling an engine (not shown) and other components (not shown) based on user operation of an accelerator pedal (not shown).

[0045] The braking force control system 28 is equipped with a braking ECU 132. The braking force control system 28 performs braking force control of the vehicle 10. The braking ECU 132 controls the braking force of the vehicle 10 by controlling a braking mechanism (not shown) and other components (not shown) based on user operation of a brake pedal (not shown).

[0046] <Automatic parking controlled by the control unit (control ECU20)> Next, the automatic parking of the vehicle 10 by the control ECU 20 will be explained with reference to Figures 4 to 10.

[0047] [Display image during automatic parking] Figure 4 shows an example of the "automatic parking menu" displayed on the touch panel 42 of the navigation device 18 when the vehicle 10 is automatically parked. The images of the automatic parking menu are displayed by touching the automatic parking button 60 displayed on the touch panel 42. The automatic parking button 60 is touched, for example, when a user who intends to park the vehicle 10 approaches the parking lot, in order to display the automatic parking menu.

[0048] As shown in Figure 4, the image of the automatic parking menu displays a new registration button 61 and a registered target location image button 62. The new registration button 61 is operated when parking the vehicle 10 at a parking location to be newly registered as a target location. The registered target location image button 62 is operated when parking the vehicle 10 at a registered target location that has already been registered. Examples of registered target location image buttons 62 include a registered target location image button for a home parking lot registered as a target location, as shown in No. 1 image button 62a, and a registered target location image button for a frequently used coin parking lot registered as a target location, as shown in No. 2 image button 62b. The image displayed on the registered target location image button 62 is an image captured by, for example, the vehicle 10's front camera 12Fr at the time of registration.

[0049] [Registering a new target location] First, the process of registering the target position when the new registration button 61 is touched will be explained with reference to Figures 5 to 8.

[0050] Figure 5 is a flowchart showing the process of automatic parking to park the vehicle 10 at a new target location. Figure 6 shows the target location 76 set on an overhead image 71 (composite image) generated from recognition data of the vehicle 10's external environment acquired by the front camera 12Fr, rear camera 12Rr, left-side camera 12L, and right-side camera 12R. Figure 7 shows the surrounding feature points for the target location 76 shown in Figure 6. Figure 8 shows an example of a map of the target location 76 registered in the memory unit 54. When the new registration button 61 is touched, the automatic driving control unit 56 starts the process shown in Figure 5.

[0051] The automatic driving control unit 56 accepts manual settings by the user to set the target position of the vehicle 10 (step S11).

[0052] As shown in Figure 6, the overhead view image 71 captures the user's residence (building) 73 and several plants (obstacles) 74a to 74f. The parking lot 72 is located between the building 73 and the obstacles 74a to 74f. The user's vehicle 10 is parked in front of the parking lot 72. The vehicle displayed in the overhead view image 71 is an image showing the vehicle 10 as seen from above, and is a vehicle image 75 that has been pre-generated (captured) and stored in the memory unit 54, etc. Manual setting of the target position can be done by touching the touch panel 42. Specifically, the rectangular dashed frame 76a displayed on the overhead view image 71 is touched and moved to surround the parking lot 72, as shown in Figure 6. As a result, the dashed frame 76a that has been moved to surround the parking lot 72 is set as the target position 76.

[0053] In step S11, upon receiving the manual setting of the target position 76, the automatic driving control unit 56 detects feature points related to the specified target position 76 from the external recognition data acquired by the front camera 12Fr, rear camera 12Rr, left side camera 12L, and right side camera 12R. Based on the detected feature points and the external recognition data acquired by the front camera 12Fr, rear camera 12Rr, left side camera 12L, and right side camera 12R, the automatic driving control unit 56 automatically parks the vehicle 10 at the target position 76 (step S12).

[0054] As shown in the overhead view 71 in Figure 7, the automatic driving control unit 56 detects feature points related to the designated target position 76, such as building feature points 77a to 77d indicating the position of the outline of the building 73 on the side closer to the target position 76, and obstacle feature points 78a to 78f indicating the positions of obstacles 74a to 74f present around the target position 76. Note that Figure 7 shows the state in which the vehicle 10 is parked at the designated target position 76.

[0055] Next, the automatic driving control unit 56 registers a feature point map in the storage unit 54, which consists of the target position 76 manually set in step S11 and the feature points related to the target position 76 detected in step S12 (step S13).

[0056] The feature point map, consisting of the target location 76 and its feature points, is registered as a feature point map 91 as shown in Figure 8. The feature point map 91 is registered as a map showing the relationship between the detected building feature points 77a to 77d and obstacle feature points 78a to 78f and the manually set target location 76, as shown in Figure 7.

[0057] As a result, the target position 76 specified by the user is associated with feature points indicating obstacles and other objects present around the target position 76 and registered in the memory unit 54, and is displayed in the automatic parking menu (see Figure 4) as one of the registered target position image buttons 62.

[0058] [Automatic parking to registered target location] Next, the automatic parking of vehicle 10 at a registered target location will be explained with reference to Figures 9 and 10. Figure 9 is a flowchart showing an example of the process for automatically parking vehicle 10 at a registered target location. Figure 10 shows an example of a query screen that appears when a target that was not present when the target location was registered is detected. The control ECU 20 recognizes the position of vehicle 10, for example, using GPS, and when it determines that vehicle 10 has reached the vicinity of the target location, it starts the process shown in Figure 9. The target location for parking vehicle 10 will be the home parking lot shown by the No. 1 image button 62a of the automatic parking menu in Figure 4, and this will be explained below.

[0059] First, the external environment recognition unit 55 acquires recognition data of the external environment of the vehicle 10 captured by the front camera 12Fr, rear camera 12Rr, left side camera 12L, and right side camera 12R (step S21).

[0060] Next, the target location detection unit 57 determines whether or not it was able to detect the target location, that is, whether or not it was able to detect the registered target location, which is the parking lot of the home (step S22). As described above, the target location is detected by comparing the feature points of the external recognition data for the currently detected target location with the feature points of the external recognition data for the registered target location.

[0061] If the target position could not be detected in step S22 (step S22: No), the target position detection unit 57 returns to step S21 and repeats each process.

[0062] In step S22, if the target position is detected (step S22: Yes), the automatic driving control unit 56 performs self-position estimation processing of the vehicle 10 relative to the target position (step S23). As described above, the self-position estimation processing is performed by comparing feature points detected in the current external recognition data with the feature points of the registered target position to recognize the current relative position of the vehicle 10 with respect to the currently detected target position.

[0063] Next, the automatic driving control unit 56 determines whether or not it was able to estimate its own position in the self-position estimation process of step S23 (step S24).

[0064] If the self-position cannot be estimated in step S24 (step S24: No), the automatic driving control unit 56 terminates the automatic parking process. If the self-position cannot be estimated, the automatic driving control unit 56 may, for example, notify the user that the self-position relative to the target position cannot be estimated, and then, if the user so desires, execute the process of registering a new target position as described in Figure 5.

[0065] In step S24, if the self-position can be estimated (step S24: Yes), the automatic driving control unit 56 determines whether or not there are any objects in the vicinity of the target position in the current external recognition data acquired in step S21 that are not included in the feature points of the registered target position (step S25).

[0066] In step S25, if there are no targets that are not included in the feature points of the registered target position (step S25: No), the automatic driving control unit 56 automatically parks the vehicle 10 at the currently detected target position (step S26). This automatic parking process is the same as the process in step S12 in Figure 5 described above.

[0067] In step S25, if there is a target that is not included in the registered target position feature points (step S25: Yes), the automatic driving control unit 56 asks the user whether or not to perform automatic parking (step S27).

[0068] For example, as shown in Figure 10, suppose a target 82 that is not registered as a characteristic point of the registered target location, the home parking lot, is detected around that location. In this case, the automatic driving control unit 56 displays an inquiry screen 80 on the touch panel 42, as shown in Figure 10. The inquiry screen 80 displays an image 81 of the current target location (home parking lot) in which the detected target 82 is being captured, along with a notification message 83, "A registered target location has been detected," and an inquiry message 84, "Something that was not present during registration has been detected. Do you want to proceed with automatic parking?" The inquiry screen 80 also displays an "Automatic Parking" button 85 to initiate automatic parking of the vehicle 10, and a "Do Not Automatic Parking" button 86 to disable automatic parking. If the detected unregistered target 82 is recognized to be, for example, a "cat," an inquiry message 84 such as "A cat has been detected. Do you want to proceed with automatic parking?" may be displayed. The inquiry screen 80 may also be displayed on a HUD or smartphone. The messages may be output as audio from a speaker 44 or smartphone.

[0069] Next, the automatic driving control unit 56 determines whether or not it has received an actionable response from the user (touch operation of the "Automatically park" button 85) in response to the inquiry in step S27 (step S28).

[0070] If a response indicating that the action should be performed is received in step S28 (step S28: Yes), the automatic driving control unit 56 proceeds to step S26 and automatically parks the vehicle 10 at the currently detected target position. If a response indicating that the action should be performed is not received in step S28 (step S28: No), the automatic driving control unit 56 terminates the automatic parking process.

[0071] As explained above, when a target position is detected, the automatic driving control unit 56 of the control device determines whether it can estimate the vehicle 10's own position relative to the target position based on a comparison between the vehicle 10's recognition data of the outside world and the target position information, which is the characteristic points of the target position. If it can estimate its own position, it determines whether there are any objects 82 that are not included in the target position information that exist around the target position. If it determines that objects 82 exist, it notifies the user of the vehicle 10 to ask whether it is necessary to perform automatic parking control to the target position. If it determines that there are no objects, it performs automatic parking control.

[0072] With this configuration, even if the vehicle 10 can estimate its own position relative to the target position, if there are objects around the target position that are not included in the target position information, the system will ask the user whether or not to perform automatic parking control. This ensures the safety of automatic parking control. Furthermore, even if there are objects in the external recognition data that are not included in the target position information, if the system receives a command from the user to perform automatic parking control in response to the inquiry, it is possible to perform automatic parking control. Therefore, unlike, for example, when automatic parking is performed based solely on image similarity, it is possible to prevent situations where automatic driving control cannot be performed at registered target positions where feature points are stored. This increases the opportunities to perform automatic driving control at registered target positions, thereby improving usability. In addition, if there are no objects in the external recognition data that are not included in the target position information, automatic driving control can be performed seamlessly without asking the user, thus reducing the burden on the user and improving usability.

[0073] Next, a modified example of the automatic parking of vehicle 10 to a registered target position will be described with reference to Figures 11 and 12.

[0074] [First variation] Figure 11 is a flowchart showing a first modified example of the process for automatically parking vehicle 10 at a registered target location. In this first modified example, as in Figure 9, vehicle 10 is parked in the home parking lot shown by the No. 1 image button 62a.

[0075] As shown in Figure 11, in the first modified example, each process from step S21 to step S24 is the same as each process from step S21 to step S24 in the automatic parking process described in Figure 9. Therefore, the explanation of steps S21 to step S24 will be omitted.

[0076] In step S24, if the self-position can be estimated (step S24: Yes), the automatic driving control unit 56 detects the user's current gaze during the current automatic parking and determines whether the detected user's gaze matches the user's gaze (gaze history information) during past automatic parking (step S25a).

[0077] For example, before the vehicle 10 is stopped and automatic parking begins, the system detects where the user is looking and determines whether the direction and position the user is looking in matches the direction and position the user was looking in during previous automatic parking operations. In this case, the automatic driving control unit 56 extracts historical information of the user's gaze history that matches the current relative position of the vehicle 10 to the target position of the vehicle 10, and determines whether the user's gaze matches. For example, when the vehicle 10 is backing into a target position to the left rear, the system extracts historical information of the same relative position from the storage unit 54 to determine whether the gaze matches.

[0078] In step S25a, if the user's gaze matches the history information (step S25a: Yes), the automatic driving control unit 56 determines that there are no objects in the vicinity of the registered target location, which is the home parking lot, that differ from the recognized registered feature points, and automatically parks the vehicle 10 at the currently detected target location (step S26). The automatic parking process is the same as the process in step S12 in Figure 5 described above.

[0079] In step S25a, if the user's gaze does not match the history information (step S25a: No), the automatic driving control unit 56 determines that there is a different object from the recognized registered feature point around the registered target location, which is the home parking lot, and asks the user whether or not to perform automatic parking (step S27).

[0080] The method of querying in step S27 (see Figure 10) and the processing in step S28 are the same as the processes in steps S27 and S28 described in Figure 9. Therefore, the explanation of steps S27 and S28 will be omitted.

[0081] Thus, according to this first modified example of automatic parking control, the automatic driving control unit 56 determines whether or not a target exists based on a comparison of the gaze detection result, which detects the user's current gaze during the current automatic parking, with the user's gaze history information from past automatic parking operations. For this reason, for example, if the current gaze deviates significantly from the representative value (e.g., the average value of the gaze history) of the user's gaze history from past automatic parking operations at the same target location, it is possible to determine that "a target exists." This makes it possible to accurately detect targets that were not present during past automatic parking operations, depending on the user's characteristics (such as habits) when automatically parking the vehicle 10 at a registered target location.

[0082] Furthermore, the automatic driving control unit 56 extracts historical information from the gaze history information that corresponds to the relative position between the target position and the vehicle 10, and determines whether or not a target exists based on a comparison of the extracted historical information with the detection result of the current user's gaze. In this way, the presence or absence of a target can be determined by narrowing down the user's gaze history from past automatic parking operations at the same target position to histories where the relative position between the target position and the vehicle 10 is close to the current relative position. This makes it possible to determine the presence or absence of a target with even greater accuracy.

[0083] [Second variation] Figure 12 is a flowchart showing a second modified example of the process for automatically parking vehicle 10 at a registered target location. In the second modified example, as in Figure 9, vehicle 10 is parked in the home parking lot shown by the No. 1 image button 62a.

[0084] As shown in Figure 12, in the second modified example, each process from step S21 to step S24 is the same as each process from step S21 to step S24 in the automatic parking process described in Figure 9. Therefore, the explanation of steps S21 to S24 will be omitted.

[0085] In step S24, if the self-position can be estimated (step S24: Yes), the automatic driving control unit 56 determines whether the automatic parking history information satisfies predetermined conditions (step S25b). The history information satisfies predetermined conditions, which means that the number of times or frequency of use by users to automatically park the vehicle 10 at the target position is low.

[0086] In step S25b, if the history information does not meet the predetermined conditions (step S25b: No), the automatic driving control unit 56 determines whether or not there are any objects in the vicinity of the target location in the current external recognition data acquired in step S21 that are not included in the feature points of the registered target location (step S25c). Alternatively, instead of the process in step S25c, the process in step S25a of the first modified example described above (Figure 11) may be executed.

[0087] In step S25b, if the history information meets the predetermined conditions (step S25b: Yes), the automatic driving control unit 56 asks the user whether or not to perform automatic parking (step S27). That is, if the user has used the automatic parking function to automatically park the vehicle 10 at the target location infrequently, the automatic driving control unit 56 asks the user whether or not to perform automatic parking, regardless of whether or not there are any objects that are not included in the registered target location feature points.

[0088] Note that the processes from step S25c onward and from step S27 onward are the same as those described in Figure 9. Therefore, the explanation of the processes from step S25c onward and from step S27 onward will be omitted.

[0089] Thus, according to this second modified example of automatic parking control, even if the automatic driving control unit 56 can estimate the vehicle 10's own position relative to the target position, if the history information of automatic parking control relative to the target position satisfies predetermined conditions, it will notify the user of the vehicle 10 to ask whether or not to perform automatic parking control, even if there are no targets that are not included in the registered feature points of the target position. This allows the system to notify users whether or not to perform automatic parking control, rather than seamlessly executing it for users who are using automatic parking control for the first time or who have used it infrequently, and then execute the automatic parking control based on the user's response. Therefore, it is possible to prevent situations where automatic parking control is executed seamlessly and causes anxiety to the user.

[0090] Although embodiments of the present invention have been described above, the present invention is not limited to the above embodiments, and modifications, improvements, etc., can be made as appropriate.

[0091] For example, the above embodiment described a case in which the vehicle 10 is automatically parked by the automatic steering of the automatic driving control unit 56, but the present invention is not limited to this. For example, the present invention may be applied to auxiliary parking assistance, which assists when the driver parks the vehicle 10.

[0092] Furthermore, in the above embodiment, the parking position of the vehicle 10 and the characteristic points of the parking position and its surroundings are explained using overhead images (composite images) from the front camera 12Fr, rear camera 12Rr, left-side camera 12L, and right-side camera 12R, but the embodiment is not limited to this. For example, the explanation may be given using an image captured by any of the cameras: the front camera 12Fr, rear camera 12Rr, left-side camera 12L, and right-side camera 12R.

[0093] Furthermore, although the above embodiment describes a case in which the automatic driving control unit 56 displays the overhead image 71, etc., on the touch panel 42 of the vehicle 10, it is not limited to this. For example, the automatic driving control unit 56 may display the overhead image 71, etc., on the display screen of an information terminal (e.g., a smartphone, etc.) held by the passenger of the vehicle 10 via the communication unit 24.

[0094] Furthermore, while the above embodiment described a case in which information such as a map containing feature points related to the target location is registered in the storage unit 54 of the vehicle 10, the invention is not limited to this. For example, this information may be registered in the storage unit of another device such as a smartphone or server that is communicatively connected to the vehicle 10.

[0095] Furthermore, although the above embodiment describes an example in which the moving object is a vehicle (a four-wheeled automobile), it is not limited to this. For example, it may be a motorcycle, a Segway, or other type of vehicle. Moreover, the concept of the present invention is not limited to vehicles, but can also be applied to robots, ships, aircraft, etc., that are equipped with a drive source and are movable by the power of the drive source.

[0096] The control method described in the above-mentioned embodiment can be implemented by executing a pre-prepared control program on a computer. This control program is recorded on a computer-readable storage medium and executed when read from the storage medium. This control program may also be provided in the form of a non-transient storage medium such as flash memory, or it may be provided via a network such as the Internet. The computer that executes this control program may be included in the control device, included in an electronic device such as a smartphone, tablet terminal, or personal computer that can communicate with the control device, or included in a server device that can communicate with these control devices and electronic devices.

[0097] Furthermore, this specification includes at least the following information. Note that the components etc. in parentheses indicate those corresponding to the embodiments described above, but are not limited thereto.

[0098] (1) An external environment recognition unit (external environment recognition unit 55) that acquires external environment recognition data of the moving object (vehicle 10), An automatic driving control unit (automatic driving control unit 56) performs automatic driving control to move the moving body to a target position, A storage unit (storage unit 54) that stores target location information related to the aforementioned target location, The system includes a target position detection unit (target position detection unit 57) that detects the target position based on the recognition data, The automatic driving control unit, When the aforementioned target position is detected, Based on a comparison of the recognition data and the target position information, a first determination process is performed to determine whether or not the moving body can estimate its own position relative to the target position. If the self-position estimation is successful in the first determination process, a second determination process is executed to determine whether or not a target (target 82) not included in the target position information exists in the vicinity of the target position. If the second determination process determines that the target exists, the user of the mobile body is notified to inquire whether or not to perform the automatic driving control to the target position. If the second determination process determines that the target does not exist, the automatic driving control is executed. Control device.

[0099] According to (1), even if the moving object's self-position estimation is possible, if there are objects in the vicinity of the target location that are not included in the target location information, the system will ask the user whether or not to execute the automated driving control, thereby improving the safety of the automated driving control. Furthermore, even if there are objects in the external recognition data that are not included in the target location information, if the system receives an instruction from the user to execute the automated driving control in response to the above inquiry, it is possible to execute the automated driving control. Unlike conventional technologies that perform automated parking based solely on image similarity, this prevents situations where automated driving control cannot be executed at locations where the target location information has already been stored. As a result, the opportunities to execute automated driving control can be increased, improving usability. In addition, if there are no objects in the external recognition data that are not included in the target location information, automated driving control can be executed seamlessly (without asking the user), thus reducing the burden on the user.

[0100] (2) The control device described in (1), The aforementioned target location information is a characteristic point of the target location. Control device.

[0101] According to (2), in order to detect an object that is not included in the target location information, it is preferable to compare the feature points of the target location.

[0102] (3) A control device as described in (1) or (2), If the automatic driving control unit determines in the second determination process that the target exists, it will issue a notification to inquire whether or not to execute the automatic driving control, and then execute the automatic driving control in response to the user of the moving body. Control device.

[0103] According to (3), by executing autonomous driving control in response to user responses, it is possible to suppress situations in which autonomous driving control cannot be executed at locations where target location information has already been stored, thereby improving usability.

[0104] (4) A control device according to any one of (1) to (3), The automatic driving control unit does not execute the automatic driving control if it is unable to estimate its own position in the first determination process. Control device.

[0105] According to (4), a high level of safety can be maintained in autonomous driving control.

[0106] (5) The control device described in (4), If the self-position estimation fails in the first determination process, the automatic driving control unit receives a selection of the target position for the automatic driving control from the user of the moving body. Control device.

[0107] According to (5), the system can accept the designation of a parking location and register it as a new target location, thereby improving usability.

[0108] (6) A control device according to any one of (1) to (5), In the second determination process, the automatic driving control unit determines whether or not the target exists based on a comparison of the user's gaze detection result and the user's gaze history information. Control device.

[0109] According to (6), for example, if the current gaze deviates significantly from the representative value (e.g., the average value) of the user's gaze history when the system automatically parked at the same target location in the past, it can be determined that "a target is present." Therefore, the presence or absence of a target can be determined with high accuracy according to the user's characteristics (such as habits).

[0110] (7) The control device described in (6), In the second determination process, the automatic driving control unit extracts history information from the gaze history information that corresponds to the relative position of the target position and the moving object, and determines whether or not the target exists based on a comparison of the gaze detection result and the extracted history information. Control device.

[0111] According to (7), the history of the user's gaze when the vehicle automatically parked at the same target location in the past can be narrowed down to history where the relative position of the target location and the moving object is close to the current relative position, and the presence or absence of a target can be determined with greater accuracy.

[0112] (8) A control device according to any one of (1) to (7), If the automatic driving control unit has been able to estimate its own position in the first determination process, and the history information of the automatic driving control for the target position satisfies predetermined conditions, it will notify the user of the moving body to inquire whether or not the automatic driving control should be executed, even if the target does not exist. Control device.

[0113] According to (8), for users who are using autonomous driving control for the first time or who have used it infrequently, instead of seamlessly executing autonomous driving control, a notification is sent to ask whether or not it is necessary to execute it, and then autonomous driving control is executed based on the user's response. This prevents situations where autonomous driving control is executed seamlessly and causes anxiety to the user, thereby improving usability.

[0114] (9) A control device according to any one of (1) to (8), The aforementioned target position is the parking position of the moving body. The aforementioned automatic driving control is automatic parking control to the parking position. Control device.

[0115] According to (9), by applying this to automatic parking control of a moving object to a parking position, usability can be improved.

[0116] (10) A control method by a control device that acquires recognition data of the external environment of a moving object, performs automatic driving control to move the moving object to a target position, stores target position information related to the target position, and detects the target position based on the recognition data, The processor of the control device When the aforementioned target position is detected, Based on a comparison of the recognition data and the target position information, a first determination process is performed to determine whether or not the moving body can estimate its own position relative to the target position. If the self-position estimation is successful in the first determination process, a second determination process is executed to determine whether or not there are targets not included in the target position information in the vicinity of the target position. If the second determination process determines that the target exists, the user of the mobile body is notified to inquire whether or not to perform the automatic driving control to the target position. If the second determination process determines that the target does not exist, the automatic driving control is executed. Control method.

[0117] According to (10), even if the moving object's own position can be estimated, if there are objects in the vicinity of the target position that are not included in the target position information, the system will ask the user whether or not to execute the automated driving control, thereby improving the safety of the automated driving control. Furthermore, even if there are objects in the external recognition data that are not included in the target position information, if the system receives an instruction from the user to execute the automated driving control in response to the above inquiry, it is possible to execute the automated driving control, thus preventing situations where automated driving control cannot be executed in locations where the target position information has already been stored. As a result, the opportunities to execute automated driving control can be increased, improving usability. In addition, if there are no objects in the external recognition data that are not included in the target position information, automated driving control can be executed seamlessly, thus reducing the burden on the user.

[0118] (11) A control program for a control device that acquires recognition data of the external environment of a moving object, performs automatic driving control to move the moving object to a target position, stores target position information related to the target position, and detects the target position based on the recognition data, The processor of the control device, When the aforementioned target position is detected, Based on a comparison of the recognition data and the target position information, a first determination process is performed to determine whether or not the moving body can estimate its own position relative to the target position. If the self-position estimation is successful in the first determination process, a second determination process is executed to determine whether or not there are targets not included in the target position information in the vicinity of the target position. If the second determination process determines that the target exists, the user of the mobile body is notified to inquire whether or not to perform the automatic driving control to the target position. If the second determination process determines that the target does not exist, the automatic driving control is executed. A control program for executing a process.

[0119] According to (11), even if the moving object's self-position estimation is possible, if there are objects in the vicinity of the target location that are not included in the target location information, the system will ask the user whether or not to execute the automated driving control, thereby improving the safety of the automated driving control. Furthermore, even if there are objects in the external recognition data that are not included in the target location information, if the system receives an instruction from the user to execute the automated driving control in response to the above inquiry, it is possible to execute the automated driving control, thus preventing situations where automated driving control cannot be executed in locations where the target location information has already been stored. As a result, the opportunities to execute automated driving control can be increased, improving usability. In addition, if there are no objects in the external recognition data that are not included in the target location information, automated driving control can be executed seamlessly, thus reducing the burden on the user. [Explanation of Symbols]

[0120] 10. Vehicles (mobile devices) 20 Control ECU (Control Unit) 54 Storage section 55 External world recognition department 56 Automated Driving Control Unit 57 Target position detection unit

Claims

1. An external environment recognition unit that acquires recognition data of the external environment of a moving object, An automatic driving control unit that performs automatic driving control to move the moving body to a target position, A storage unit that stores target location information related to the aforementioned target location, The system includes a target position detection unit that detects the target position based on the recognition data, The automatic driving control unit, When the aforementioned target position is detected, Based on a comparison of the recognition data and the target position information, a first determination process is performed to determine whether or not the moving body's own position can be estimated relative to the target position. If the self-position estimation fails in the first determination process, the user of the moving body selects the target position for the automatic driving control. Control device.

2. A control device according to claim 1, The storage unit stores the target location information relating to the registered target location registered by the user. The target position detection unit detects the registered target positions and unregistered target positions that have not been registered by the user. Control device.

3. A control device according to claim 2, If the self-position estimation fails in the first determination process, the automatic driving control unit receives a specification from the user for a target position to which the moving body will move from the unregistered target position. Control device.

4. A control device according to claim 2, If the automatic driving control unit fails to estimate its own position in the first determination process, it receives a request from the user to specify a new target position to be registered from the unregistered target position. Control device.

5. A control device according to claim 3 or 4, If the automatic driving control unit fails to estimate its own position in the first determination process, it will accept a designation from the user without suspending the automatic driving control. Control device.

6. A control method using a control device that acquires recognition data of the external environment of a moving object, performs automatic driving control to move the moving object to a target position, stores target position information related to the target position, and detects the target position based on the recognition data, The processor of the control device When the aforementioned target position is detected, Based on a comparison of the recognition data and the target position information, a first determination process is performed to determine whether or not the moving body's own position can be estimated relative to the target position. If the self-position estimation fails in the first determination process, the user of the moving body selects the target position for the automatic driving control. Control method.

7. A control program for a control device that acquires recognition data of the external environment of a moving object, performs automatic driving control to move the moving object to a target position, stores target position information related to the target position, and detects the target position based on the recognition data, The processor of the control device, When the aforementioned target position is detected, Based on a comparison of the recognition data and the target position information, a first determination process is performed to determine whether or not the moving body's own position can be estimated relative to the target position. If the self-position estimation fails in the first determination process, the user of the moving body selects the target position for the automatic driving control. A control program for executing a process.

Images