Mobile body control device, mobile body control method, and mobile body control program

Abstract

To provide a movable body control device which can accurately perform automatic parking at a specified parking position even when external field environment (illumination intensity in a time zone) of the movable body is changed.SOLUTION: A movable body control device comprises: a front camera 12Fr, a rear camera 12Rr, a left side camera 12L, and a right side camera 12R which acquire recognition data of external field of a vehicle 10; a touch panel 42 which receives specification of a prescribed parking position from a user of the vehicle 10; and an automatic parking control part 55 which performs an automatic parking control that makes the vehicle 10 park at the prescribed parking position based on the recognition data of the external field and the prescribed parking position and registers the prescribed parking position as a specified parking position. The automatic parking control part 55 registers characteristic points about the specified parking position by a plurality of slot patterns based on illumination intensity of the external field calculated from the recognition data of the external field.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present invention relates to a mobile object control device, a mobile object control method, and a mobile object control program. [Background technology]

[0002] In recent years, efforts to provide access to sustainable transportation systems that take into consideration vulnerable transport participants have been gaining momentum. To achieve this, we are focusing on research and development into autonomous driving technology to further improve traffic safety and convenience.

[0003] Conventionally, automatic parking control that automatically moves and parks a vehicle in a designated parking space has been known. Patent Document 1 describes a parking assistance device that performs manual parking by allowing a user to drive the vehicle and park it in a designated parking space, acquires feature point information around the vehicle during the manual parking and stores it in a memory, and performs automatic parking control based on the feature point information. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2020-117128 Summary of the Invention [Problem to be solved by the invention]

[0005] Even when parking a vehicle in the same parking space, differences may occur in the acquired feature point information around the vehicle due to, for example, differences in the environment around the vehicle. Therefore, if there is a discrepancy between the feature point information stored in memory and the feature point information at the time of the next automatic parking execution, the target parking space may not be recognized, and automatic parking control may not be possible. However, Patent Document 1 does not describe how to control the vehicle when differences occur in the acquired feature point information around the vehicle due to differences in the environment. Therefore, there is room for improvement in the prior art in this regard.

[0006] The present invention aims to provide a mobile object control device, a mobile object control method, and a mobile object control program that are capable of automatically parking a mobile object at a designated parking position with high accuracy even if the external environment of the mobile object changes, thereby contributing to the development of a sustainable transportation system. [Means for solving the problem]

[0007] The present invention provides an external environment recognition unit that acquires recognition data of the external environment of the moving object; a reception unit that receives a designation of a predetermined parking position from a user of the mobile body; a control unit that performs automatic parking control to park the mobile body at the predetermined parking position based on the external recognition data and the predetermined parking position, and registers the predetermined parking position as a designated parking position, The control unit calculates the illuminance of the external environment based on the recognition data of the external environment. For each of the above illuminance Registering feature points related to the specified parking position using multiple patterns death , If there is no specific pattern among the plurality of patterns of the feature points that corresponds to the illuminance acquired from the external environment recognition data when the automatic parking control to the designated parking position is executed, a notification is given to prompt the user to register the specific pattern. It is a mobile object control device.

[0008] The present invention provides A processor of the mobile control device Acquires recognition data of the external world of the moving object, Accepting a designation of a predetermined parking position from a user of the mobile body; performing automatic parking control for parking the mobile body at the predetermined parking position based on the external recognition data and the predetermined parking position, and registering the predetermined parking position as a designated parking position; Based on the illuminance of the outside world calculated from the recognition data of the outside world For each of the above illuminance Registering feature points related to the specified parking position using multiple patterns death , If there is no specific pattern among the plurality of patterns of the feature points that corresponds to the illuminance acquired from the external environment recognition data when the automatic parking control to the designated parking position is executed, a notification is given to prompt the user to register the specific pattern. A mobile object control method.

[0009] The present invention provides A processor of the mobile control device Acquires recognition data of the external world of the moving object, Accepting a designation of a predetermined parking position from a user of the mobile body; performing automatic parking control for parking the mobile body at the predetermined parking position based on the external recognition data and the predetermined parking position, and registering the predetermined parking position as a designated parking position; Based on the illuminance of the outside world calculated from the recognition data of the outside world For each of the above illuminance Registering feature points related to the specified parking position using multiple patterns death , If there is no specific pattern among the plurality of patterns of the feature points that corresponds to the illuminance acquired from the external environment recognition data when the automatic parking control to the designated parking position is executed, a notification is given to prompt the user to register the specific pattern. This is a mobile object control program for executing the process. [Effects of the Invention]

[0010] According to the mobile body control device, mobile body control method, and mobile body control program of the present invention, even if the external environment of the mobile body changes, it is possible to automatically park the mobile body at a designated parking position with high accuracy. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a side view showing an example of a vehicle equipped with a mobile object control device according to an embodiment of the present invention. [Figure 2] FIG. 2 is a top view of the vehicle shown in FIG. [Figure 3] FIG. 2 is a block diagram showing the internal configuration of the vehicle shown in FIG. [Figure 4] FIG. 10 is a diagram showing an example of an image displayed on the navigation device when the vehicle is automatically parked. [Figure 5] 10 is a flowchart showing an automatic parking control for parking a vehicle at a new parking position. [Figure 6] FIG. 10 is a diagram showing a parking position set on an overhead image generated based on an image captured by a camera. [Figure 7] FIG. 7 is a diagram showing characteristic points in the vicinity of the parking position shown in FIG. 6. [Figure 8] FIG. 10 is a diagram showing an example of registration content registered for a parking position. [Figure 9] FIG. 9 is a diagram showing an example of a map included in the registered contents of FIG. 8. [Figure 10] 10 is a flowchart illustrating an example of automatic parking control for parking a vehicle at a registered parking position. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, an embodiment of a mobile object control device, a mobile object control method, and a mobile object control program according to the present invention will be described with reference to the accompanying drawings. The drawings are to be viewed in accordance with the directions of the reference numerals. For simplicity and clarity of explanation, the front-rear, left-right, and up-down directions in this specification and the like are described according to the directions as seen by the driver of the vehicle 10 shown in Figures 1 and 2. In the drawings, the front of the vehicle 10 is indicated as Fr, the rear as Rr, the left as L, the right as R, the top as U, and the bottom as D.

[0013] <Vehicle 10 equipped with the mobile body control device of the present invention> Fig. 1 is a side view of a vehicle 10 equipped with a mobile object control device of the present invention. Fig. 2 is a top view of the vehicle 10 shown in Fig. 1. The vehicle 10 is an example of a mobile object of the present invention.

[0014] The vehicle 10 is an automobile having a drive source (not shown) and wheels including drive wheels driven by the power of the drive source and steerable wheels. In this embodiment, the vehicle 10 is a four-wheel automobile having a pair of left and right front wheels and rear wheels. The drive source of the vehicle 10 is, for example, an electric motor. The drive source of the vehicle 10 may be an internal combustion engine such as a gasoline engine or a diesel engine, or may be a combination of an electric motor and an internal combustion engine. The drive source of the vehicle 10 may drive a pair of left and right front wheels, a pair of left and right rear wheels, or all four wheels, including a pair of left and right front wheels and rear wheels. Both the front wheels and the rear wheels may be steerable wheels, or either one of them may be steerable wheels.

[0015] The vehicle 10 further includes side mirrors 11L and 11R. The side mirrors 11L and 11R are mirrors (rearview mirrors) provided on the outside of the front doors of the vehicle 10 to enable the driver to check what is behind and to the rear sides. The side mirrors 11L and 11R are fixed to the body of the vehicle 10 by a rotation shaft that extends vertically, and can be opened and closed by rotating around this rotation shaft.

[0016] The vehicle 10 further includes a front camera 12Fr, a rear camera 12Rr, a left side camera 12L, and a right side camera 12R. The front camera 12Fr is a digital camera provided in front of the vehicle 10 and captures an image in front of the vehicle 10. The rear camera 12Rr is a digital camera provided in the rear of the vehicle 10 and captures an image in rear of the vehicle 10. The left side camera 12L is a digital camera provided in the left side mirror 11L of the vehicle 10 and captures an image in the left side of the vehicle 10. The right side camera 12R is a digital camera provided in the right side mirror 11R of the vehicle 10 and captures an image in the right side of the vehicle 10. The front camera 12Fr, the rear camera 12Rr, the left side camera 12L, and the right side camera 12R are examples of an external environment recognition unit of the present invention.

[0017] <Internal configuration of vehicle 10> Fig. 3 is a block diagram showing an example of the internal configuration of vehicle 10 shown in Fig. 1. As shown in Fig. 3, vehicle 10 has a sensor group 16, a navigation device 18, a control ECU (Electronic Control Unit) 20, an EPS (Electric Power Steering) system 22, and a communication unit 24. Vehicle 10 also has a driving force control system 26 and a braking force control system 28.

[0018] The sensor group 16 acquires various detection values ​​used for control by the control ECU 20. The sensor group 16 includes a front camera 12Fr, a rear camera 12Rr, a left side camera 12L, and a right side camera 12R. The sensor group 16 also includes a front sonar group 32a, a rear sonar group 32b, a left side sonar group 32c, and a right side sonar group 32d. The sensor group 16 also includes wheel sensors 34a and 34b, a vehicle speed sensor 36, and an operation detection unit 38.

[0019] The front camera 12Fr, rear camera 12Rr, left side camera 12L, and right side camera 12R capture images of the periphery of the vehicle 10 to obtain recognition data (e.g., peripheral images) for recognizing the external environment of the vehicle 10. The peripheral images captured by the front camera 12Fr, rear camera 12Rr, left side camera 12L, and right side camera 12R are referred to as a front image, a rear image, a left side image, and a right side image, respectively. An image composed of the left side image and the right side image may also be referred to as a side image.

[0020] The front sonar group 32a, rear sonar group 32b, left side sonar group 32c, and right side sonar group 32d emit sound waves around the vehicle 10 and receive reflected sound from other objects. The front sonar group 32a includes, for example, four sonars. The sonars that make up the front sonar group 32a are provided diagonally forward left, front left, front right, and diagonally forward right of the vehicle 10, respectively. The rear sonar group 32b includes, for example, four sonars. The sonars that make up the rear sonar group 32b are provided diagonally rear left, rear left, rear right, and diagonally rear right of the vehicle 10, respectively. The left side sonar group 32c includes, for example, two sonars. The sonars that make up the left side sonar group 32c are provided respectively on the front left side and rear left side of the vehicle 10. The right side sonar group 32d includes, for example, two sonars. The sonars that make up the right side sonar group 32d are provided at the front and rear of the right side of the vehicle 10, respectively.

[0021] The wheel sensors 34a, 34b detect the rotation angles of the wheels of the vehicle 10. The wheel sensors 34a, 34b may be configured as angle sensors or displacement sensors. The wheel sensors 34a, 34b output a detection pulse each time the wheel rotates a predetermined angle. The detection pulses output from the wheel sensors 34a, 34b are used to calculate the wheel rotation angle and wheel rotation speed. The travel distance of the vehicle 10 is calculated based on the wheel rotation angle. The wheel sensor 34a detects, for example, the rotation angle θa of the left rear wheel. The wheel sensor 34b detects, for example, the rotation angle θb of the right rear wheel.

[0022] The vehicle speed sensor 36 detects the speed of the body of the vehicle 10, that is, the vehicle speed V, and outputs the detected vehicle speed V to the control ECU 20. The vehicle speed sensor 36 detects the vehicle speed V based on, for example, the rotation of a countershaft of the transmission.

[0023] The operation detection unit 38 detects the operation content performed by the user using the operation input unit 14, and outputs the detected operation content to the control ECU 20. The operation input unit 14 includes various user interfaces, such as a side mirror switch that switches the open / closed state of the side mirrors 11L, 11R, and a shift lever (selector lever or selector).

[0024] The navigation device 18 detects the current position of the vehicle 10 using, for example, a GPS (Global Positioning System), and provides the user with guidance on a route to the destination. The navigation device 18 has a storage device (not shown) that has a map information database.

[0025] The navigation device 18 is equipped with a touch panel 42 and a speaker 44. The touch panel 42 functions as an input device and a display device for the control ECU 20. The speaker 44 outputs various types of guidance information to passengers of the vehicle 10 by voice.

[0026] The touch panel 42 is configured to allow various commands to be input to the control ECU 20. For example, a user can input commands related to parking assistance for the vehicle 10 via the touch panel 42. The touch panel 42 is also configured to display various screens related to the control details of the control ECU 20. For example, the touch panel 42 displays a screen related to parking assistance for the vehicle 10. Specifically, the touch panel 42 displays a parking assistance button for requesting parking assistance for the vehicle 10. The parking assistance button includes an automatic parking button for requesting parking using automatic steering by the control ECU 20, and a parking assistance button for requesting assistance when parking using driver operation. Note that components other than the touch panel 42, such as a smartphone, may be used as the input device or display device. The touch panel 42 (navigation device 18) and the smartphone are examples of a reception unit of the present invention.

[0027] Note that "parking" is synonymous with "parking," for example. For example, "parking" refers to a stop that involves passengers getting in and out of the vehicle, and does not include temporary stops at traffic signals, etc. Also, a "parking position" refers to a position where a mobile object is stopped, i.e., a parking position.

[0028] The control ECU 20 has an input / output unit 50, a calculation unit 52, and a storage unit 54. The calculation unit 52 is configured by, for example, a CPU (Central Processing Unit). The calculation unit 52 performs various controls by controlling each unit based on a program stored in the storage unit 54. The calculation unit 52 also inputs and outputs signals to and from each unit connected to the control ECU 20 via the input / output unit 50.

[0029] The calculation unit 52 has an automatic parking control unit 55 for controlling automatic parking of the vehicle 10. The automatic parking control unit 55 is an example of a control unit of the present invention. The automatic parking control unit 55 performs parking assistance for the vehicle 10 by automatic steering, which automatically operates the steering wheel 110 under the control of the automatic parking control unit 55. With the assistance of automatic steering, the operation of the accelerator pedal (not shown), brake pedal (not shown), and operation input unit 14 is performed automatically. In addition, the automatic parking control unit 55 provides auxiliary assistance when the driver operates the accelerator pedal, brake pedal, and operation input unit 14 to manually park the vehicle 10.

[0030] For example, the automatic parking control unit 55 controls automatic parking to park the vehicle 10 in a specified parking position based on the recognition data of the outside world of the vehicle 10 acquired by the front camera 12Fr, the rear camera 12Rr, the left side camera 12L, and the right side camera 12R, and the specified parking position specified by the user.

[0031] Furthermore, the automatic parking control unit 55 registers a predetermined parking position designated by the user as a designated parking position in the storage unit 54. The automatic parking control unit 55 calculates the illuminance of the outside world from the recognition data of the outside world of the vehicle 10, and generates, for example, a plurality of slots (time slots) according to the calculated illuminance of the outside world. The automatic parking control unit 55 registers the designated parking positions by associating feature points related to the designated parking positions for each slot.

[0032] The illuminance calculated from the external recognition data refers to the actual illuminance of the external environment, not the illuminance of the image, which changes depending on the exposure setting, etc. Illuminance refers to the brightness of the environment, which changes depending on, for example, the time of day (daytime or nighttime, etc.) or the weather (sunny or cloudy, etc.). The external recognition data used to calculate the illuminance are images of the surroundings of the vehicle 10 captured by the front camera 12Fr, rear camera 12Rr, left side camera 12L, and right side camera 12R. However, values ​​detected by an illuminance sensor may also be used as the illuminance. Furthermore, illuminance is not limited to numerical values ​​in a specific unit system, such as lux (Lx). Feature points related to the designated parking position include feature points of the designated parking position itself and feature points around the designated parking position. Feature points of the designated parking position itself include, for example, the word "parking" displayed within the designated parking position. Feature points around the designated parking position include, for example, distinctive buildings and obstacles in the vicinity.

[0033] Furthermore, when executing automatic parking control to a designated parking position designated by the user, the automatic parking control unit 55 acquires the illuminance of the outside world from the external recognition data, and selects a map containing feature points related to the designated parking position that is registered in association with a slot corresponding to that illuminance from the registered data in the storage unit 54. The slot corresponding to that illuminance refers to a slot with an illuminance that matches or is close to the illuminance acquired at the time of execution.

[0034] Furthermore, the automatic parking control unit 55 selects a specific slot pattern consisting of slots for which a map is registered, and determines whether the vehicle 10 is able to recognize its own position based on that slot pattern. Self-position recognition refers to recognizing the relative position of the vehicle 10 with respect to the designated parking position by comparing a map including feature points related to the registered designated parking position with the current camera image. If the automatic parking control unit 55 is able to recognize the vehicle 10's own position, it controls automatic parking to the designated parking position based on the selected specific slot pattern. If the automatic parking control unit 55 is not able to recognize the vehicle 10's own position, it does not control automatic parking to the designated parking position based on the selected specific slot pattern.

[0035] Furthermore, if a map including feature points related to the designated parking position is not registered in the slot corresponding to the external illuminance and no specific slot pattern exists, the automatic parking control unit 55 determines whether self-position recognition of the vehicle 10 is possible based on any of the other slot patterns for which maps are registered. If self-position recognition is possible, the automatic parking control unit 55 performs automatic parking control to the designated parking position based on that slot pattern, and if self-position recognition is not possible, does not perform automatic parking control to the designated parking position.

[0036] In addition, if the automatic parking control unit 55 cannot recognize the vehicle 10's own position based on any of the registered slot patterns, it accepts manual setting of a specified parking position from the user of the vehicle 10 and generates and registers a new slot pattern with a registered map.

[0037] The EPS system 22 includes a steering angle sensor 100, a torque sensor 102, an EPS motor 104, a resolver 106, and an EPS ECU 108. The steering angle sensor 100 detects the steering angle θst of a steering wheel 110. The torque sensor 102 detects the torque TQ applied to the steering wheel 110.

[0038] The EPS motor 104 applies a driving force or a reaction force to a steering column 112 connected to a steering wheel 110, thereby enabling assistance to the occupant in operating the steering wheel 110 and automatic steering during parking assistance. A resolver 106 detects the rotation angle θm of the EPS motor 104. The EPS ECU 108 is responsible for overall control of the EPS system 22. The EPS ECU 108 is equipped with an input / output unit (not shown), a calculation unit (not shown), and a storage unit (not shown).

[0039] The communication unit 24 enables wireless communication with other communication devices 120. The other communication devices 120 include base stations, communication devices of other vehicles, and information terminals such as smartphones carried by passengers of the vehicle 10.

[0040] The driving force control system 26 is equipped with a driving ECU 130. The driving force control system 26 executes driving force control of the vehicle 10. The driving ECU 130 controls the driving force of the vehicle 10 by controlling an engine (not shown) and the like based on a user's operation of an accelerator pedal (not shown).

[0041] The braking force control system 28 is equipped with a braking ECU 132. The braking force control system 28 executes braking force control of the vehicle 10. The braking ECU 132 controls the braking force of the vehicle 10 by controlling a brake mechanism (not shown) and the like based on a user's operation of a brake pedal (not shown).

[0042] <Automatic Parking Control by Automatic Parking Control Unit 55> Next, the automatic parking control of the vehicle 10 by the automatic parking control unit 55 will be described with reference to FIGS.

[0043] 4 is a diagram showing an example of an "automatic parking menu" that is displayed on the touch panel 42 of the navigation device 18 when automatically parking the vehicle 10. The image of the automatic parking menu is displayed by touching an automatic parking button 60 that is displayed on the touch panel 42. For example, when a user who intends to park the vehicle 10 approaches a parking lot, the automatic parking button 60 is touched to display the automatic parking menu.

[0044] As shown in FIG. 4, the image of the automatic parking menu displays a new registration button 61 and a registered parking position image button 62. The new registration button 61 is a button that is operated when parking the vehicle 10 at a parking position that is to be newly registered as a designated parking position. The registered parking position image button 62 is a button that is operated when parking the vehicle 10 at a designated parking position that has already been registered. Examples of the registered parking position image button 62 include a parking position image button in which a home parking lot is registered as a designated parking position, as shown in No. 1 image button 62a, and a parking position image button in which a frequently parked coin parking lot is registered as a designated parking position, as shown in No. 2 image button 62b. The image displayed on the registered parking position image button 62 is an image captured by, for example, the front camera 12Fr of the vehicle 10 at the time of registration.

[0045] [Automatic parking control to a new parking position] First, the automatic parking control by the automatic parking control unit 55 when the new registration button 61 is touched will be described with reference to FIGS.

[0046] Fig. 5 is a flowchart showing automatic parking control for parking the vehicle 10 at a new parking position. Fig. 6 is a diagram showing a parking position 76 set in an overhead image 71 (synthetic image) generated from recognition data of the outside world of the vehicle 10 acquired by the front camera 12Fr, rear camera 12Rr, left side camera 12L, and right side camera 12R. Fig. 7 is a diagram showing feature points around the parking position 76 shown in Fig. 6. Fig. 8 is a diagram showing an example of registration content registered for the parking position. Fig. 9 is a diagram showing an example of a map included in the registration content of Fig. 8. When the new registration button 61 is touched, the automatic parking control unit 55 starts the processing shown in Fig. 5.

[0047] First, the automatic parking control unit 55 calculates the illuminance of the external environment from the recognition data of the external environment of the vehicle 10 acquired by the front camera 12Fr, the rear camera 12Rr, the left side camera 12L, and the right side camera 12R, and determines which of a plurality of preset slots the calculated illuminance corresponds to (step S11). The plurality of slots are set according to the brightness of the surrounding environment, for example, "daytime" with high illuminance and "nighttime" with low illuminance.

[0048] For example, as shown in the overhead image 71 in FIG. 6, vehicle 10 is parked in front of a parking lot 72 at home. When new registration button 61 is touched to register a new designated parking position, recognition data of the outside world is acquired by front camera 12Fr, rear camera 12Rr, left side camera 12L, and right side camera 12R, and the illuminance of the outside world is calculated. Note that overhead image 71 captures the driver's residence (building) 73 and multiple plants (obstacles) 74a-74f. Parking lot 72 is designated between building 73 and obstacles 74a-74f. Note that the vehicle displayed in overhead image 71 is an image showing vehicle 10 as seen from above, and is vehicle image 75 that has been generated (captured) in advance and stored in storage unit 54 or the like.

[0049] Next, the automatic parking control unit 55 accepts manual setting by the driver to set the parking position of the vehicle 10 (step S12).

[0050] As shown in FIG. 6, the parking position can be manually set by touching the rectangular parking position 76, which indicates the area where the vehicle 10 is to be parked, on the touch panel 42 and sliding the vehicle 10 to the desired parking position.

[0051] In step S12, when the manual setting of the parking position 76 is accepted, the automatic parking control unit 55 detects characteristic points related to the specified parking position 76 from the external recognition data acquired by the front camera 12Fr, the rear camera 12Rr, the left side camera 12L, and the right side camera 12R, and automatically parks the vehicle 10 in the parking position 76 based on the detected characteristic points (step S13).

[0052] For example, as shown in the overhead image 71 of Figure 7, the automatic parking control unit 55 detects, as feature points related to the specified parking position 76, building feature points 77a to 77d indicating the position of the outline of the building 73 closer to the parking position 76, obstacle feature points 78a to 78f indicating the positions of obstacles 74a to 74f existing around the parking position 76, and the like.

[0053] Next, the automatic parking control unit 55 associates a map consisting of the feature points related to the designated parking position 76 detected in step S13 and the parking position 76 set in step S12 with the slot corresponding to the illuminance calculated in step S11 and registers them in the memory unit 54 (step S14).

[0054] The details registered for the specified parking position 76 are classified into a plurality of patterns, such as "daytime" where the illuminance is high and "nighttime" where the illuminance is low, as shown in the correspondence table 81 in Fig. 8. Also, corresponding to the "daytime" and "nighttime" slots, respectively, maps of the parking position 76 and characteristic points related to the parking position 76 are registered, such as "map a" and "map b."

[0055] Specifically, a map of parking positions 76 and feature points related to those parking positions 76 is registered as a map showing the relationship between building feature points 77a-77d and obstacle feature points 78a-78f detected in the overhead image 71 of Fig. 7 and the manually set parking position 76, as in map 91 shown in Fig. 9. For ease of explanation, "map a" is registered in correspondence table 81 of Fig. 8 corresponding to "daytime" and "map b" is registered corresponding to "nighttime", but since this example is processing when new registration button 61 is touched, the map actually registered in step S14 above is only the map of the slot corresponding to the illuminance at the time of registration.

[0056] As a result, the parking position 76 specified by the user is registered in the memory unit 54 as a designated parking position having characteristic points corresponding to the external illuminance, and is displayed in the automatic parking menu as one of the registered parking position image buttons 62.

[0057] [Automatic parking control at registered parking locations] Next, the automatic parking control by the automatic parking control unit 55 when the registered parking position image button 62 is touched will be described with reference to FIG.

[0058] Fig. 10 is a flowchart showing an example of automatic parking control for parking the vehicle 10 at a registered parking position. When the registered parking position image button 62 on the automatic parking menu shown in Fig. 4 is touched, the automatic parking control unit 55 starts the process shown in Fig. 10. In this example, it is assumed that the driver of the vehicle 10 has touched the No. 1 image button 62a.

[0059] The automatic parking control unit 55 determines the illuminance of the outside world from the recognition data of the outside world of the vehicle 10 acquired by the front camera 12Fr, the rear camera 12Rr, the left side camera 12L, and the right side camera 12R (step S21). This determination process is the same as step S11 in FIG. 5 described above.

[0060] Next, the automatic parking control unit 55 determines whether or not a map is registered in the slot corresponding to the external illuminance determined in step S21 for the parking position designated by the No. 1 image button 62a (step S22).

[0061] In step S22, if a map is registered in the corresponding slot (step S22: Yes), the automatic parking control unit 55 performs self-position recognition processing to recognize the relative position of the vehicle 10 with respect to the designated parking position based on the slot pattern in which the map is registered and the current camera image (step S23).

[0062] Next, the automatic parking control unit 55 determines whether or not the self-position has been recognized in the self-position recognition process of step S23 (step S24).

[0063] In step S24, if the vehicle's own position is recognized (step S24: Yes), the automatic parking control unit 55 automatically parks the vehicle 10 at the designated parking position of the No. 1 image button 62a designated by the driver of the vehicle 10 based on the slot pattern in which this map is registered (step S25).

[0064] If the vehicle's own position cannot be recognized in step S24 (step S24: No), the automatic parking control unit 55 notifies the driver, for example, via the touch panel 42 or the speaker 44, that the vehicle 10 cannot be recognized with the currently registered data (step S26). In other words, even if a map is registered in the corresponding slot, if the vehicle's own position cannot be recognized, the automatic parking control unit 55 will not automatically park the vehicle 10 in the designated parking position.

[0065] On the other hand, if no map is registered in the corresponding slot in step S22 (step S22: No), the automatic parking control unit 55 accepts manual setting by the driver to set the parking position of the vehicle 10 (step S27). To inform the driver that manual setting of the parking position has been accepted, the automatic parking control unit 55 displays an overhead image similar to that shown in FIG. 6 on the touch panel 42, and also displays, for example, "Insufficient data for automatic parking. Please make settings." The method for manually setting the parking position is the same as step S12 in FIG. 5 described above.

[0066] In step S27, when the automatic parking control unit 55 accepts the manual setting of the parking position 76 (see FIG. 6), the automatic parking control unit 55 detects feature points related to the specified parking position 76 from the external recognition data acquired by the front camera 12Fr, the rear camera 12Rr, the left side camera 12L, and the right side camera 12R, and automatically parks the vehicle 10 in the parking position 76 based on the detected feature points (step S28). This automatic parking process is the same as step S13 in FIG. 5 described above. For example, as shown in FIG. 7, the automatic parking control unit 55 detects building feature points 77a to 77d and obstacle feature points 78a to 78f related to the specified parking position 76.

[0067] Next, the automatic parking control unit 55 prompts the driver to newly register a map that is generated based on the feature points related to the specified parking position 76 detected in step S28 (step S29). For example, the automatic parking control unit 55 may cause the touch panel 42 to display an overhead image as shown in Fig. 7, and may also display a message such as "Registering a parking position in a bright scene may increase the success rate of parking. Do you want to register?" or "Registering a parking position in a dark scene may increase the success rate of parking. Do you want to register?" At this time, a "register button" that is touched to register and a "non-register button" that is touched to not register may be displayed.

[0068] Next, the automatic parking control unit 55 determines whether or not the driver has performed an operation to newly register a map (step S30).

[0069] In step S30, if an operation to register a new map is performed (step S30: Yes), the automatic parking control unit 55 associates a map consisting of feature points related to the specified parking position 76 detected in step S28 and the parking position manually set in step S27 with the slot corresponding to the illuminance calculated in step S21 and registers the map in the memory unit 54 (step S31).

[0070] As a result, a map showing slots corresponding to illuminance and their characteristic points is added and registered in the correspondence table 81 shown in FIG. 8, which is provided for the parking position designated by the No. 1 image button 62a.

[0071] In step S30, if an operation to newly register a map has not been performed (step S30: No), the automatic parking control unit 55 ends the automatic parking control process without registering a map.

[0072] As described above, the automatic parking control unit 55 of the mobile object control device performs automatic parking control to park the vehicle 10 at a predetermined parking position based on the external recognition data and the predetermined parking position, and registers the predetermined parking position as a designated parking position. Then, a map including feature points related to the designated parking position is registered using a plurality of slot patterns based on the illuminance of the external environment calculated from the external recognition data. With this configuration, because the feature points related to the designated parking position are registered using a plurality of slot patterns based on the illuminance of the external environment, automatic parking control to the designated parking position can be performed even if the illuminance of the external environment of the vehicle 10 changes due to time of day (daytime vs. nighttime, etc.), weather (sunny vs. cloudy, etc.), etc.

[0073] Furthermore, the automatic parking control unit 55 selects a specific slot pattern from the multiple slot patterns of the feature points that corresponds to the illuminance acquired from the external recognition data when executing automatic parking control of the vehicle 10 at the designated parking position, and performs automatic parking control of the vehicle 10 at the designated parking position based on the selected specific slot pattern. This makes it possible to select an appropriate slot pattern of the feature points according to the illuminance of the external environment of the vehicle 10, thereby improving the accuracy of the automatic parking control.

[0074] Furthermore, if the automatic parking control unit 55 is unable to recognize the vehicle 10's own position based on a specific slot pattern corresponding to the illuminance acquired from the external recognition data when executing automatic parking control to a designated parking position, the automatic parking control unit 55 will not perform automatic parking control to a designated parking position based on the corresponding specific slot pattern. In this way, safety can be maintained by not performing automatic parking control when the vehicle 10 is unable to recognize its own position.

[0075] Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and modifications, improvements, etc. are possible as appropriate.

[0076] For example, in the above embodiment, the slots are classified into "daytime" and "nighttime" according to the illuminance of the outside world, but this is not limiting. For example, the slots may be classified into "high" and "low" according to the level of direct illuminance. Furthermore, the slots may be classified into three or more types, such as "high," "medium," and "low."

[0077] In the above embodiment, the case where the vehicle 10 is automatically parked by automatic steering of the automatic parking control unit 55 has been described, but the present invention is not limited to this. For example, the present invention may be applied to a parking assistance that assists the driver in parking the vehicle 10 through an operation by the driver.

[0078] In the above embodiment, the parking position of the vehicle 10 and the surrounding characteristic points are described using overhead images (composite images) from the front camera 12Fr, rear camera 12Rr, left side camera 12L, and right side camera 12R, but the present invention is not limited to this. For example, the description may be given using an image captured by any one of the front camera 12Fr, rear camera 12Rr, left side camera 12L, and right side camera 12R.

[0079] In the above embodiment, the automatic parking control unit 55 displays the overhead image 71 and the like on the touch panel 42 of the vehicle 10, but the present invention is not limited to this. For example, the automatic parking control unit 55 may display the overhead image 71 and the like on a display screen of an information terminal (e.g., a smartphone) carried by a passenger of the vehicle 10 via the communication unit 24.

[0080] In the above embodiment, the case where information such as a map including characteristic points related to the designated parking position and slots based on the illuminance of the outside world is registered in the storage unit 54 of the vehicle 10 has been described, but the present invention is not limited to this. For example, such information may be registered in the storage unit of another device such as a smartphone or server that is communicably connected to the vehicle 10.

[0081] Furthermore, in the above embodiment, a case has been described in which the user touches the registered parking position image button 62 (when selecting and touching the No. 1 image button 62a in FIG. 4) when parking the vehicle 10 in a registered parking position, but this is not limiting. For example, when the user touches the automatic parking button 60 displayed on the touch panel 42 when automatically parking the vehicle 10, the automatic parking control unit 55 may compare the overhead image captured during execution of automatic parking with the registered overhead image, automatically determine the parking position in which the vehicle is to be parked this time, and start the processing of FIG. 10, for example. This eliminates the need for the user of the vehicle 10 to find and select the button for the parking position in which the vehicle is to be parked this time from among the registered parking position image buttons 62.

[0082] In the above embodiment, the moving body is a vehicle (four-wheeled automobile), but the present invention is not limited to this. For example, the moving body may be a motorcycle, a Segway, or other vehicle. Furthermore, the concept of the present invention is not limited to vehicles, and may also be applied to robots, ships, aircraft, and the like that are equipped with a drive source and can move using the power of the drive source.

[0083] The mobile object control method described in the above-described embodiment can be realized by executing a prepared mobile object control program on a computer. The mobile object control program is recorded on a computer-readable storage medium and executed by being read from the storage medium. The mobile object control program may be provided in a form stored on a non-transitory storage medium such as a flash memory, or may be provided via a network such as the Internet. The computer that executes the mobile object control program may be included in a mobile object control device, or may be included in an electronic device such as a smartphone, tablet terminal, or personal computer that can communicate with the mobile object control device, or may be included in a server device that can communicate with these mobile object control devices and electronic devices.

[0084] This specification also describes at least the following: Note that the components in parentheses correspond to those in the above-described embodiment, but are not limited to these.

[0085] (1) An external environment recognition unit (a front camera 12Fr, a rear camera 12Rr, a left side camera 12L, and a right side camera 12R) that acquires external environment recognition data of a moving body (vehicle 10); a reception unit (touch panel 42) that receives a designation of a predetermined parking position from a user of the mobile body; a control unit (automatic parking control unit 55) that performs automatic parking control to park the moving body at the predetermined parking position based on the external recognition data and the predetermined parking position, and registers the predetermined parking position as a designated parking position; The control unit registers feature points related to the designated parking position using a plurality of patterns based on the illuminance of the external environment calculated from the external environment recognition data. Mobile control device.

[0086] According to (1), the characteristic points of the designated parking position are registered in multiple patterns based on the external illumination, so that automatic parking control to the designated parking position can be performed even if the external illumination of the moving body changes depending on the time of day (daytime, nighttime, etc.) or weather (sunny, cloudy, etc.).

[0087] (2) The mobile object control device according to (1), The control unit selects a specific pattern corresponding to the illuminance acquired from the external recognition data when the automatic parking control to the designated parking position is executed from among the plurality of patterns of the feature points, and performs the automatic parking control to the designated parking position based on the specific pattern. Mobile control device.

[0088] According to (2), an appropriate pattern of feature points can be selected according to the illuminance of the outside world of the moving object, thereby improving the accuracy of automatic parking control.

[0089] (3) The mobile object control device according to (2), the control unit does not perform the automatic parking control to the designated parking position based on the specific pattern when the self-position recognition of the moving object is not possible based on the specific pattern. Mobile control device.

[0090] According to (3), safety can be improved by not performing automatic parking control if the moving object is unable to recognize its own position.

[0091] (4) The processor of the mobile control device Acquires recognition data of the external world of the moving object, Accepting a designation of a predetermined parking position from a user of the mobile body; performing automatic parking control for parking the mobile body at the predetermined parking position based on the external recognition data and the predetermined parking position, and registering the predetermined parking position as a designated parking position; registering feature points related to the designated parking position using a plurality of patterns based on the illuminance of the external environment calculated from the external environment recognition data; A mobile object control method.

[0092] According to (4), the characteristic points of the designated parking position are registered in multiple patterns based on the external illumination, so that automatic parking control to the designated parking position can be performed even if the external illumination of the moving body changes depending on the time of day (daytime, nighttime, etc.) or weather (sunny, cloudy, etc.).

[0093] (5) A processor of the mobile control device Acquires recognition data of the external world of the moving object, Accepting a designation of a predetermined parking position from a user of the mobile body; performing automatic parking control for parking the mobile body at the predetermined parking position based on the external recognition data and the predetermined parking position, and registering the predetermined parking position as a designated parking position; registering feature points related to the designated parking position using a plurality of patterns based on the illuminance of the external environment calculated from the external environment recognition data; A mobile control program for executing processing.

[0094] According to (5), the characteristic points of the designated parking position are registered in multiple patterns based on the external illumination, so that automatic parking control to the designated parking position can be performed even if the external illumination of the moving body changes depending on the time of day (daytime, nighttime, etc.) or weather (sunny, cloudy, etc.). [Explanation of symbols]

[0095] 10 Vehicles (moving objects) 12Fr front camera (external recognition unit) 12Rr Rear camera (external recognition unit) 12L Left side camera (external recognition unit) 12R Right side camera (external recognition unit) 42 Touch panel (reception area) 55 Automatic parking control unit (control unit)

Claims

1. An external environment recognition unit that acquires recognition data of the external environment of a moving object using a sensor equipped in the moving object; a reception unit that receives a designation of a predetermined parking position from a user of the mobile body; a control unit that performs automatic parking control to park the mobile body at the predetermined parking position based on the external recognition data and the predetermined parking position, and registers the predetermined parking position as a designated parking position, The control unit registering feature points related to the designated parking position based on a plurality of patterns for each illuminance based on the illuminance of the external environment calculated from the external environment recognition data; If there is no specific pattern among the plurality of patterns of the feature points that corresponds to the illuminance acquired from the external environment recognition data when the automatic parking control to the designated parking position is executed, a notification is given to prompt the user to register the specific pattern. Mobile control device.

2. 2. The mobile object control device according to claim 1, The control unit selects the specific pattern from among a plurality of patterns of the feature points, and performs the automatic parking control to the designated parking position based on the specific pattern. Mobile control device.

3. The mobile object control device according to claim 2, the control unit does not perform the automatic parking control to the designated parking position based on the specific pattern when the self-position recognition of the moving object is not possible based on the specific pattern. Mobile control device.

4. A processor of the mobile control device Acquires recognition data of the external world of the moving object, Accepting a designation of a predetermined parking position from a user of the mobile body; performing automatic parking control for parking the mobile body at the predetermined parking position based on the external recognition data and the predetermined parking position, and registering the predetermined parking position as a designated parking position; registering feature points related to the designated parking position based on a plurality of patterns for each illuminance based on the illuminance of the external environment calculated from the external environment recognition data; If there is no specific pattern among the plurality of patterns of the feature points that corresponds to the illuminance acquired from the external environment recognition data when the automatic parking control to the designated parking position is executed, a notification is given to prompt the user to register the specific pattern. A mobile object control method.

5. A processor of the mobile control device Acquires recognition data of the external world of the moving object, Accepting a designation of a predetermined parking position from a user of the mobile body; performing automatic parking control for parking the mobile body at the predetermined parking position based on the external recognition data and the predetermined parking position, and registering the predetermined parking position as a designated parking position; registering feature points related to the designated parking position based on a plurality of patterns for each illuminance based on the illuminance of the external environment calculated from the external environment recognition data; If there is no specific pattern among the plurality of patterns of the feature points that corresponds to the illuminance acquired from the external environment recognition data when the automatic parking control to the designated parking position is executed, a notification is given to prompt the user to register the specific pattern. A mobile control program for executing processing.

Images