Information processing device and vehicle

Abstract

The present technology relates to an information processing device and a vehicle that make it possible to perform automatic parking in accordance with a user's preferences. The information processing device comprises a parking lot map generating unit that generates a parking lot map of a parking lot on the basis of a sensing result of a sensor mounted in the vehicle, and a contextual information generating unit that generates contextual information of the parking lot on the basis of the sensing result and the parking lot map, wherein the parking lot map generating unit generates a contextual parking lot map in which the contextual information is added to the parking lot map. The present technology is applicable to vehicle control systems for vehicles that perform autonomous driving, for example.

Description

Information Processing Device and Vehicle

[0007] ,

[0006] , ,

[0005] ,

[0001] This technology relates to an information processing device and a vehicle, and particularly relates to an information processing device and a vehicle that enable automatic parking according to user preferences.

[0002] Patent Document 1 discloses an in-vehicle processing device that allows a user to park a vehicle at a target parking position in a recording phase and store it as parking lot data, and to automatically park at the stored parking position in an automatic parking phase.

[0003] Japanese Patent Application Laid-Open No. 2018-112506

[0004] According to the automatic parking function disclosed in Patent Document 1, it is possible to automatically park at a pre-registered parking position. However, in the case of a public parking lot, it is not guaranteed that the pre-registered parking position is available, and parking fails when it is not available. In that case, it is not known which parking space to park in. When selecting a parking position, the user often has preferences, such as a parking space in an area close to the entrance / exit or a parking space in an area close to the facility.

[0005] This technology has been made in view of such a situation, and enables automatic parking according to user preferences.

[0006] The information processing device according to the first aspect of this technology includes a parking lot map generation unit that generates a parking lot map of a parking lot based on the sensing results of sensors mounted on a vehicle, and a context information generation unit that generates context information of the parking lot based on the sensing results and the parking lot map. The parking lot map generation unit generates a context parking lot map in which the context information is added to the parking lot map.

[0007] The second aspect of this technology is a vehicle comprising: a parking map generation unit that generates a parking map of a parking lot based on sensing results from sensors mounted on the vehicle; a context information generation unit that generates context information of the parking lot based on the sensing results and the parking map; and an operation control unit that controls the operation of the vehicle. The parking map generation unit generates a context parking map by adding the context information to the parking map, and the operation control unit controls the operation of the vehicle to park at a parking position determined based on the context parking map.

[0008] In the first and second aspects of this technology, a parking map of a parking lot is generated based on the sensing results of sensors mounted on a vehicle, contextual information of the parking lot is generated based on the sensing results and the parking map, and a contextual parking map is generated by adding the contextual information to the parking map.

[0009] Furthermore, the first aspect of this technology, the information processing device, can be realized by having a computer execute a program. The program to be executed by the computer to realize this information processing device can be provided by transmitting it via a transmission medium or by recording it on a recording medium.

[0010] An information processing device may be an independent device or an internal block that constitutes a single device.

[0011] This is a block diagram showing an example configuration of a vehicle control system. This is a diagram showing an example of the sensing area of ​​the external environment recognition sensor of the vehicle control system in Figure 1. This is a plan view of a parking lot where automatic parking is performed in the first embodiment. This is a diagram showing an example of a parking lot map generated for the parking lot in Figure 3. This is a diagram showing an example of a context map generated for the parking lot in Figure 3. This is a block diagram showing an example configuration of a vehicle control system according to the first embodiment of this technology. This is a diagram explaining the generation of context information. This is a flowchart explaining the context parking map generation and presentation process. This is a diagram showing an example of a screen displaying a context parking map. This is a flowchart explaining the context parking map update process. This is a diagram showing another example of a context map. This is a diagram showing an example of a modified context map. This is a block diagram showing an example configuration of a vehicle control system according to the second embodiment of this technology. This is a plan view showing a first presentation example of presenting parking position candidates. This is a plan view showing a second presentation example of presenting parking position candidates. This is a plan view showing a third presentation example of presenting parking position candidates. This is a plan view showing a fourth presentation example of presenting parking position candidates. This is a plan view showing a fifth presentation example of presenting parking position candidates. This is a plan view showing a sixth presentation example of presenting parking position candidates. This is a diagram showing an example of a preference information registration screen. This is a flowchart explaining the contextual parking map generation and presentation process. This is a diagram explaining the parking location candidate update process. This is a block diagram showing an example configuration of a vehicle control system according to the third embodiment of this technology. This is a diagram explaining an example of modifying a parking lot. This is a block diagram showing an example configuration of a cloud server for learning individual recognition models. This is a flowchart explaining the training data transmission process. This is a flowchart explaining the individual recognition model generation process. This is a block diagram showing an example configuration of a computer to which this technology is applied.

[0012] The following describes the embodiments for implementing this technology. The explanation will proceed in the following order: 1. Example of vehicle control system configuration 2. First embodiment (presentation of contextual parking map) 3. Second embodiment (suggestion of parking location candidates, determination of parking location based on user preference information) 4. Third embodiment (generation of recognition models specialized for individual parking lots) 5. Example of computer configuration

[0013] <<1. Example of Vehicle Control System Configuration>> Figure 1 is a block diagram showing an example of the configuration of a vehicle control system 11, which is a non-limiting example of a mobile device control system to which this technology is applied.

[0014] The vehicle control system 11 is installed in the vehicle 1 and performs processing related to driving assistance, autonomous driving, and autonomous parking of the vehicle 1. The vehicle control system 11 consists of one or more information processing devices.

[0015] The vehicle control system 11 includes a vehicle control ECU (Electronic Control Unit) 21, a communication unit 22, a map information storage unit 23, a location information acquisition unit 24, an external environment recognition sensor 25, an in-vehicle sensor 26, a vehicle sensor 27, a memory unit 28, an automated driving control unit 29, a DMS (Driver Monitoring System) 30, an HMI (Human Machine Interface) 31, and a vehicle control unit 32.

[0016] Two or more (or, in some cases, all) of the following components are connected to communicate with each other via a communication network 41: the vehicle control ECU 21, the communication unit 22, the map information storage unit 23, the location information acquisition unit 24, the external environment recognition sensor 25, the in-vehicle sensor 26, the vehicle sensor 27, the memory unit 28, the driving automation control unit 29, the DMS 30, the HMI 31, and the vehicle control unit 32. The communication network 41 is composed of an in-vehicle communication network or bus that conforms to digital bidirectional communication standards such as CAN (Controller Area Network), LIN (Local Interconnect Network), LAN (Local Area Network), FlexRay®, and Ethernet®. In some embodiments, the communication network 41 may have two or more types of communication networks, and different types of communication networks may be used depending on the type of data being transmitted. For example, CAN may be applied to data related to vehicle control, and Ethernet may be applied to large-capacity data. In some embodiments, two or more (or possibly all) units of the vehicle control system 11 may be directly connected using wireless communication (e.g., relatively short-range communication) without going through the communication network 41. In some embodiments, the wireless communication may use near-field wireless communication technology. Non-limiting examples of near-field wireless communication technology include near-field communication (NFC) and Bluetooth®. In some embodiments, two or more (or possibly all) units of the vehicle control system 11 may be connected using the communication network 41 and wireless communication technology (e.g., near-field wireless communication technology).

[0017] In the following embodiment, where two or more units of the vehicle control system 11 communicate via the communication network 41, the description of the communication network 41 will be omitted. For example, in an embodiment where the vehicle control ECU 21 and the communication unit 22 communicate via the communication network 41, it will simply be described as the vehicle control ECU 21 and the communication unit 22 communicating.

[0018] The vehicle control ECU 21 is composed of various processors, such as a CPU (Central Processing Unit) and an MPU (Micro Processing Unit). The vehicle control ECU 21 controls the functions of the entire vehicle control system 11 or a part of it.

[0019] The communication unit 22 communicates with various devices inside the vehicle 1 (hereinafter referred to as in-vehicle devices), various devices outside the vehicle 1 (hereinafter referred to as external devices), other vehicles, base stations, etc., and transmits and receives various types of data. In some embodiments, the communication unit 22 may use multiple communication technologies to perform communication.

[0020] A non-limiting example of communication between the communication unit 22 and external equipment will be briefly described. In some embodiments, the communication unit 22 may communicate with servers (hereinafter referred to as "external servers") located on an external network via a base station or access point using wireless communication technology. Examples of non-limiting wireless communication technologies include 5G (fifth-generation mobile communication system), LTE (Long Term Evolution), DSRC (Dedicated Short Range Communications), etc. External networks that the communication unit 22 can communicate with may include, for example, the internet, a cloud network, or a network specific to a carrier. The communication technology used by the communication unit 22 to communicate with an external network is not particularly limited, as long as it is a wireless communication technology that enables digital two-way communication at a predetermined communication speed and over a predetermined distance.

[0021] In some embodiments, the communication unit 22 may communicate with terminals located near the vehicle using P2P (Peer To Peer) technology. Terminals located near the vehicle include, for example, terminals worn by relatively slow-moving objects such as pedestrians and cyclists, terminals installed in fixed locations such as stores, and / or MTC (Machine Type Communication) terminals. In some embodiments, the communication unit 22 may perform V2X (Vehicle to Everything) communication. V2X communication generally refers to communication between the vehicle and other entities. Non-exclusive examples of V2X communication include vehicle-to-vehicle communication with other vehicles, vehicle-to-infrastructure communication with roadside devices, etc., vehicle-to-home communication with homes, and vehicle-to-pedestrian communication with terminals carried or worn by pedestrians.

[0022] In some embodiments, the communication unit 22 may receive a program from outside the vehicle 1 to update the software that controls the operation of the vehicle control system 11 (for example, over the air). In some embodiments, the communication unit 22 may receive map information, traffic information, information about the vehicle 1's surroundings, etc., from outside the vehicle 1. In some embodiments, the communication unit 22 may transmit information about the vehicle 1, information about the vehicle 1's surroundings, etc., to an external device or external network. Non-limiting examples of information about the vehicle 1 that the communication unit 22 transmits to an external device or external network include data indicating the status of the vehicle 1, recognition results from the recognition unit 73, etc. In some embodiments, the communication unit 22 may communicate with a vehicle emergency call system. Non-limiting examples of a vehicle emergency call system include e-Call, etc.

[0023] In some embodiments, the communication unit 22 may receive electromagnetic waves transmitted by a road traffic information communication system. In some embodiments, such electromagnetic waves may be transmitted using radio beacons, optical beacons, FM multiplex broadcasting, etc.

[0024] A non-limiting example of communication with in-vehicle equipment that the communication unit 22 can perform will be outlined below. In some embodiments, the communication unit 22 may communicate with in-vehicle equipment using wireless communication. For example, in some embodiments, the communication unit 22 may communicate with in-vehicle equipment wirelessly using wireless communication technology that enables digital bidirectional communication at a predetermined or higher communication speed. Non-limiting examples of wireless communication technologies include wireless LAN, Bluetooth, NFC, and WUSB (Wireless USB). Not limited to these, the communication unit 22 may also communicate with in-vehicle equipment using wired communication (in addition to or as an alternative to wireless communication). For example, in some embodiments, the communication unit 22 may communicate with in-vehicle equipment via wired communication through a cable connected to a connection terminal (not shown). In some embodiments, the communication unit 22 may communicate with in-vehicle equipment using wired communication technology that enables digital bidirectional communication at a predetermined or higher communication speed. Non-exclusive examples of wired communication technologies include USB (Universal Serial Bus), HDMI (High-Definition Multimedia Interface) (registered trademark), and MHL (Mobile High-definition Link).

[0025] Here, in-vehicle equipment refers to, for example, equipment located inside vehicle 1 that is not connected to the communication network 41. In-vehicle equipment is divided into equipment that constitutes the vehicle control system 11 and equipment that does not. Non-exclusive examples of in-vehicle equipment that does not constitute the vehicle control system 11 include mobile devices and wearable devices owned by users of vehicle 1 (e.g., the driver, passengers), and information equipment temporarily installed inside vehicle 1. These devices can, for example, be moved outside vehicle 1 and become external equipment.

[0026] The map information storage unit 23 stores maps acquired from external devices or external networks and / or maps created by the vehicle 1. For example, the map information storage unit 23 may store three-dimensional high-precision maps, global maps with lower precision than high-precision maps but covering a wide area, etc.

[0027] High-precision maps include, for example, dynamic maps, point cloud maps, and vector maps. A dynamic map may be a map consisting of four layers: dynamic information, semi-dynamic information, semi-static information, and static information, and may be provided to vehicle 1 from an external server or the like. A point cloud map may be a map composed of point clouds (point cloud data). A vector map may be a map adapted for automated driving by associating traffic information, such as the locations of lanes and traffic lights, with a point cloud map.

[0028] The point cloud map and vector map may be provided from, for example, an external server, or they may be created in the vehicle 1 as maps for matching with the local map described later, based on sensing results from the camera 51, radar 52, LiDAR 53, etc., and stored in the map information storage unit 23. In addition, if high-precision maps are provided from an external server, in order to reduce communication capacity, map data of, for example, several hundred meters square, relating to the planned route that the vehicle 1 will travel may be obtained from the external server.

[0029] The location information acquisition unit 24 acquires location information of the vehicle 1. The acquired location information may be supplied to the driving automation control unit 29. In some embodiments, the location information acquisition unit 24 may receive GNSS (Global Navigation Satellite System) signals from GNSS satellites. In some embodiments, the location information acquisition unit 24 may receive signals from beacons or the like.

[0030] The external environment recognition sensor 25 is equipped with various sensors used to recognize the external conditions of the vehicle 1, and supplies sensor data from one or more (or, in some cases, all) sensors to one or more (or, in some cases, all) units of the vehicle control system 11. The types and number of sensors equipped in the external environment recognition sensor 25 are arbitrary.

[0031] In some embodiments, the external environment recognition sensor 25 may include a camera 51, a radar 52, a LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) 53, an ultrasonic sensor 54, and a microphone 55. However, it is not limited to this configuration, and the external environment recognition sensor 25 may include one or more of the cameras 51, radar 52, LiDAR 53, ultrasonic sensor 54, and microphone 55. The number of cameras 51, radar 52, LiDAR 53, ultrasonic sensor 54, and microphone 55 is not particularly limited as long as it is a number that can be realistically installed in the vehicle 1. Furthermore, the types of sensors included in the external environment recognition sensor 25 are not limited to this example, and the external environment recognition sensor 25 may include other types of sensors. Examples of the sensing areas of each sensor included in the external environment recognition sensor 25 will be described later.

[0032] Camera 51 can use any suitable shooting method. In some embodiments, camera 51 may use a shooting method capable of distance measurement. Non-limited examples of cameras using a shooting method capable of distance measurement include ToF (Time Of Flight) cameras, stereo cameras, monocular cameras, and infrared cameras. However, camera 51 may not be capable of distance measurement and may simply be for acquiring images. Microphone 55 is used for detecting sounds around the vehicle 1 and the location of sound sources, etc.

[0033] In some embodiments, the external environment recognition sensor 25 may include environmental sensors for detecting characteristics of the environment around the vehicle 1. Non-limiting examples of detectable environmental characteristics include weather, climate, brightness, etc. In some embodiments, the environmental sensors may include various sensors such as raindrop sensors, fog sensors, sunshine sensors, snow sensors, and illuminance sensors.

[0034] The in-vehicle sensor 26 is equipped with various sensors for detecting information inside the vehicle 1, and supplies sensor data from one or more (or, in some cases, all) sensors to one or more (or, in some cases, all) units of the vehicle control system 11. The types and number of sensors equipped in the in-vehicle sensor 26 are not particularly limited, as long as they are of a type and number that can be realistically installed in the vehicle 1.

[0035] In some embodiments, the in-vehicle sensor 26 may include one or more sensors from among a camera, radar, seat sensor, microphone, and biosensor. In some embodiments, the camera included in the in-vehicle sensor 26 may use a distance-measuring shooting method. Non-limiting examples of cameras using a distance-measuring shooting method include ToF cameras, stereo cameras, monocular cameras, and infrared cameras. However, the camera included in the in-vehicle sensor 26 may not be for distance measurement and may simply be for acquiring captured images. The biosensor included in the in-vehicle sensor 26 may be installed, for example, on the seat or steering wheel, and may detect various biometric information of the user.

[0036] The vehicle sensor 27 is equipped with various sensors for detecting the state of the vehicle 1 and supplies sensor data from one or more (or, in some cases, all) sensors to one or more (or, in some cases, all) units of the vehicle control system 11. The types and number of sensors equipped in the vehicle sensor 27 are not particularly limited, as long as they are of a type and number that can be realistically installed on the vehicle 1.

[0037] In some embodiments, the vehicle sensor 27 may include a speed sensor, an acceleration sensor, an angular velocity sensor (gyro sensor), and / or an inertial measurement unit (IMU) integrating them. In some embodiments, the vehicle sensor 27 may include a steering angle sensor for detecting the steering angle of the steering wheel, a yaw rate sensor, an accelerator sensor for detecting the amount of operation of the accelerator pedal (e.g., pedal force, pedal stroke), and / or a brake sensor for detecting the amount of operation of the brake pedal (e.g., pedal force, pedal stroke). In some embodiments, the vehicle sensor 27 may include a rotation sensor for detecting the rotational speed of the engine or motor, an air pressure sensor for detecting the air pressure of the tires, a slip ratio sensor for detecting the slip ratio of the tires, and / or a wheel speed sensor for detecting the rotational speed of the wheels. In some embodiments, the vehicle sensor 27 may include a battery sensor for detecting the remaining charge and temperature of the battery, and / or an impact sensor capable of detecting external impacts.

[0038] The storage unit 28 includes at least one of a non-volatile storage medium and a volatile storage medium, and stores data and programs. Non-limiting examples of storage mediums include magnetic storage devices such as EEPROM (Electrically Erasable Programmable Read Only Memory), RAM (Random Access Memory), and / or HDD (Hard Disc Drive), semiconductor storage devices, optical storage devices, and magneto-optical storage devices. The storage unit 28 stores various programs and data used by one or more (or in some cases all) units of the vehicle control system 11. In some embodiments, the storage unit 28 may include an EDR (Event Data Recorder) or a DSSAD (Data Storage System for Automated Driving) to store information about the vehicle 1 before and after an event such as an accident, and information acquired by the in-vehicle sensors 26.

[0039] The automated driving control unit 29 controls the automated driving functions of the vehicle 1. In some embodiments, the automated driving control unit 29 may also include an analysis unit 61, an action planning unit 62, and an operation control unit 63.

[0040] The analysis unit 61 performs analysis processing of the vehicle 1 and / or the surrounding conditions. The analysis unit 61 includes a self-position estimation unit 71, a sensor fusion unit 72, and a recognition unit 73.

[0041] In some embodiments, the self-position estimation unit 71 may estimate the vehicle's position based on sensor data from the external environment recognition sensor 25 and a high-precision map stored in the map information storage unit 23. For example, the self-position estimation unit 71 may generate a local map based on sensor data from the external environment recognition sensor 25 and estimate the vehicle's position by matching the local map with a high-precision map. The position of the vehicle 1 may be based on, for example, the center of the rear wheel relative to the axle.

[0042] In some embodiments, the local map may be a three-dimensional high-precision map, an occupancy grid map, or the like, created using technologies such as SLAM (Simultaneous Localization and Mapping). The three-dimensional high-precision map may be, for example, the point cloud map described above. The occupancy grid map may be a map that divides the three-dimensional or two-dimensional space around the vehicle 1 into grids of a predetermined size and shows the occupancy status of objects on a grid-by-grid basis. The occupancy status of objects may be indicated, for example, by the presence or absence or probability of existence of an object. In some embodiments, the local map may also be used, for example, for detection and / or recognition processing of the external conditions of the vehicle 1 by the recognition unit 73.

[0043] In some embodiments, the self-position estimation unit 71 may estimate the vehicle 1's own position based on position information acquired by the position information acquisition unit 24 and / or sensor data from the vehicle sensor 27.

[0044] The sensor fusion unit 72 performs sensor fusion processing to obtain information by combining a plurality of different types of sensor data (for example, image data supplied from the camera 51 and sensor data supplied from the radar 52). The methods of combining different types of sensor data include, but are not limited to, composite, integrated, fused, associated, etc.

[0045] The recognition unit 73 executes detection processing for detecting the situation outside the vehicle 1 and / or recognition processing for recognizing the situation outside the vehicle 1.

[0046] For example, the recognition unit 73 may perform detection processing and / or recognition processing of the situation outside the vehicle 1 based on information from the external recognition sensor 25, information from the self-position estimation unit 71, information from the sensor fusion unit 72, etc.

[0047] Specifically, for example, the recognition unit 73 may perform detection processing and / or recognition processing of an object around the vehicle 1. The object detection processing may be, for example, processing for detecting the presence or absence, size, shape, position, movement, etc. of an object. The object recognition processing may be, for example, processing for recognizing attributes such as the type of an object or for identifying a specific object. The detection processing and the recognition processing are not necessarily clearly separated and may overlap at least partially.

[0048] In some embodiments, the recognition unit 73 may detect an object around the vehicle 1 by performing clustering that classifies a point cloud based on sensor data from the radar 52 and / or the LiDAR 53, etc. into clusters for each group of points. Thereby, the presence or absence, size, shape, and position of an object around the vehicle 1 can be detected.

[0049] In some embodiments, the recognition unit 73 may detect the movement of an object around the vehicle 1 by tracking the movement of a group of points classified by clustering. Thereby, the speed and / or traveling direction (movement vector) of an object around the vehicle 1 can be detected.

[0050] In some embodiments, the recognition unit 73 may detect and / or recognize vehicles (including bicycles), people, obstacles, structures, roads, traffic lights, traffic signs, road markings, etc., based on image data supplied from the camera 51. In some embodiments, the recognition unit 73 may recognize the types of objects around the vehicle 1 by performing recognition processing such as semantic segmentation.

[0051] In some embodiments, the recognition unit 73 may perform a recognition process of traffic rules around the vehicle 1 based on the map stored in the map information storage unit 23, the self-position estimation result by the self-position estimation unit 71, and / or the recognition result of objects around the vehicle 1 by the recognition unit 73. Through this process, the recognition unit 73 may recognize the location and / or status of traffic signals, the content of traffic signs and / or road markings, the content of traffic regulations, and / or drivable lanes.

[0052] In some embodiments, the recognition unit 73 may perform recognition processing of the environment surrounding the vehicle 1. In some embodiments, the recognition unit 73 may recognize weather characteristics (temperature, humidity, brightness), and / or road surface conditions, etc.

[0053] The action planning unit 62 creates an action plan for vehicle 1. For example, the action planning unit 62 may create an action plan by performing route planning and route following.

[0054] In some embodiments, the path planning may include global path planning and local path planning. Global path planning may include the process of planning a rough route from the start to the goal. Local path planning, also called trajectory planning, may include the generation of a trajectory that allows the vehicle 1 to travel safely and smoothly along the planned route in the vicinity of the vehicle 1, taking into account the motion characteristics of the vehicle 1 and the presence of any obstacles.

[0055] In some embodiments, route following may involve planning actions to safely and accurately travel along the route planned by the route planner within a planned time. The action planning unit 62 may, for example, calculate the target speed and / or target angular velocity of the vehicle 1 based on the results of this route following process.

[0056] The motion control unit 63 controls the operation of the vehicle 1 in order to realize the action plan created by the action planning unit 62.

[0057] For example, in some embodiments, the motion control unit 63 may control the steering control unit 81, brake control unit 82, and / or drive control unit 83, which are included in the vehicle control unit 32 described later, to perform lateral vehicle motion control and / or longitudinal vehicle motion control so that the vehicle 1 travels along the trajectory calculated by the trajectory plan. For example, the motion control unit 63 may perform one or more driver assistance functions and / or control for the purpose of driving automation (e.g., lateral vehicle motion control, longitudinal vehicle motion control). Non-limited examples of driver assistance functions include collision avoidance or impact mitigation, inter-vehicle distance control (e.g., control to maintain a specific distance from a vehicle traveling in front of the vehicle 1), vehicle speed control (e.g., control to maintain a specific speed), vehicle collision warning, and lane departure warning. Non-limited examples of driving automation include driving without operation by the driver or remote driver.

[0058] In some embodiments, the DMS 30 may perform driver authentication processing and / or driver status recognition processing based on sensor data from the in-vehicle sensor 26 and / or input data input to the HMI 31, which will be described later. Non-limited examples of driver status that may be recognized include physical condition, alertness level, concentration level, fatigue level, gaze direction, intoxication level, driving operation, posture, etc.

[0059] In some embodiments, the DMS 30 may perform authentication processing for users other than the driver (e.g., passengers) and / or recognition processing for the status of such users. In some embodiments, the DMS 30 may perform recognition processing for the internal conditions of the vehicle 1 based on sensor data from the in-vehicle sensors 26. Non-limiting examples of characteristics of the internal conditions of the vehicle 1 that may be recognized include temperature, humidity, brightness, odor, etc.

[0060] HMI31 receives various data and instructions as input and presents various data to the user.

[0061] A brief overview of data input to the HMI 31 is provided. The HMI 31 is equipped with an input device for a person to input data, instructions, etc. Based on the data, instructions, etc. input by the input device, the HMI 31 generates an input signal and supplies it to one or more (or, in some cases, all) units of the vehicle control system 11. In some embodiments, the HMI 31 may be equipped with a touch panel, buttons, switches, and / or levers as input devices. Not limited to these, the HMI 31 may be equipped with an input device that allows information to be input by methods other than manual operation, such as voice or gestures. In some embodiments, the HMI 31 may be equipped with a remote control device using infrared and / or radio waves, or an external connection device that corresponds to the operation of the vehicle control system 11, as an input device. Non-limited examples of external connection devices include mobile devices (e.g., smartphones) and wearable devices (e.g., smartwatches).

[0062] A brief explanation of data presentation by HMI31 is provided below. HMI31 generates visual, auditory, and / or tactile information for the user and / or people outside of vehicle 1. HMI31 may also perform output control to control the output, output content, output timing, and / or output method of each generated piece of information. Non-limited examples of visual information that can be generated and output by HMI31 include information shown by images and light, such as operation screens, vehicle 1 status displays, warning displays, and monitor images showing the surroundings of vehicle 1. Non-limited examples of auditory information that can be generated and output by HMI31 include voice guidance, warning sounds, and warning messages. Non-limited examples of tactile information that can be generated and output by HMI31 include information given to the user's sense of touch through force, vibration, movement, etc.

[0063] In some embodiments, the HMI 31 may include, as an output device capable of outputting visual information, a display device that presents visual information by displaying an image itself, or a projector device that presents visual information by projecting an image. In some embodiments, the display device may be a device that displays visual information within the user's field of view, such as a head-up display, a transparent display, or a wearable device with AR (Augmented Reality) functionality, in addition to or as an alternative to a normal display device. In some embodiments, the HMI 31 may include, as an output device capable of outputting visual information, a display device provided in the vehicle 1, such as a navigation device, instrument panel, CMS (Camera Monitoring System), electronic mirror, lamp, etc.

[0064] In some embodiments, the HMI31 may include an audio speaker, headphones, or earphones as an output device capable of outputting auditory information.

[0065] In some embodiments, the HMI 31 may include a haptic element using haptic technology as an output device capable of outputting tactile information. The haptic element may be provided, for example, on a part of the vehicle 1 that the user comes into contact with, such as the steering wheel or the seat.

[0066] The vehicle control unit 32 controls one or more (or, in some cases, all) units of the vehicle 1. The vehicle control unit 32 includes a steering control unit 81, a brake control unit 82, a drive control unit 83, a body system control unit 84, a light control unit 85, and a horn control unit 86.

[0067] The steering control unit 81 detects and / or controls the state of the steering system of the vehicle 1. The steering system includes, for example, a steering mechanism with a steering wheel, an electric power steering system, etc. The steering control unit 81 includes, for example, a steering ECU that controls the steering system, an actuator that drives the steering system, etc.

[0068] The brake control unit 82 detects and / or controls the state of the brake system of the vehicle 1. The brake system includes, for example, a brake mechanism including a brake pedal, an ABS (Antilock Brake System), a regenerative braking mechanism, etc. The brake control unit 82 includes, for example, a brake ECU that controls the brake system, an actuator that drives the brake system, etc.

[0069] The drive control unit 83 detects and / or controls the state of the vehicle 1's drive system. The drive system includes, for example, an accelerator pedal, a drive force generating device for generating driving force such as an internal combustion engine or drive motor, and a drive force transmission mechanism for transmitting driving force to the wheels. The drive control unit 83 also includes, for example, a drive ECU for controlling the drive system and an actuator for driving the drive system.

[0070] The body system control unit 84 detects and / or controls the state of the body system of the vehicle 1. The body system includes, for example, a keyless entry system, a smart key system, power window devices, power seats, an air conditioning system, airbags, seat belts, a shift lever, etc. The body system control unit 84 also includes, for example, a body system ECU that controls the body system, actuators that drive the body system, etc.

[0071] The light control unit 85 detects and / or controls the state of various lights on the vehicle 1. Non-exclusive examples of lights that can be controlled by the light control unit 85 include headlights, taillights, fog lights, turn signals, brake lights, projector lights, bumper indicators, etc. The light control unit 85 includes a light ECU for controlling the lights, actuators for driving the lights, etc.

[0072] The horn control unit 86 detects and / or controls the state of the car horn of the vehicle 1. The horn control unit 86 includes, for example, a horn ECU for controlling the car horn, an actuator for driving the car horn, and the like.

[0073] Figure 2 shows an example of the sensing area of ​​the external environment recognition sensor 25 in Figure 1, including the camera 51, radar 52, LiDAR 53, and ultrasonic sensor 54. In Figure 2, a schematic view of the vehicle 1 from above is shown.

[0074] Sensing regions 101F and 101B show examples of sensing regions for ultrasonic sensors 54. Sensing region 101F (for example, the sensing region of multiple ultrasonic sensors 54) covers the area around the front end of vehicle 1. Sensing region 101B (for example, the sensing region of multiple ultrasonic sensors 54) covers the area around the rear end of vehicle 1.

[0075] The sensing results in sensing region 101F and / or sensing region 101B may be used, for example, for parking assistance of vehicle 1.

[0076] Sensing areas 102F, 102B, 102L, and 102R are examples of sensing areas for short-range or medium-range radar 52. Sensing area 102F covers a position further in front of vehicle 1 than sensing area 101F. Sensing area 102B covers a position further in rear of vehicle 1 than sensing area 101B. Sensing area 102L covers the area around the left rear of vehicle 1. Sensing area 102R covers the area around the right rear of vehicle 1.

[0077] The sensing results in sensing region 102F may be used, for example, to detect vehicles or pedestrians in front of vehicle 1. The sensing results in sensing region 102B may be used, for example, to prevent collisions behind vehicle 1. The sensing results in sensing region 102L and / or sensing region 102R may be used, for example, to detect one or more objects in the blind spots on the left and / or right sides of vehicle 1.

[0078] Sensing areas 103F, 103B, 103L, and 103R show examples of sensing areas by camera 51. Sensing area 103F covers a position further in front of vehicle 1 than sensing area 102F. Sensing area 103B covers a position further in rear of vehicle 1 than sensing area 102B. Sensing area 103L covers the left side of vehicle 1. Sensing area 103R covers the right side of vehicle 1.

[0079] The sensing results in sensing region 103F may be used, for example, for recognition of traffic lights and traffic signs, lane departure prevention support systems, and automatic headlight control systems. The sensing results in sensing region 103B may be used, for example, for parking assistance and / or surround view systems. The sensing results in sensing region 103L and / or sensing region 103R may be used, for example, for surround view systems.

[0080] Sensing area 104 shows an example of the sensing area of ​​LiDAR 53. Sensing area 104 covers a position further in front of vehicle 1 than sensing area 103F. On the other hand, sensing area 104 has a narrower range in the left-right direction of vehicle 1 than sensing area 103F.

[0081] The sensing results in the sensing region 104 may be used, for example, to detect objects such as surrounding vehicles.

[0082] Sensing area 105 shows an example of the sensing area of ​​the long-range radar 52. Sensing area 105 covers a position further in front of vehicle 1 than sensing area 104. On the other hand, sensing area 105 has a narrower range in the left-right direction of vehicle 1 than sensing area 104.

[0083] The sensing results in the sensing area 105 may be used, for example, for ACC (Adaptive Cruise Control), emergency braking, collision avoidance, etc.

[0084] In some embodiments, the sensing areas of each sensor in the external environment recognition sensor 25 (e.g., camera 51, radar 52, LiDAR 53, ultrasonic sensor 54) may take various configurations other than those shown in Figure 2. Specifically, in some embodiments, the ultrasonic sensor 54 may also sense the sides of the vehicle 1, or the LiDAR 53 may be configured to sense the rear of the vehicle 1. Furthermore, the installation positions of each sensor are not limited to the examples described above. Also, there may be one or more sensors.

[0085] <<2. First Embodiment>> Next, the first embodiment of this technology will be described with reference to Figures 3 to 12.

[0086] The vehicle control system 11 according to the first embodiment relates to the control of automatic parking of a vehicle 1 in a parking lot, and in particular, to enabling the driver user to select a parking position according to their preference in the parking lot. Specifically, the vehicle control system 11 generates a context map by generating context information about the parking lot, such as "near the parking lot entrance" and "near the facility entrance / exit," on a parking lot map generated as a local map in the parking lot, and presents it, thereby enabling the user to automatically park in their preferred parking position.

[0087] <Parking Map and Context Map> First, with reference to Figures 3 to 5, we will explain the parking map and context map that the vehicle control system 11 generates and presents to the user.

[0088] Figure 3 is a plan view of the parking lot where automatic parking is performed in the first embodiment.

[0089] Parking lot 201 faces road 203 across sidewalk 202. Part of the perimeter of parking lot 201 is covered by wall 211. An entrance gate 212 is located at the entrance of parking lot 201, and an exit gate 213 is located at the exit. Parking lot 201 has multiple parking spaces 221 along passageway 214, and there are pillars 215 and other structures between the designated parking spaces 221. Each parking space 221 is a designated parking space for one vehicle. In addition, there is a facility entrance / exit 216 in a designated location in parking lot 201 that serves as an entrance / exit to an adjacent facility.

[0090] Figure 4 shows an example of a parking map generated for parking lot 201 in Figure 3.

[0091] The parking lot map 231 in Figure 4 is a map that visualizes landmarks recognized using sensor data from the external environment recognition sensor 25. Landmarks are static objects in the parking lot 201, including parking spaces on the road surface, arrows, white lines such as stop lines, and obstacles such as pillars and walls. In this embodiment, moving objects such as vehicles and people such as pedestrians are not included as landmarks.

[0092] The parking map 231 shows a wall 211, an entrance gate 212, an exit gate 213, a passageway 214, a pillar 215, a facility entrance / exit 216, and multiple parking spaces 221. Each parking space 221 is either a general parking space 221P, an EV parking space 221EV, or a disabled parking space 221BB. Parking spaces 221 other than EV parking spaces 221EV and disabled parking spaces 221BB are general parking spaces 221P. EV parking spaces 221EV can be identified, for example, by the presence of an EV mark or charging equipment. Disabled parking spaces 221BB can be identified, for example, by the presence of a wheelchair mark, a designated color such as blue, or by being wider than general parking spaces 221P. For example, while the width of a general parking space 221P is approximately 2.5m, the width of a parking space for disabled persons 221BB is stipulated to be 3.5m or more by the Barrier-Free Law and other regulations.

[0093] Figure 5 shows an example of a context map generated for the parking lot 201 in Figure 3.

[0094] The context map 232 in Figure 5 is a map that visualizes the context information related to the parking lot 201 in Figure 3. Context information generally refers to information that represents a predetermined meaning, situation, or scene. In the control of automatic parking in this embodiment, it refers to information that represents a predetermined meaning, situation, or scene to provide criteria for judging the automatic parking operation and for selecting the automatic parking position.

[0095] In the context map 232, context information such as "near parking lot entrance," "near parking lot exit," "near facility entrance / exit," "for EVs," and "for disabled persons" is displayed for parking spaces 221 that meet predetermined conditions. When multiple parking spaces 221 are adjacent to each other within a certain distance, these multiple parking spaces 221 are grouped into an area, and context information is displayed for each area. However, depending on the context information assigned, a single parking space 221 may also be considered an area.

[0096] In the context map 232 of Figure 5, context information 241, "Near the parking lot entrance," is displayed for the area close to the entrance gate 212 (multiple parking spaces 221). Similarly, context information 242, "Near the parking lot exit," is displayed for the area close to the exit gate 213. If it is not possible to determine whether the gate of the parking lot 201 is an entrance gate or an exit gate, context information "Near the parking lot entrance / exit" may be displayed.

[0097] In the context map 232, context information 243, "Near facility entrance / exit," is displayed for areas close to the facility entrance / exit 216. Context information 244, "For EVs," is displayed for areas with multiple EV parking spaces 221EV. Context information 245, "For disabled persons," is displayed for areas with multiple disabled parking spaces 221BB. Context information 245, "For disabled persons," is represented by a wheelchair symbol (pictogram).

[0098] Areas other than those with context information 241 for "near the parking lot entrance," 242 for "near the parking lot exit," 243 for "near the facility entrance / exit," 244 for "for EVs," and 245 for "for people with disabilities" are displayed as general areas 247, to which no specific context information is assigned.

[0099] In the context map 232, context information 251 for a "hazard area" is also displayed at a predetermined location on the passage 214. The context information 251 for a "hazard area" indicates an area that the driver user should pay attention to.

[0100] The vehicle control system 11 generates a contextual parking map by superimposing the contextual map 232 in Figure 5 onto the parking map 231 in Figure 4, and displays it on a display device such as a navigation system. The user refers to the contextual parking map displayed on the display device and selects their preferred parking space 221 from among the available parking spaces 221 as their parking location. The vehicle control system 11 then automatically drives the vehicle 1 to park in the parking space 221 selected by the user.

[0101] <Example of Vehicle Control System Configuration> Figure 6 is a block diagram showing an example of the configuration of a vehicle control system 11 according to the first embodiment.

[0102] The vehicle control system 11 includes an automatic parking control unit 301. The automatic parking control unit 301 corresponds to the part of the analysis unit 61 in Figure 3 that displays a contextual parking map and parks the vehicle in a desired parking space 221. The automatic parking control unit 301 may include at least one of the self-position estimation unit 71, sensor fusion unit 72, and recognition unit 73 of the analysis unit 61 in Figure 3, or it may be provided separately from them. The automatic parking control unit 301 also includes a map generation unit 311. The map generation unit 311 includes a parking map generation unit 321 and a context information generation unit 322.

[0103] The HMI 31 has a display unit 341 and an input unit 342. The vehicle control system 11 has a parking information storage unit 351 as part of the storage unit 28 in Figure 3. Note that the parking information storage unit 351 may be located on a cloud server instead of being part of the storage unit 28 in the vehicle 1.

[0104] The parking map generation unit 321 generates a parking map of a parking lot based on the sensing results of the external environment recognition sensor 25. For example, the parking map generation unit 321 generates the parking map 231 shown in Figure 4 for the parking lot 201 shown in Figure 3. Based on the sensor data from the external environment recognition sensor 25, the parking map generation unit 321 recognizes landmarks and moving objects such as vehicles and pedestrians, and generates a parking map as a result of the recognition.

[0105] The parking map generation unit 321 supplies the generated parking map to the context information generation unit 322 and obtains the context information generated by the context information generation unit 322. The parking map generation unit 321 generates a context parking map by adding the context information obtained from the context information generation unit 322 to the generated parking map. For example, the parking map generation unit 321 generates a context parking map by compositing the generated parking map with the context map supplied as context information from the context information generation unit 322.

[0106] The parking lot map generation unit 321 may use the location information of the vehicle 1 acquired by the location information acquisition unit 24 and the self-position estimation result by the self-position estimation unit 71 when generating the parking lot map and context map.

[0107] The parking map generation unit 321 supplies the generated context parking map to the HMI 31. Parking location information indicating the parking location selected by the user based on the context parking map is supplied from the HMI 31 to the parking map generation unit 321. Based on the parking location information supplied from the HMI 31, the parking map generation unit 321 generates target parking location information indicating the target parking location to be parked by autonomous driving, and supplies it to the action planning unit 62 along with the parking map.

[0108] The parking map generation unit 321 stores the generated context parking map in the parking information storage unit 351. The context parking map, along with the parking location information represented by the latitude and longitude of the parking lot, is stored in the parking information storage unit 351. When the user (or their vehicle 1) visits the same parking lot again, the context parking map is retrieved from the parking information storage unit 351 and used. The parking map generation unit 321 can update the context parking map stored in the parking information storage unit 351. For example, the parking map generation unit 321 checks for differences between the context parking map stored in the parking information storage unit 351 and the context parking map generated during the most recent drive, and if there are differences, it updates with the latest context parking map. Alternatively, it may update with the context parking map generated during the most recent drive at regular intervals.

[0109] The context information generation unit 322 generates context information for a parking lot based on the sensing results of the external environment recognition sensor 25 and the parking lot map from the parking lot map generation unit 321. The context information generation unit 322 may also use sensing results from sensors other than the external environment recognition sensor 25, such as the vehicle sensor 27, to generate context information. For example, for the parking lot 201 shown in Figure 3, the context information generation unit 322 generates the context map 232 shown in Figure 5 as context information. The context information generation unit 322 supplies the generated context map to the parking lot map generation unit 321.

[0110] The context information generated by the context information generation unit 322 does not have to be a map as shown in Figure 5. For example, the context information generation unit 322 may identify a predetermined area on the parking lot map 231 and associate that predetermined area with the context information. In this case, the parking lot map generation unit 321, which has acquired the context information, adds the context information to the parking lot map to generate a context parking lot map.

[0111] The action planning unit 62 uses at least the target parking location information and parking map supplied from the parking map generation unit 321 to generate (plan) a route to the target parking location, and supplies the generated route to the operation control unit 63.

[0112] The motion control unit 63 controls the movement of vehicle 1 so that it moves along the trajectory calculated by the route plan. As a result, vehicle 1 is automatically driven to the target parking position specified by the user and parked.

[0113] The display unit 341 is composed of a display device such as an LCD or projector. The display unit 341 displays and presents to the user a contextual parking map supplied from the parking map generation unit 321.

[0114] The input unit 342 is comprised of input devices such as a touch panel, buttons, switches, and levers. The input unit 342 functions as an acquisition unit that obtains information entered by the user. For example, the input unit 342 obtains the parking location selected by the user based on the presented contextual parking map. Parking location information indicating the parking location selected by the user is supplied from the HMI 31 to the parking map generation unit 321.

[0115] The parking information storage unit 351 stores the context parking map generated by the parking map generation unit 321, linking it to the parking location information. The parking information storage unit 351 may store the parking map and the context map separately. The parking information storage unit 351 stores parking maps of parking lots that have been visited in the past. Parking maps of parking lots that have not been visited in the past may be obtained from a cloud server or the like and stored in advance.

[0116] <Generation of Context Information> Figure 7 is a diagram illustrating the generation of context information by the context information generation unit 322.

[0117] The context information generation unit 322 can generate various types of context information shown in the context map 232 of Figure 5 based on the following criteria. It should be assumed that the navigation device does not possess map information such as the plot information within the parking lot 201 shown in Figure 3.

[0118] For context information 241 near the parking lot entrance and context information 242 near the parking lot exit, the context information generation unit 322 first uses the parking lot map 231 supplied by the parking lot map generation unit 321, the recognition results based on sensor data from the external environment recognition sensor 25, and the vehicle's trajectory information to recognize parking spaces 221 that match the following criteria. For example, parking spaces 221 close to the location where the entrance gate 212 or exit gate 213 was recognized, parking spaces 221 close to the sidewalk 202 recognized before entering the parking lot 201, parking spaces 221 close to the location where the GPS signal (an example of a GNSS signal) became unstable when entering the parking lot 201 if the parking lot 201 is underground or indoors, parking spaces 221 close to the location where the GPS signal became unstable, and parking spaces 221 close to the location where the vehicle deviated from the road 203 indicated by the navigation device and entered the parking facility area are recognized. Only one of these criteria may be used, or multiple criteria may be used. The context information generation unit 322 then divides one or more of the recognized parking spaces 221 within a certain distance from the entrance or exit of the parking lot 201 along the vehicle's trajectory into areas, and generates and assigns context information 241 for "near the parking lot entrance" or context information 242 for "near the parking lot exit" to each area.

[0119] Regarding the context information 243 for "near facility entrance / exit," the context information generation unit 322 first uses the parking map 231 supplied by the parking map generation unit 321, the recognition results based on sensor data from the external environment recognition sensor 25, and the vehicle's trajectory information to recognize parking spaces 221 that match the following criteria. For example, parking spaces 221 close to a location where a glass door was recognized, parking spaces 221 close to a location where signs indicating entrances / exits or emergency exits were recognized, parking spaces 221 close to areas with frequent pedestrian traffic, parking spaces 221 close to a parking space for disabled persons, and parking spaces 221 close to an EV parking space are recognized. Only one of these criteria may be used, or multiple criteria may be used. Then, the context information generation unit 322 areas one or more parking spaces 221 within a certain distance from the facility entrance / exit 216 on the vehicle's trajectory, and generates and assigns the context information 243 for "near facility entrance / exit" to each area.

[0120] Regarding the "EV-specific" context information 244, the context information generation unit 322 first uses the parking map 231 supplied by the parking map generation unit 321 and the recognition results based on the sensor data of the external environment recognition sensor 25 to recognize parking spaces 221 that match the following criteria. For example, parking spaces 221 with an EV mark, parking spaces 221 with charging equipment, parking spaces 221 close to a parking lot entrance / exit, and parking spaces 221 close to a facility entrance / exit are recognized. Only one of these criteria may be used, or multiple criteria may be used. The context information generation unit 322 then divides one or more of the recognized parking spaces 221 into areas and generates and assigns the "EV-specific" context information 244 to each area.

[0121] Regarding the context information 245 for "parking spaces for people with disabilities," the context information generation unit 322 first uses the parking map 231 supplied by the parking map generation unit 321 and the recognition results based on the sensor data of the external environment recognition sensor 25 to recognize parking spaces 221 that match the following criteria. For example, parking spaces 221 with a wheelchair symbol, parking spaces 221 with a wider width than usual, parking spaces 221 with a predetermined color such as blue, and parking spaces 221 close to the facility entrance are recognized. Only one of these criteria may be used, or multiple criteria may be used. Then, the context information generation unit 322 divides one or more of the recognized parking spaces 221 into areas and generates and assigns the context information 245 for "parking spaces for people with disabilities" to each area.

[0122] Regarding the context information 251 for "hazard areas," the context information generation unit 322 first uses the parking map 231 supplied by the parking map generation unit 321, the sensor data from the external environment recognition sensor 25, and the sensor data from the vehicle sensor 27 to recognize locations that meet the following criteria. For example, locations with frequent pedestrian traffic or locations where the vehicle has made sudden stops or sharp turns in the past may be recognized. Only one of these criteria may be used, or multiple criteria may be used. The context information generation unit 322 then generates and assigns the context information 251 for "hazard areas" to areas within a certain distance from the recognized locations.

[0123] <Contextual Parking Map Generation and Presentation Process> Next, the contextual parking map generation and presentation process by the vehicle control system 11 according to the first embodiment will be described with reference to the flowchart in Figure 8. This process is started, for example, when the vehicle control system 11 determines that vehicle 1 has arrived at the target parking lot. Alternatively, it may be started, for example, when vehicle 1 has arrived at the target parking lot and the user instructs the start of an operating mode such as automatic parking mode displayed on the navigation system's display device. Hereinafter, the parking lot to which vehicle 1 has arrived will also be referred to as the arrival parking lot.

[0124] First, in step S11, the parking map generation unit 321 determines whether or not a context parking map of the destination parking lot is stored in the parking information storage unit 351. If vehicle 1 has visited the same parking lot in the past, a previously generated context parking map is stored in the parking information storage unit 351. The parking map generation unit 321 determines whether or not a context parking map of the destination parking lot is stored in the parking information storage unit 351 by detecting whether or not the location information of the destination parking lot linked to the context parking map in the parking information storage unit 351 matches the current location information of vehicle 1.

[0125] If, in step S11, it is determined that the context parking map of the arrival parking lot is stored in the parking information storage unit 351, the process proceeds to step S12, and the parking map generation unit 321 retrieves the context parking map of the arrival parking lot stored in the parking information storage unit 351. If, in step S11, it is determined that the context parking map of the arrival parking lot is not stored in the parking information storage unit 351, the process in step S12 is skipped.

[0126] In step S13, the parking map generation unit 321 generates a parking map of the arrived parking lot based on the sensing results of the external environment recognition sensor 25. The generated parking map of the parking lot is supplied to the context information generation unit 322.

[0127] In step S14, the context information generation unit 322 generates context information for the parking lot based on the judgment criteria described in Figure 7, using the sensing results of the external environment recognition sensor 25 and the parking lot map from the parking lot map generation unit 321. For example, in the parking lot 201 in Figure 3, the context information generation unit 322 recognizes the entrance gate 212 and generates context information 241 of "near the parking lot entrance" for an area of ​​one or more parking spaces 221 within a certain distance from the entrance gate 212. Alternatively, the context information generation unit 322 recognizes a parking space 221 with an EV mark drawn on it, defines an area of ​​multiple adjacent parking spaces 221 with EV marks drawn on them within a certain distance, and generates context information 244 of "for EVs". The context information generation unit 322 supplies the generated context information for the parking lot to the parking lot map generation unit 321.

[0128] In step S15, the parking map generation unit 321 generates a context parking map by adding context information obtained from the context information generation unit 322 to the parking map, and supplies it to the HMI 31. If the context parking map of the arrival parking lot stored in the parking information storage unit 351 was obtained in step S12 described above, the context parking map updated as necessary by the update process described later in Figure 10 is supplied to the HMI 31.

[0129] In step S16, the presentation unit 341 of the HMI 31 presents the context parking map supplied by the parking map generation unit 321 to the user. For example, the presentation unit 341 presents the context parking map to the user by displaying it on the display device of the navigation system.

[0130] Figure 9 shows an example screen displaying a contextual parking map on the navigation system's display device.

[0131] The screen 233 in Figure 9 shows an example of a screen displayed while vehicle 1 is driving near the entrance gate 212 of parking lot 201 in Figure 3.

[0132] Screen 233 displays a contextual parking map, which is an overlay of the parking map and context map recognized in the vicinity of the entrance gate 212 of the parking lot 201 in Figure 3. For example, contextual information 241, "Near the parking lot entrance," is displayed for some of the recognized parking spaces 221. Also, on screen 233, the user's own vehicle 401A is displayed at the user's own position, and other parked vehicles 401B are displayed in some of the parking spaces 221. The message "Please specify a parking location" is also displayed on screen 233.

[0133] Furthermore, if the user has previously visited parking lot 201 in Figure 3, and the context parking map of the arrival parking lot is stored in the parking information storage unit 351, then the screen 233 can display the context parking map of the entire parking lot, as shown in Figures 4 and 5.

[0134] In step S17 of Figure 8, the input unit 342 determines whether the user has specified a parking location based on the presented contextual parking map. If it is determined in step S17 that the user has not yet specified a parking location, the process returns to step S13, and the processes described in steps S13 to S17 are repeated. As a result, the contextual parking map is updated according to the movement of vehicle 1 and presented to the user.

[0135] If it is determined in step S17 that the user has specified a parking position, the process proceeds to step S18, where the input unit 342 supplies parking position information indicating the parking position (parking space) specified by the user to the parking map generation unit 321. The parking map generation unit 321 then acquires the parking position information and executes the automatic parking process. Specifically, the parking map generation unit 321 supplies target parking position information indicating the parking position specified by the user as the target parking position, along with the parking map, to the action planning unit 62. The action planning unit 62 creates a route plan for vehicle 1 to the target position and supplies it to the operation control unit 63. The operation control unit 63 controls the operation of vehicle 1 so that vehicle 1 proceeds along the trajectory calculated by the route plan. As a result, vehicle 1 is automatically driven to the target parking position specified by the user and parked.

[0136] In step S19, the parking map generation unit 321 stores the generated context parking map in the parking information storage unit 351. If the context parking map has been updated by the update process described later in Figure 10, the updated context parking map is stored in the parking information storage unit 351. Steps S13 to S15 may also be executed during the automated driving to the target parking position, and the context parking map updated during that time may be stored in the parking information storage unit 351.

[0137] This completes the contextual parking map generation and presentation process.

[0138] Next, referring to the flowchart in Figure 10, we will describe the context parking map update process, which updates the context parking map as needed using the context parking map generated in the most recent drive, when the context parking map of the arrival parking lot is stored in the parking information storage unit 351. This process is executed in parallel with the generation of the context parking map in step S15 of Figure 8.

[0139] In step S41, the parking map generation unit 321 calculates the degree of agreement between the context parking map of the arrival parking lot obtained from the parking information storage unit 351 and the context parking map generated in the most recent drive.

[0140] In step S42, the parking map generation unit 321 determines whether the stored context parking map matches the context parking map generated in the most recent drive. For example, if the degree of match between the stored context parking map and the context parking map generated in the most recent drive is greater than or equal to a predetermined threshold, the two context parking maps are determined to be correlated and match. Conversely, if the degree of match is lower than the predetermined threshold, the two context parking maps are determined to be uncorrelated and not match. If it is determined in step S42 that the two context parking maps match, the process returns to step S41 described above. Therefore, if the two context parking maps match, the context parking map is not updated.

[0141] On the other hand, if it is determined in step S42 that the two context parking maps do not match, the process proceeds to step S43, where the parking map generation unit 321 determines whether the accuracy of the self-position estimation by the self-position estimation unit 71 has deteriorated. If it is determined in step S43 that the accuracy of the self-position estimation has not deteriorated, the process returns to step S41 described above. Therefore, if the accuracy of the self-position estimation is maintained well, the accuracy of the self-position estimation for the current drive and the parking map are accurate, and the context parking map is not updated.

[0142] On the other hand, if in step S43 the accuracy of self-position estimation is below a predetermined value and it is determined that the accuracy of self-position estimation has deteriorated, the process proceeds to step S44, and the parking map generation unit 321 statistically updates the stored context parking map. More specifically, the parking map generation unit 321 checks the difference between the stored context parking map and the context information generated from the most recent multiple drives and updates it statistically. Here, "statistically updating" means updating the context information of the context parking map by determining whether the context information has been detected stably multiple times in multiple drives, expressing the recognition and extraction accuracy of the context information as a score, calculating the probability of correct answering the context information of a landmark, and adopting the context information for each landmark whose probability of correct answering is above a predetermined value.

[0143] After step S44, the process returns to step S41, and the processes described in steps S41 to S44 are repeated.

[0144] Through the above contextual parking map update process, the contextual parking map is updated as needed, and the updated contextual parking map is presented to the user.

[0145] In the context parking map update process shown in Figure 10, the parking map generation unit 321 statistically updates the stored context parking map if it determines in step S42 that the two context parking maps do not match, and in step S43 that the accuracy of self-localization has deteriorated. However, the context parking map may be statistically updated only if either the two context parking maps do not match or the accuracy of self-localization has deteriorated is met.

[0146] The update process in Figure 10 is a process that updates the context parking map as needed when it is generated during the most recent drive, but it may also be set up to update periodically after a certain period of time has elapsed. Since facilities such as parking lots are considered to change less than roads, it is not necessary to consider updating them very often, and for example, updates may be performed periodically after a period of six months or one year. Even when performing periodic updates, for example, the parking map generation unit 321 stores the generated context parking map in the parking information storage unit 351 each time it visits a parking lot, checks the difference between the stored context parking map and the context information generated during the most recent multiple drives, and updates it statistically.

[0147] The parking information storage unit 351 may not be part of the storage unit 28 inside the vehicle 1, but may be located on a cloud server. Also, if the context parking map is located in the parking information storage unit 351 on the cloud server, the context parking map may be generated based on data collected by multiple users (drivers), rather than generating a context parking map for each individual user. In this case, the parking map generation unit 321 checks for differences between the context parking map stored on the cloud server and the context information generated from the latest multiple drives acquired by multiple users, and updates it statistically.

[0148] <Summary of the First Embodiment> According to the vehicle control system 11 of the first embodiment described above, when the user's vehicle 1 arrives at the parking lot, contextual information about the parking lot, such as "near the parking lot entrance" and "near the facility entrance / exit," is generated to create a context map, which is then displayed to the user, for example, by being superimposed on the parking lot map. This allows the user to specify their preferred parking location, such as a parking space near the facility entrance / exit or a parking space near the parking lot exit, and have the vehicle automatically parked. Therefore, the user can select a parking location according to their preference and have the vehicle automatically parked. Since prior registration of parking locations is not required, there is no need to worry about the pre-registered parking location being unavailable.

[0149] According to contextual information 251 for "hazard areas," even in the absence of clear landmarks such as pedestrian crossings, it is possible to encourage users to drive safely, such as by slowing down or stopping.

[0150] <Another example of a context map> Figure 11 shows another example of a context map generated by the parking lot map generation unit 321.

[0151] In the context map 232 of Figure 11, in addition to the context map 232 shown in Figure 5, context information 261 for "covered areas" and context information 262 for "outdoor areas" are further added. In the context information 261 for "covered areas" in Figure 11, context information is assigned to all parking spaces 221 that meet the same condition of being "covered" as a single area. However, as in the example above, the context information 261 for "covered areas" may also be assigned to area units that consist of multiple adjacent parking spaces 221 within a certain distance.

[0152] <Context Information Correction Function> False detections or omissions may occur in the context information generated by the context information generation unit 322. The map generation unit 311 may have a function to correct the context information based on user instructions.

[0153] Figure 12 shows an example of modifying the context map based on user correction instructions.

[0154] The left side of Figure 12 shows an example of the context map 232 before modification, and the right side of Figure 12 shows an example of the context map 232 after modification. The modified context map 232 is identical to the context map 232 shown in Figure 5.

[0155] In the uncorrected context map 232 on the left side of Figure 12, the context information 244 for "EV use" is assigned to five parking spaces 221, consisting of three EV parking spaces 221EV and two adjacent general parking spaces 221P. The user operates the touch panel or the like to modify the context information 244 for "EV use" to correspond to the three EV parking spaces 221EV on the right, and modifies the two parking spaces 221 on the left to be assigned to the general area 247. The modified data for the context map 232 is supplied from the input unit 342 of the HMI 31 to the parking map generation unit 321, and the context map and context parking map are modified.

[0156] <<3. Second Embodiment>> Next, a second embodiment of the present technology will be described with reference to Figures 13 to 22.

[0157] In the first embodiment described above, the vehicle control system 11 was configured to present the user with a contextual parking map, which included a parking map and a contextual map, and for the user to select and indicate a desired parking location based on the presented contextual parking map.

[0158] In the second embodiment, the vehicle control system 11 is configured to present (suggest) a predetermined parking position to the user from among the available parking spaces 221. For example, the vehicle control system 11 presents a predetermined available parking space 221 to the user based on the situation of parked vehicles and obstacles around the available parking space 221. Alternatively, the vehicle control system 11 may register the user's preferred parking position in advance and present the user with a parking space 221 that matches the user's preference from among the available parking spaces 221. Furthermore, the vehicle control system 11 may learn the user's preferred parking position based on the parking position the user has previously selected and present a predetermined available parking space 221 to the user based on the learning result.

[0159] <Example of Vehicle Control System Configuration> Figure 13 is a block diagram showing an example of the configuration of a vehicle control system 11 according to the second embodiment.

[0160] In Figure 13, parts corresponding to the first embodiment shown in Figure 6 are denoted by the same reference numerals. Descriptions of these parts will be omitted as appropriate, and the explanation will focus on the different parts.

[0161] In the second embodiment, the automatic parking control unit 301 is newly equipped with a parking position candidate determination unit 361 and a user preference information storage unit 362, in addition to the map generation unit 311.

[0162] In the first embodiment, the context parking map generated by the parking map generation unit 321 of the map generation unit 311 was supplied to the HMI 31, but in the second embodiment, it is supplied to the parking position candidate determination unit 361. In addition, sensor data is supplied to the parking position candidate determination unit 361 from the external environment recognition sensor 25 and the in-vehicle sensor 26, respectively.

[0163] The parking position candidate determination unit 361 determines the status of parked vehicles and obstacles around the available parking spaces 221 based on the contextual parking map and determines parking position candidates to present to the user. The parking position candidate determination unit 361 supplies the determined parking position candidates and the contextual parking map to the HMI 31. Furthermore, if user preference information, such as the user's preferred parking position, is stored in the user preference information storage unit 362, the parking position candidate determination unit 361 determines parking position candidates to present to the user based on the user preference information. In addition, the parking position candidate determination unit 361 can determine parking position candidates to present to the user by recognizing the user attributes of passengers, including the driver, based on sensor data from the in-vehicle sensor 26.

[0164] The user preference information storage unit 362 stores the user's preference information and user attributes. The user's preference information and user attributes can be registered in advance by the user. The user preference information is preference information related to the context information generated by the parking map generation unit 321, such as prioritizing "near facility entrances and exits," prioritizing parking spaces 221 that are adjacent to an empty space, or prioritizing "covered areas." User attributes are information such as the age and gender of the driver and passengers. The user preference information may also be information generated by the parking location candidate determination unit 361 based on information such as parking location information previously selected by the user.

[0165] The presentation unit 341 presents the user with a contextual parking map and parking location candidates supplied by the parking location candidate determination unit 361. If the user is satisfied with one of the presented parking location candidates, they perform an operation to specify that parking location as their parking location. On the other hand, if the user wants to park in a parking space 221 that is not one of the presented parking location candidates, they perform an operation to change their parking location to another available parking space 221. The input unit 342 acquires the parking location selected by the user. Parking location information indicating the parking location selected by the user is supplied from the HMI 31 to the parking location candidate determination unit 361. The parking location candidate determination unit 361 acquires parking location information indicating the parking location selected by the user from the HMI 31. Based on the parking location information, the parking location candidate determination unit 361 generates target parking location information indicating a target parking location to be parked in by autonomous driving, and supplies it to the action planning unit 62 along with the parking map.

[0166] <Example of presenting parking location options> Referring to Figures 14 to 19, an example of presenting parking location options to the user based on information such as a contextual parking map will be explained.

[0167] Figure 14 is a plan view showing a first example of a presentation of parking location candidates.

[0168] In Figure 14, in front of vehicle 401A, there are six parking spaces 221A to 221F separated by a passageway 214, with vehicle 401B parked in parking spaces 221D and 221F.

[0169] The parking position candidate determination unit 361 determines the availability of parking spaces 221, the presence of obstacles, etc., based on the contextual parking map, and determines parking position candidates to present to the user. For example, from the standpoint of ease of parking and getting in and out of the vehicle when entering the parking lot, and avoidance of collisions with obstacles, the parking position candidate determination unit 361 determines parking spaces 221 that do not have vehicles or obstacles in adjacent parking spaces 221, or parking spaces 221 that are far enough away from adjacent vehicles or obstacles, as parking position candidates to present to the user.

[0170] In the situation shown in Figure 14, of the available parking spaces 221A to 221C and 221E, other vehicles 401B are parked in the adjacent parking spaces 221D and 221F. Therefore, the parking space candidate determination unit 361 lowers the priority of parking space 221E as a parking space candidate. The parking space candidate determination unit 361 then determines one of the remaining parking spaces 221A to 221C, for example, parking space 221B, which allows the adjacent parking spaces 221A and 221C to be kept empty, as the parking space candidate. The parking space candidate determination unit 361 then has the presentation unit 341 present a context map 232 in which the parking space candidate frame 411, which indicates the parking space candidate, is superimposed on parking space 221B.

[0171] Figure 15 is a plan view showing a second example of presenting parking location options.

[0172] In Figure 15, of the six parking spaces 221A to 221F, a small vehicle 401B is parked in parking space 221A, and a pylon 412, which is an obstacle, is placed in parking space 221C. Larger vehicles 401B are parked in parking spaces 221D and 221F.

[0173] In the situation shown in Figure 14, parking spaces 221B and 221E are available. Between parking spaces 221B and 221E, parking space 221B, which has a pylon 412 next to it, is further from adjacent vehicles and obstacles than parking space 221E, which has large vehicles 401B parked on both sides. The parking space candidate determination unit 361 lowers the priority of parking space 221E as a parking space candidate and determines parking space 221B as the parking space candidate. The parking space candidate determination unit 361 causes the presentation unit 341 to present a context map 232 in which the parking space candidate frame 411 indicating the parking space candidate is superimposed on parking space 221B.

[0174] Figure 16 is a plan view showing a third example of a parking location candidate.

[0175] In Figure 16, of the six parking spaces 221A to 221F, another vehicle 401B is parked in parking spaces 221B, 221D, and 221F. A wall 211 is located around parking spaces 221A to 221F. The distance between parking spaces 221A and 221D and wall 211 is short, while the distance between parking spaces 221C and 221F and wall 211 is long due to the presence of a passageway 214.

[0176] The parking position candidate determination unit 361 determines parking spaces 221 that are easy for the driver to enter and exit, taking into account factors such as the number of maneuvers required when entering and exiting the parking space, and the distance from obstacles, and presents these as parking position candidates to the user.

[0177] In the situation shown in Figure 16, of the available parking spaces 221A and 221C, parking space 221A is difficult to park in reverse due to the adjacent wall 211. Parking is possible in reverse, but in that case, it is necessary to back out, making driving difficult. Parking space 221C can be parked in reverse by using the adjacent passage 214, and if reverse parking is possible, it is possible to back out, making driving easy. The parking space candidate determination unit 361 lowers the priority of parking space candidates for parking space 221A and determines parking space 221C as the parking space candidate. The parking space candidate determination unit 361 causes the presentation unit 341 to present a context map 232 in which the parking space candidate frame 411 indicating the parking space candidate is superimposed on parking space 221C.

[0178] Figure 17 is a plan view showing a fourth example of a suggested parking location.

[0179] In Figure 17, of the six parking spaces 221A to 221F, parking spaces 221B, 221D, and 221E are occupied by another vehicle 401B. A wall 211 surrounds parking spaces 221A to 221F. The distance between parking spaces 221A and 221D and wall 211 is short, while the distance between parking spaces 221C and 221F and wall 211 is long due to the presence of a passageway 214. In addition, parking spaces 221D to 221F are assigned the context information 261 of a "covered area".

[0180] The parking position candidate determination unit 361 determines parking spaces 221 that are not affected by or are less affected by weather conditions such as rain or snow, such as "covered areas," as parking position candidates to present to the user.

[0181] In the situation shown in Figure 17, among the available parking spaces 221A, 221C, and 221F, parking space 221A is given a lower priority for the same reasons as in the third example shown in Figure 16. Between parking spaces 221C and 221F, parking space 221F, which is in a "covered area" unaffected by weather, is given priority, and the priority of parking space 221C is lowered, or the priority of parking space 221F is raised.

[0182] The parking position candidate determination unit 361 determines the parking space 221F as a parking position candidate. The parking position candidate determination unit 361 causes the presentation unit 341 to present a context map 232 in which the parking position candidate frame 411 indicating the parking position candidate is superimposed on the parking space 221F.

[0183] Figure 18 is a perspective view showing a fifth example of a presentation of parking location options.

[0184] In Figure 18, the multi-story parking garage is assigned context information 413 for "indoors" and context information 413 for "rooftop".

[0185] The parking position candidate determination unit 361 prioritizes "indoor" parking spaces 221 that are not affected by weather, and determines a predetermined parking space 221G from among the available "indoor" parking spaces 221 as a parking position candidate. The parking position candidate determination unit 361 causes the presentation unit 341 to present a context map 232 in which the parking position candidate frames 411 indicating the parking position candidates are superimposed on the parking space 221G.

[0186] For example, a user can register a parking space they have parked in the past as their preferred parking space. The registered preferred parking spaces are stored in the user preference information storage unit 362 as user preference information. Preferred parking spaces for parking lots that the user has not visited in the past may also be stored in the user preference information storage unit 362 as user preference information. For example, a preferred parking space can be registered in the parking information storage unit 351 by obtaining a parking map or context parking map of a parking lot that the user has not visited in the past from a cloud server or the like in advance. For example, suppose parking space 221G is stored as a preferred parking space. If parking space 221G, which is registered as a preferred parking space in the user preference information storage unit 362, is available, the parking space candidate determination unit 361 will determine parking space 221G as a parking space candidate. In this case as well, a context map 232 in which the parking space candidate space 411 is superimposed on parking space 221G is presented to the presentation unit 341.

[0187] Figure 19 is a plan view showing a sixth example of a suggested parking location.

[0188] In Figure 19, of the six parking spaces 221A to 221F, a small vehicle 401B is parked in parking space 221A, while larger vehicles 401B are parked in parking spaces 221C, 221D, and 221F.

[0189] In the situation shown in Figure 19, parking space 221B is an occlusion area that cannot be seen by camera 51 due to another large vehicle 401B parked in parking space 221C. Therefore, to the camera 51 mounted on vehicle 401A, parking spaces 221B and 221E appear to be empty.

[0190] Therefore, the parking position candidate determination unit 361 recognizes the possibility that another parked vehicle exists in parking space 221B based on the sensor data from the external environment recognition sensor 25, specifically the engine sound of another vehicle 401B detected by the microphone 55, and lowers the priority of parking position candidates for parking space 221B, determining parking space 221E as the parking position candidate. The parking position candidate determination unit 361 then has the presentation unit 341 present a context map 232 in which the parking position candidate frame 411, which indicates the parking position candidate, is superimposed on the parking space 221E.

[0191] In the first to sixth examples presented above, the parking position candidate determination unit 361 may exclude parking spaces 221 with lower priority from the list of parking position candidates, rather than lowering their priority.

[0192] <Registration of User Preference Information> The parking location candidate determination unit 361 can generate user preference information using the result of the parking location selected by the user from the parking location candidates presented to the user, and store it in the user preference information storage unit 362. The parking location candidate determination unit 361 generates and stores user preference information using context information, such as prioritizing "near facility entrances," prioritizing parking spaces 221 with empty spaces next to them, or prioritizing "covered areas," based on the relationship between the context information of the context map and the parking location selected by the user. If user preference information, such as the user's preferred parking location, is stored in the user preference information storage unit 362, the parking location candidate determination unit 361 can determine the parking location candidates to present to the user based on the user preference information.

[0193] In addition to generating user preference information from the user's parking location selection as described above, user preference information can also be registered in advance.

[0194] Figure 20 shows an example of a preference information registration screen for registering user preference information.

[0195] The preference information registration screen 421 in Figure 20 is displayed, for example, on the display device of a navigation system. The preference information registration screen 421 displays the message "Please specify your preferred parking location" along with several selection buttons 422 related to preferred parking locations. Selection button 422A is selected when prioritizing a parking space 221 close to the entrance / exit of the parking lot. Selection button 422B is selected when prioritizing a parking space 221 close to the entrance / exit of the facility. Selection button 422C is selected when prioritizing a parking space 221 with an empty space next to it. Selection button 422D is selected when prioritizing a parking space 221 in a covered area. Selection button 422E is selected when prioritizing a parking space 221 for electric vehicles. Selection button 422F is selected when prioritizing a parking space 221 for disabled persons.

[0196] The user can specify, for example, up to three selection buttons 422 from among the multiple selection buttons 422 on the preference information registration screen 421 as information regarding preferred parking locations. In the example in Figure 20, the user has selected and specified selection button 422B to prioritize parking spaces 221 near the facility entrance and selection button 422D to prioritize parking spaces 221 in the covered area. The parking location candidate determination unit 361 determines parking location candidates based on the user's preference information registered in advance and the contextual parking map.

[0197] Furthermore, if no specific user preference information is designated on the preference information registration screen 421 in Figure 20, preference information that is generally considered to be preferred by the user may be used as pre-registered preference information. Pre-registered preference information may also be preset on the preference information registration screen 421 in Figure 20.

[0198] <Determination of parking location candidates based on user attributes> The parking location candidate determination unit 361 recognizes the user attributes of the occupant based on the sensor data from the in-vehicle sensor 26, and can determine parking location candidates based on the user attributes and the contextual parking map.

[0199] For example, the parking position candidate determination unit 361 recognizes whether there is only one person (the driver) or multiple people inside the vehicle. The parking position candidate determination unit 361 also recognizes information such as the age and gender of the people inside the vehicle. If there is only one person (the driver) inside the vehicle, priority is determined based on the driver's preference information mentioned above, and a parking position candidate is determined. For example, if an elderly person or a child is recognized as being inside the vehicle, the parking position candidate determination unit 361 considers the ease of getting in and out of the vehicle 1, increases the priority of parking spaces 221 with an empty space next to them, and determines a parking position candidate.

[0200] As described above, the parking position candidate determination unit 361 can recognize the user attributes of the passengers, determine a priority based on the user attributes and the contextual parking map, and determine a parking position candidate based on the priority. In addition to real-time recognition by the in-vehicle sensor 26, the user attributes of one or more users riding in the vehicle 1 may be registered in advance, as shown in the preference information registration screen 421 in Figure 20. User attributes and user preference information may be registered for each of the one or more users riding in the vehicle. The parking position candidate determination unit 361 can determine a parking position candidate using the registered user attributes and user preference information.

[0201] <Contextual Parking Map Generation and Presentation Process> Next, the contextual parking map generation and presentation process by the vehicle control system 11 according to the second embodiment will be explained with reference to the flowchart in Figure 21. The start of this process is the same as the contextual parking map generation and presentation process of the first embodiment described in Figure 8.

[0202] First, in step S61, the parking map generation unit 321 acquires a context parking map of the arrived parking lot (arrival parking lot) and supplies it to the parking position candidate determination unit 361. In the process of step S61, the context parking map may be acquired by reading it from the parking information storage unit 351 as in step S12, or it may be acquired by generating a context parking map by executing the processes of steps S11 to S15 in Figure 8.

[0203] In step S62, the parking location candidate determination unit 361 determines parking location candidates to present to the user, using at least the contextual parking map. When determining parking location candidates, for example, as explained with reference to Figures 14 to 19, the priority of the parking space 221 can be determined by judging the situation of parked vehicles and obstacles around the available parking space 221, and a parking location candidate can be determined. Alternatively, the parking location candidate determination unit 361 can determine parking location candidates based on the contextual parking map and user preference information. Or, the parking location candidate determination unit 361 can determine parking location candidates based on the contextual parking map and user attributes.

[0204] In step S63, the parking location candidate determination unit 361 supplies the determined parking location candidate and the context parking map to the HMI 31. The presentation unit 341 of the HMI 31 presents the parking location candidate and context parking map supplied by the parking location candidate determination unit 361 to the user. For example, the presentation unit 341 presents them to the user by displaying the parking location candidate and context parking map on the display device of the navigation system.

[0205] If the user is satisfied with one of the suggested parking space options, they will perform an operation to specify that option as their parking space. On the other hand, if the user wants to park in a parking space 221 other than one of the suggested options, they will perform an operation to change their parking space to another available parking space 221.

[0206] In step S64, the input unit 342 determines whether the user has specified a parking location based on the presented context parking map. If it is determined in step S64 that the user has not yet specified a parking location, the process returns to step S61, and the processes described in steps S61 to S64 are repeated. As a result, the context parking map is updated according to the movement of vehicle 1 and presented to the user.

[0207] If it is determined in step S64 that the user has specified a parking space, the process proceeds to step S65, where the input unit 342 supplies parking space information indicating the parking space (parking frame) specified by the user to the parking space candidate determination unit 361. The parking space candidate determination unit 361 then acquires the parking space information and executes the automatic parking process. This process is the same as the process in step S18 of Figure 8.

[0208] In step S66, the parking map generation unit 321 stores the generated context parking map in the parking information storage unit 351. This process is the same as the process in step S19 in Figure 8.

[0209] In step S67, the parking location candidate determination unit 361 generates user preference information based on the parking location specified by the user and stores it in the user preference information storage unit 362. For example, the parking location candidate determination unit 361 generates and stores user preference information using context information, such as prioritizing "near facility entrances," prioritizing parking spaces 221 that are adjacent to empty spaces, or prioritizing "covered areas," based on the relationship between the context information of the context map and the parking location selected by the user. In other words, the parking location candidate determination unit 361 may generate and update user preference information based on the parking location selected by the user from among the parking location candidates.

[0210] This completes the contextual parking map generation and presentation process.

[0211] According to the contextual parking map generation and presentation process of the second embodiment described above, the optimal parking location candidate can be determined and presented to the user.

[0212] <Fully Automatic Parking Process> In the example described above, the system was configured to determine and present the optimal parking position candidate to the user, allowing the user to accept the presented parking position candidate or change to another parking position. However, by omitting the user's operation to change to another parking position, the parking position candidate determination unit 361 determines the parking position candidate as the target parking position, and supplies target parking position information indicating the target parking position to the action planning unit 62, a fully automatic parking process can be achieved. The presentation unit 341 can display a notification to the user of the target parking position determined by the parking position candidate determination unit 361. Since the user does not need to perform a parking position selection operation, operability and convenience can be improved.

[0213] <Updating Parking Position Candidates> The parking position candidate determination unit 361 determines a target parking position, and while driving towards the target parking position, a parking position (available parking space 221) that better matches the user's preferences or user attributes may be detected. Based on the sensing results of the external environment recognition sensor 25 during driving, the parking position candidate determination unit 361 can change the parking position candidate determined as the target parking position.

[0214] Figure 22 illustrates the process of updating parking location candidates by the parking location candidate determination unit 361.

[0215] At a certain point in time, vehicle 1 is stopped at the position of vehicle 401C. At the position of vehicle 401C, vehicle 1 can recognize the parking space 221 and other vehicles 401B within the range indicated by the recognition area 431. At the position of vehicle 401C, the parking position candidate determination unit 361 determines parking space 221X as the target parking position, and automatic parking is started. On the screen displayed on the display unit 341, the target parking position frame 441A is displayed in the parking space 221X, indicating that it is the target parking position.

[0216] For example, suppose that in the preference information registration screen 421 of Figure 20, the selection button 422B that prioritizes parking space 221 closer to the facility entrance is selected and registered as the user's preference information. During automatic parking, at the position of the vehicle 401D, a parking space 221Y that better matches the user's preference is detected. Since parking space 221Y is closer to the facility entrance than parking space 221X, it is a parking position that better matches the user's preference. The recognition area 432 indicates the range in which parking space 221 and other vehicles 401B can be recognized at the position of the vehicle 401D.

[0217] At the position of the vehicle 401D, the parking position candidate determination unit 361 updates the parking position candidate from parking frame 221X to parking frame 221Y. The parking position candidate determination unit 361 causes the display unit 341 to display a screen indicating that the parking position candidate has been changed from parking frame 221X to parking frame 221Y. When the user performs an operation on the HMI 31 to change the parking position candidate to parking frame 221Y, the parking position candidate determination unit 361 changes the target parking position from parking frame 221X to parking frame 221Y and supplies the new target parking position information to the action planning unit 62. On the screen displayed on the display unit 341, the target parking position frame 441A that was displayed in parking frame 221X is erased, and the target parking position frame 441B is displayed in parking frame 221Y. It is desirable that the change in the parking position candidate occurs before the vehicle 1 begins a forward turning operation relative to the target parking position. In the case of fully automated parking, the parking position candidate is automatically changed to parking position 221Y (based on the judgment of the parking position candidate determination unit 361) without requiring the user to perform an operation to change the parking position candidate to parking position 221Y.

[0218] <Summary of the Second Embodiment> According to the vehicle control system 11 of the second embodiment described above, the parking position candidate determination unit 361 determines parking position candidates based on the context parking map, and the presentation unit 341 can present the context parking map and parking position candidates. The parking position candidate determination unit 361 determines parking position candidates based on context information and user preference information, such as ease of parking and getting in and out of the vehicle when entering the parking lot, and "covered area". Therefore, the user can automatically park in their preferred parking position. In fully automatic parking processing, where the user's operation to select a parking position candidate is omitted, the vehicle can automatically park in the user's preferred parking position without any user operation.

[0219] <<4. Third Embodiment>> Next, a third embodiment of the present technology will be described with reference to Figures 23 to 27.

[0220] In the case of parking lots in large facilities or public facilities, such as shopping malls, the parking spaces on the road surface, arrows, white lines such as stop lines, and obstacles such as pillars and walls are generally standardized to a certain extent, and a general-purpose recognition model can be used to generate parking maps and context maps (contextual information).

[0221] On the other hand, in places like a user's home parking lot or company parking lot, there may be a unique environment around the parking spot, which could increase the probability of false detections or failures in a general-purpose recognition model. For example, posters or signs around the parking lot might be misdetected, or the presence of shadows and patterns during a specific time of day might cause false detections, or pillars or signs might constantly cause occlusion, resulting in failure to detect the spot. Also, faded parking spaces can lead to false detections. Furthermore, there may be places that the user does not want to recognize as parking spaces because they are someone else's private space, or places that are not recognized as parking spaces because surrounding structures such as walls are broken or malfunctioning.

[0222] The vehicle control system 11 according to the third embodiment is configured to handle the recognition of parking lots that are difficult to recognize with the general-purpose recognition model described above. In other words, the vehicle control system 11 according to the third embodiment is configured to generate and use a recognition model that is specialized for specific parking lots used by individuals (hereinafter referred to as individual parking lots), rather than a general-purpose model, as the recognition model for recognizing parking lots.

[0223] <Example of Vehicle Control System Configuration> Figure 23 is a block diagram showing an example of the configuration of a vehicle control system 11 according to the third embodiment.

[0224] In Figure 23, parts corresponding to the second embodiment shown in Figure 13 are denoted by the same reference numerals. Descriptions of these parts will be omitted as appropriate, and the explanation will focus on the different parts.

[0225] The automatic parking control unit 301 of the third embodiment, like the second embodiment, includes a map generation unit 311, a parking position candidate determination unit 361, and a user preference information storage unit 362. In Figure 23, the parking information storage unit 351 shown in Figure 13 has been changed to an individual recognition model storage unit 371.

[0226] In the third embodiment, the automatic parking control unit 301 has the functions of the second embodiment described above as a general-purpose model of the recognition model that recognizes parking spaces and context information. Furthermore, the automatic parking control unit 301 has the function of generating and utilizing a recognition model specialized for the recognition of individual parking spaces, and the configuration in Figure 23 corresponds to a block diagram relating to the generation and utilization of a recognition model specialized for the recognition of individual parking spaces. Hereinafter, the recognition model specialized for the recognition of individual parking spaces will be referred to as the individual recognition model.

[0227] In the HMI 31's presentation unit 341, the context parking map and parking location candidates supplied by the parking location candidate determination unit 361 are presented to the user. If there are any erroneous or undetected locations in the presented parking map, the user performs an operation to correct the erroneous or undetected locations in the parking map. For example, if the system fails to properly recognize the user's personal parking space, the user operates the touch panel or similar to input the correct parking space corresponding to their home parking space. The input unit 342 acquires the information of the parking space corrected by the user regarding the parking map as correction data and supplies it to the parking map generation unit 321.

[0228] Figure 24 illustrates an example of a correction made when the system fails to properly recognize a user's personal parking space at their home.

[0229] As shown in Figure 24, a parking lot 451 is provided adjacent to the user's home 450. However, due to a block wall 452 surrounding the parking lot 451 and the shadow 453 of the block wall 452, the parking lot 451 cannot be recognized and remains undetected. For example, the user sets a parking space 454 by operating a touch panel or the like. This information about the parking space 454 is supplied to the parking lot map generation unit 321 as correction data.

[0230] Returning to Figure 23, when the correction data is supplied from the HMI 31, the parking map generation unit 321 links the sensor data from the external recognition sensor 25 used for parking recognition (for example, image data from the camera 51) and the correction data of the parking map modified by the user to the parking location information and transmits them to the cloud server (for example, the cloud server 501 in Figure 25) via the communication unit 22. The cloud server then transmits the model parameters of the individual recognition model, which has been learned by machine learning using the transmitted correction data. In other words, the model parameters of the individual recognition model, which is specific to the user's parking lot, are learned and transmitted.

[0231] The communication unit 22 transmits the parking lot location information supplied by the parking lot map generation unit 321, the sensor data from the external environment recognition sensor 25 used for parking lot recognition, and the correction data for the parking lot map to the cloud server. The communication unit 22 also receives the model parameters of the learned individual recognition model from the cloud server and supplies them to the parking lot map generation unit 321.

[0232] The parking lot map generation unit 321 stores the model parameters of the individual recognition model transmitted from the cloud server in the individual recognition model storage unit 371, linking them to the parking lot location information. The individual recognition model storage unit 371 stores the individual recognition model linked to the parking lot location information.

[0233] After the model parameters of the individual recognition model are stored in the individual recognition model storage unit 371, linked to the parking lot location information, the parking lot map generation unit 321 generates a parking map using the individual recognition model, rather than a general-purpose recognition model, for the parking location indicated by that location information. Based on the parking lot location information, the parking lot map generation unit 321 collects sensor data and correction data each time it visits the same parking lot (for example, a home parking lot or a company parking lot) and sends it to the cloud server. As the training data increases and the individual recognition model (and its model parameters) are updated, the performance of the individual recognition model improves, and individual parking lots can be recognized with even greater accuracy.

[0234] Figure 25 shows an example configuration of a cloud server (server device) that learns an individual recognition model specific to individual parking lots using data transmitted from the parking lot map generation unit 321 in Figure 23.

[0235] The cloud server 501 in Figure 25 includes a communication unit 511, a learning data storage unit 512, an annotation unit 513, an annotated learning data storage unit 514, and a learner 515.

[0236] The communication unit 511 communicates with the communication unit 22 of the vehicle control system 11. By communicating with the communication unit 22, the communication unit 511 acquires sensor data linked to the parking lot location information and correction data for the parking lot map from the vehicle 1 and supplies it to the learning data storage unit 512. The communication unit 511 also transmits the parameters of the individual recognition model, which is specific to the user's parking lot and has been learned by the learner 515, to the communication unit 22 of the vehicle control system 11.

[0237] The learning data storage unit 512 stores learning data supplied from the communication unit 511, specifically sensor data linked to the user's individual parking location information and parking map correction data.

[0238] The annotation unit 513 performs annotation of the training data stored in the training data storage unit 512. Specifically, the annotation unit 513 recognizes and labels landmarks in the sensor data. Annotation can be performed automatically (without human intervention) using a large-scale language model (LLM). The training data labeled by the annotation unit 513 is stored in the annotated training data storage unit 514.

[0239] The learning device 515 uses the annotated learning data stored in the annotated learning data storage unit 514 to learn an individual recognition model (learning model) specific to each user's individual parking space. Through the learning of the individual recognition model, the parameters of the individual recognition model (model parameters) are obtained. The parameters of the individual recognition model obtained through learning are transmitted via the communication unit 511 to the parking map generation unit 321 of the automatic parking control unit 301 of the vehicle 1.

[0240] <Training Data Transmission Process> Next, referring to the flowchart in Figure 26, the training data transmission process for transmitting training data to generate an individual recognition model in the third embodiment will be described. This process starts at the same timing as the context parking map generation and presentation process in Figure 21.

[0241] First, in step S81, the parking lot map generation unit 321 determines whether or not an individual recognition model linked to the parking lot location information is stored in the individual recognition model storage unit 371.

[0242] If it is determined in step S81 that an individual recognition model is stored in the individual recognition model storage unit 371, the process proceeds to step S82, and the parking lot map generation unit 321 generates a parking lot map using the individual recognition model.

[0243] On the other hand, if it is determined in step S81 that no individual recognition model is stored in the individual recognition model storage unit 371, the process proceeds to step S83, and the parking lot map generation unit 321 generates a parking lot map using a general-purpose recognition model.

[0244] In step S84, the parking map generation unit 321 recognizes the parking location based on the sensor data of the external environment recognition sensor 25 using an individual recognition model or a general-purpose recognition model. The recognized parking location is supplied to the presentation unit 341 as a contextual parking map.

[0245] In step S85, the display unit 341 presents the recognized parking location to the user by displaying a contextual parking map. The user determines whether the parking location has been correctly recognized. If the location has not been correctly recognized due to a false detection or failure to detect, the user operates the touch panel or the like to input the correct parking space corresponding to the parking location (for example, the parking lot at home).

[0246] In step S86, the parking map generation unit 321 determines whether a parking position correction operation has been performed on the HMI 31, or in other words, whether parking position correction data has been supplied from the HMI 31.

[0247] If it is determined in step S86 that a parking position correction operation has been performed, the process proceeds to step S87, where the parking map generation unit 321 stores the parking location information in the individual recognition model storage unit 371, and transmits the parking location information, the sensor data from the external recognition sensor 25 used for parking recognition, and the correction data to the cloud server 501 via the communication unit 22. After step S87, the process returns to step S81, and the above process is repeated.

[0248] On the other hand, if it is determined in step S86 that no parking position correction operation has been performed, the process proceeds to step S88, where the parking map generation unit 321 determines whether or not the current location information is stored in the individual recognition model storage unit 371. If a parking position correction operation has been performed at least once in a designated parking lot, the current location information is stored in the individual recognition model storage unit 371 as parking lot location information by the process in step S87. If it is determined in step S88 that the current location information is not stored in the individual recognition model storage unit 371, the process returns to step S81, and the above process is repeated.

[0249] On the other hand, if it is determined in step S88 that the current location information is stored in the individual recognition model storage unit 371, the process proceeds to step S89, where the parking lot map generation unit 321 transmits the parking lot location information and the sensor data from the external recognition sensor 25 used for recognizing the parking lot to the cloud server 501 via the communication unit 22. After step S89, the process returns to step S81, and the above-described process is repeated.

[0250] According to the above training data transmission process, if the general-purpose recognition model fails to correctly recognize the parking position of a given parking lot and the user performs a correction operation to the parking position, the parking lot location information is stored in the individual recognition model storage unit 371, and the parking lot location information, sensor data, and correction data are sent to the cloud server 501 as training data. After the parking lot location information is stored in the individual recognition model storage unit 371, regardless of whether a correction operation has been performed, at least the parking lot location information and sensor data are sent to the cloud server 501, and training data for generating an individual recognition model for the individual parking lot is accumulated in the cloud server 501.

[0251] On the other hand, if the user does not perform any operation to correct the parking position, the general-purpose recognition model is able to sufficiently recognize the parking position in the individual parking lot, so the generation of an individual recognition model is unnecessary, and the training data is not sent to the cloud server 501.

[0252] <Contextual Parking Map Generation and Presentation Process> Figure 27 is a flowchart of the individual recognition model generation process executed on the cloud server 501, corresponding to the training data transmission process in Figure 26.

[0253] In the individual recognition model generation process, first, in step S101, the communication unit 511 determines whether or not training data has been transmitted from the user's vehicle 1, and waits until it is determined that training data has been transmitted. The training data transmitted from vehicle 1 consists of parking lot location information, sensor data, and correction data if the user has performed a parking position correction operation, and consists of parking lot location information and sensor data if the user has not performed a parking position correction operation.

[0254] If it is determined in step S101 that training data has been transmitted, the process proceeds to step S102, and the communication unit 511 stores the received training data in the training data storage unit 512.

[0255] Following step S102, in step S103, the annotation unit 513 determines whether a predetermined amount of training data has been stored in the training data storage unit 512. The predetermined amount of training data is any amount determined to be sufficient for training the recognition model. If it is determined in step S103 that the predetermined amount of training data has not yet been stored, the process returns to step S101, and the above process is repeated. That is, the storage of training data continues.

[0256] On the other hand, if it is determined in step S103 that a predetermined amount of training data has been accumulated, the process proceeds to step S104, where the annotation unit 513 annotates the training data stored in the training data storage unit 512. Specifically, the annotation unit 513 recognizes landmarks and other elements of the sensor data and labels them. Annotation can be performed using a large-scale language model (LLM). The training data labeled by the annotation unit 513 is stored in the annotated training data storage unit 514.

[0257] In step S105, the learner 515 uses the annotated learning data stored in the annotated learning data storage unit 514 to learn an individual recognition model (learning model) specialized for each user's individual parking space. Through the learning of the individual recognition model, the parameters of the individual recognition model (model parameters) are obtained.

[0258] In step S106, the learner 515 transmits the model parameters of the individual recognition model obtained through learning to the vehicle 1 via the communication unit 511.

[0259] In step S107, the parking map generation unit 321 of the automatic parking control unit 301 of vehicle 1 acquires model parameters of the individual recognition model via the communication unit 22, links the model parameters of the individual recognition model with the parking location information, and stores them in the individual recognition model storage unit 371.

[0260] After step S107, the process returns to step S101, and the above-described process is repeated. After the first execution of steps S101 to S107, from the second time onward, the individual recognition model is retrained using newly acquired training data, and the model parameters are updated.

[0261] Once an individual recognition model is generated and stored in the individual recognition model storage unit 371 of the user's vehicle 1, step S81 of the learning data transmission process in Figure 26 determines that the individual recognition model is stored in the individual recognition model storage unit 371. Therefore, the parking lot map generation unit 321 generates a parking lot map using the individual recognition model. This enables accurate recognition even of parking lots with difficult recognition conditions, such as the parking lot of a user's personal home.

[0262] <Summary of the Third Embodiment> According to the vehicle control system 11 of the third embodiment described above, when a general-purpose recognition model cannot correctly recognize the parking position in a parking lot, an individual recognition model specialized for a specific parking lot can be generated. This makes it possible to generate a recognition model specialized for an individual's use case, enabling accurate recognition even in parking lots with difficult recognition conditions. By improving recognition performance, safer automatic parking can be achieved. The individual recognition model may be a combination of a machine learning model and a rule-based parking lot recognition model. The individual recognition model is a recognition model specialized for a specific individual user and is not intended for use by other users. However, if other users were to use it, some kind of determination would be necessary to determine whether or not another person's individual recognition model can be applied to their own individual parking lot.

[0263] In the example described above, we explained how to accept input for correcting parking locations in a parking lot, i.e., correcting a parking lot map, and generate an individual recognition model specific to a given parking lot. However, the same method can be used to correct context information (context map). That is, if the context information cannot be correctly recognized, as shown in the example in Figure 12, input for correcting the context map can be accepted, and an individual recognition model that can accurately recognize the context map can be generated and used.

[0264] In the example described above, a specific individual recognition model tailored to a designated parking lot was generated on a cloud server, but it may also be generated within vehicle 1.

[0265] <<5. Computer Configuration Examples>> The series of processes performed by the automatic parking control unit 301 or the cloud server 501 in the first to third embodiments described above can be performed by hardware or by software. When the series of processes are performed by software, the programs that constitute the software are installed from a program recording medium to a computer built into dedicated hardware or a general-purpose personal computer.

[0266] Figure 28 is a block diagram showing an example of the hardware configuration of a computer that executes the series of processes described above by a program. Part of the vehicle control system 11, including the automatic parking control unit 301, and the cloud server 501 are configured, for example, by the computer shown in Figure 28.

[0267] The CPU (Central Processing Unit) 601, ROM (Read Only Memory) 602, and RAM (Random Access Memory) 603 are interconnected by a bus 604.

[0268] An input / output interface 605 is further connected to the bus 604. An input unit 606 consisting of a keyboard, mouse, etc., and an output unit 607 consisting of a display, speakers, etc. are connected to the input / output interface 605. In addition, a storage unit 608 consisting of a hard disk, non-volatile memory, etc., a communication unit 609 consisting of a network interface, etc., and a drive 610 that drives removable media 611 are connected to the input / output interface 605.

[0269] In a computer configured as described above, the CPU 601 loads, for example, a program stored in the memory unit 608 into the RAM 603 via the input / output interface 605 and the bus 604, and executes it, thereby performing the series of processes described above.

[0270] The program executed by the CPU 601 is recorded on removable media 611, for example, or provided via a wired or wireless transmission medium such as a local area network, the internet, or digital broadcasting, and installed in the storage unit 608.

[0271] The programs executed by the computer may be programs that are processed chronologically in the order described herein, or they may be programs that are processed in parallel or at necessary times, such as when they are called.

[0272] In this specification, a system refers to a collection of multiple components (devices, modules (parts), etc.), regardless of whether all components are located in the same enclosure. Therefore, multiple devices housed in separate enclosures and connected via a network, and a single device containing multiple modules within a single enclosure, are both considered systems.

[0273] Furthermore, the effects described herein are merely illustrative and not limiting, and other effects may also occur.

[0274] The embodiments of this technology are not limited to those described above, and various modifications are possible without departing from the spirit of this technology.

[0275] For example, this technology can be configured as cloud computing, where a single function is shared and processed collaboratively by multiple devices via a network.

[0276] Furthermore, each step described in the flowchart above can be performed by a single device, or it can be divided and performed by multiple devices. In addition, if a single step includes multiple processes, those processes can be performed by a single device, or they can be divided and performed by multiple devices.

[0277] Furthermore, this technology can adopt the following configurations: (1) An information processing device comprising: a parking map generation unit that generates a parking map of a parking lot based on sensing results from sensors mounted on a vehicle; and a context information generation unit that generates context information of the parking lot based on the sensing results and the parking map, wherein the parking map generation unit generates a context parking map by adding the context information to the parking map. (2) The information processing device according to (1) wherein the context information generation unit sets a plurality of adjacent parking spaces into one parking area and generates the context information on a parking area basis. (3) The information processing device according to (1) or (2) wherein the context information is information representing the vicinity of the parking lot entrance, the vicinity of the parking lot exit, the vicinity of the facility entrance / exit, for EVs, or for disabled persons. (4) The information processing device according to any one of (1) to (3) wherein the context information is information representing a hazard area. (5) The information processing device according to any one of (1) to (4) wherein the parking map generation unit updates the context information by adopting context information whose correct answer probability is greater than or equal to a predetermined value. (6) The information processing apparatus according to any one of (1) to (5), wherein the parking map generation unit updates the context parking map based on the difference between the stored context information and the context information generated during driving. (7) The information processing apparatus according to any one of (1) to (6), wherein the parking map generation unit updates the context parking map based on the accuracy of self-position estimation. (8) The information processing apparatus according to any one of (1) to (7), further comprising a presentation unit for presenting the context parking map and an acquisition unit for acquiring a parking position of the parking lot selected by the user based on the presented context parking map, wherein the vehicle is controlled to park at the parking position selected by the user. (9) The information processing apparatus according to any one of (1) to (8), further comprising a parking position candidate determination unit for determining parking position candidates based on the context parking map and a presentation unit for presenting the context parking map and the parking position candidates.(10) The parking location candidate determination unit determines the parking location candidate based on the context information and the user's preference information, as described in (9). (11) The parking location candidate determination unit updates the user's preference information based on the parking location selected by the user from among the parking location candidates, as described in (10). (12) The parking location candidate determination unit determines the parking location candidate based on user attributes, as described in any of (9) to (11). (13) The parking location candidate determination unit determines the priority of the parking location candidate based on the context information and determines the parking location candidate based on the priority, as described in any of (9) to (12). (14) The sensor includes a microphone, and the parking location candidate determination unit determines the priority of the parking location candidate based on the engine sound of other vehicles and determines the parking location candidate based on the priority, as described in any of (9) to (13). (15) The parking position candidate determination unit is an information processing device according to any one of (9) to (14) above, which changes the determined parking position candidate based on the sensing results during driving. (16) The parking map generation unit is an information processing device according to any one of (1) to (15) above, which modifies the parking map or the context information based on user instructions. (17) The information processing device according to any one of (1) to (16) above, which further comprises a storage unit for storing a recognition model for recognizing the parking lot, and the parking map generation unit generates a parking map of the parking lot using the recognition model. (18) The information processing device according to (17) above, which further comprises a communication unit for transmitting the parking lot location information, sensor data from the sensor and modified data of the parking lot recognition result to a server device, and for receiving the model parameters of the recognition model from the server device, and for storing the model parameters of the recognition model.(19) A vehicle comprising: a parking map generation unit that generates a parking map of a parking lot based on sensing results from sensors mounted on the vehicle; a context information generation unit that generates context information of the parking lot based on the sensing results and the parking map; and an operation control unit that controls the operation of the vehicle, wherein the parking map generation unit generates a context parking map by adding the context information to the parking map, and the operation control unit controls the operation of the vehicle to park at a parking position determined based on the context parking map.

[0278] 1 Vehicle, 11 Vehicle control system, 21 Vehicle control ECU, 22 Communication unit, 23 Map information storage unit, 24 Location information acquisition unit, 25 Outside world recognition sensor, 26 In-vehicle sensor, 27 Vehicle sensor, 28 Memory unit, 29 Driving automation control unit, 32 Vehicle control unit, 41 Communication network, 51 Camera, 52 Radar, 54 Ultrasonic sensor, 55 Microphone, 61 Analysis unit, 62 Action planning unit, 63 Motion control unit, 71 Self-position estimation unit, 72 Sensor fusion unit, 73 Recognition unit, 201 Parking lot, 202 Sidewalk, 203 Road, 211 Wall, 212 Entrance gate, 213 Exit gate, 214 Passageway, 215 Pillar, 216 Facility entrance / exit, 221 Parking space, 221BB Disabled parking space, 221EV EV parking space, 231 Parking map, 232 Context map, 233 Screen, 301 Automatic parking control unit, 311 Map generation unit, 321 Parking map generation unit, 322 Context information generation unit, 341 Presentation unit, 342 Input unit, 351 Parking information storage unit, 361 Parking position candidate determination unit, 362 User preference information storage unit, 371 Individual recognition model storage unit, 421 Preference information registration screen, 501 Cloud server, 511 Communication unit, 512 Learning data storage unit, 513 Annotation unit, 514 Learning data storage unit, 515 Learner, 601 CPU, 602 ROM, 603 RAM, 604 Bus, 605 Input / Output Interface, 606 Input Section, 607 Output Section, 608 Storage Section, 609 Communication Section, 610 Drive, 611 Removable Media

Claims

1. An information processing device comprising: a parking map generation unit that generates a parking map of a parking lot based on sensing results from sensors mounted on a vehicle; and a context information generation unit that generates context information of the parking lot based on the sensing results and the parking map, wherein the parking map generation unit generates a context parking map by adding the context information to the parking map.

2. The information processing apparatus according to claim 1, wherein the context information generation unit sets a plurality of adjacent parking spaces into a single parking area and generates the context information on a parking area basis.

3. The information processing device according to claim 1, wherein the context information is information representing the vicinity of the parking lot entrance, the vicinity of the parking lot exit, the vicinity of the facility entrance / exit, for EVs, or for persons with disabilities.

4. The information processing apparatus according to claim 1, wherein the context information is information representing a hazard area.

5. The information processing device according to claim 1, wherein the parking lot map generation unit updates the context information by adopting context information whose probability of being correct is equal to or greater than a predetermined value.

6. The information processing device according to claim 1, wherein the parking lot map generation unit updates the context parking lot map based on the difference between the stored context information and the context information generated during driving.

7. The information processing apparatus according to claim 1, wherein the parking lot map generation unit updates the context parking lot map based on the accuracy of self-position estimation.

8. The information processing apparatus according to claim 1, further comprising: a presentation unit that presents the contextual parking map; and an acquisition unit that acquires a parking position in the parking lot selected by the user based on the presented contextual parking map, wherein the vehicle is controlled to park in the parking position selected by the user.

9. The information processing apparatus according to claim 1, further comprising: a parking location candidate determination unit that determines parking location candidates based on the context parking map; and a presentation unit that presents the context parking map and the parking location candidates.

10. The information processing apparatus according to claim 9, wherein the parking location candidate determination unit determines the parking location candidate based on the context information and the user preference information.

11. The information processing device according to claim 10, wherein the parking position candidate determination unit updates the user's preference information based on the parking position selected by the user from among the parking position candidates.

12. The information processing apparatus according to claim 9, wherein the parking position candidate determination unit determines the parking position candidate based on user attributes.

13. The information processing apparatus according to claim 9, wherein the parking position candidate determination unit determines the priority of the parking position candidates based on the context information, and determines the parking position candidates based on the priority.

14. The information processing apparatus according to claim 9, wherein the sensor includes a microphone, the parking position candidate determination unit determines the priority of the parking position candidates based on the engine sounds of other vehicles, and determines the parking position candidates based on the priority.

15. The information processing device according to claim 9, wherein the parking position candidate determination unit changes the determined parking position candidate based on the sensing results during driving.

16. The information processing apparatus according to claim 1, wherein the parking map generation unit modifies the parking map or the context information based on user instructions.

17. The information processing apparatus according to claim 1, further comprising a storage unit for storing a recognition model for recognizing the parking lot, wherein the parking lot map generation unit generates a parking lot map of the parking lot using the recognition model.

18. The information processing apparatus according to claim 17, further comprising a communication unit that transmits the location information of the parking lot, the sensor data of the sensor and the correction data of the recognition result of the parking lot to a server device, and receives the model parameters of the recognition model from the server device, wherein the storage unit stores the model parameters of the recognition model.

19. A vehicle comprising: a parking map generation unit that generates a parking map of a parking lot based on sensing results from sensors mounted on the vehicle; a context information generation unit that generates context information of the parking lot based on the sensing results and the parking map; and an operation control unit that controls the operation of the vehicle, wherein the parking map generation unit generates a context parking map by adding the context information to the parking map, and the operation control unit controls the operation of the vehicle to park at a parking position determined based on the context parking map.

Images