Information processing device, parking assistance device, and method

Abstract

To provide an information processing device capable of determining whether or not a vehicle is allowed to travel a registered travel route before the vehicle starts travel based on the travel route.SOLUTION: An information processing device includes a communication unit, a detection unit, and a determination unit. The communication unit receives vehicle identification information capable of identifying a vehicle from an on-vehicle device. The detection unit detects an obstacle. The determination unit determines whether or not the vehicle can travel along a travel route according to a position of the detected obstacle and the travel route. Further, the communication unit transmits determination results of travel permission by the determination unit to the on-vehicle device.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, a parking assistance device, and a method. [Background technology]

[0002] Conventionally, parking assistance technologies that move a vehicle by automatic driving when parking the vehicle are known. One such parking assistance technology is a technology that records a driving route based on a teacher driving by the driver and performs parking assistance using the recorded driving route. This technology is used when parking in a fixed parking position repeatedly, such as a parking lot at a user's home or workplace. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2007-161119 A Summary of the Invention [Problem to be solved by the invention]

[0004] When an obstacle is present on the driving route when the vehicle moves to a parking position by such parking assistance, it is difficult to determine that the driving route is impossible until the obstacle is recognized by the on-board sensor. Therefore, after the vehicle actually starts driving, the driver is notified that the registered driving route is impossible, which may cause the driver to have to redo the parking operation that has been partially completed. In such a case, the driver may feel inconvenienced.

[0005] The present disclosure provides an information processing device that can determine whether or not a vehicle can travel along a registered travel route before the vehicle starts traveling based on the travel route. [Means for solving the problem]

[0006] The information processing device according to the present disclosure includes a communication unit and A storage unit; The vehicle is equipped with a detection unit and a determination unit. The communication unit performs automatic driving in a parking lot along a driving route recorded based on a teacher driving by a driver. Multiple car Both The vehicle identification information that can identify the vehicle is then received. The storage unit stores the vehicle identification information in association with a travel area, which is an area through which the body of the vehicle passes when the vehicle travels along the travel route. The detection unit detects an obstacle. A travel area of ​​a vehicle identified among the plurality of vehicles based on the received vehicle identification information; Detected obstacle locations And In response to the determination, the communication unit determines whether the vehicle can travel along the travel route. Identified vehicle among multiple vehicles Send to. Effect of the Invention

[0007] According to the information processing device according to the present disclosure, before a vehicle starts traveling based on a registered travel route, it is possible to determine whether or not the vehicle can travel on the registered travel route. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of a vehicle equipped with a parking assistance device according to a first embodiment. [Diagram 2] FIG. 2 is a diagram showing an example of a configuration in the vicinity of a driver's seat of a vehicle according to the first embodiment. [Diagram 3] FIG. 3 is a diagram showing an example of a parking lot where a vehicle according to the first embodiment is parked. [Figure 4] FIG. 4 is a diagram illustrating an example of the parking assistance system according to the first embodiment. [Diagram 5] FIG. 5 is a diagram illustrating an example of the traveling area information according to the first embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of a hardware configuration of the parking assistance device according to the first embodiment. [Figure 7] FIG. 7 is a sequence diagram showing an example of the flow of a driving route recording process executed by the parking assistance device and the server device according to the first embodiment. [Figure 8]FIG. 8 is a sequence diagram showing an example of the flow of the parking assistance process executed by the parking assistance device and the server device according to the first embodiment. [Figure 9] FIG. 9 is a diagram showing an example of a notification screen indicating that traveling is not permitted according to the first embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of the traveling area information according to the second embodiment. [Figure 11] FIG. 11 is a diagram showing an example of steering / braking information according to the second embodiment. [Figure 12] FIG. 12 is a diagram showing an example of association between a surrounding image, a travel route ID, and a priority order according to the second embodiment. [Figure 13] FIG. 13 is a diagram showing an example of a notification screen of a possible driving route according to the second embodiment. [Figure 14] FIG. 14 is a diagram showing an example of a relationship between a fixed camera and a server device according to the first modification. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, embodiments of an information processing device, a parking assistance device, and a method according to the present disclosure will be described with reference to the drawings.

[0010] (First embodiment) Fig. 1 is a diagram showing an example of a vehicle 1 equipped with a parking assistance device 100 according to the first embodiment. As shown in Fig. 1, the vehicle 1 includes a vehicle body 12 and two pairs of wheels 13 arranged along a predetermined direction on the vehicle body 12. The two pairs of wheels 13 include a pair of front tires 13f and a pair of rear tires 13r.

[0011] The front tire 13f shown in Fig. 1 is an example of a first wheel in this embodiment. The rear tire 13r is an example of a second wheel in this embodiment. Although the vehicle 1 shown in Fig. 1 has four wheels 13, the number of wheels 13 is not limited to this. For example, the vehicle 1 may be a two-wheeled vehicle.

[0012] The vehicle body 12 is coupled to wheels 13 and can move by the wheels 13. In this case, the predetermined direction in which the two pairs of wheels 13 are arranged is the traveling direction of the vehicle 1. The vehicle 1 can move forward or backward by switching gears (not shown) or the like. The vehicle 1 can also turn right or left by steering.

[0013] The vehicle body 12 has a front end F, which is an end on the front tire 13f side, and a rear end R, which is an end on the rear tire 13r side. The vehicle body 12 has a substantially rectangular shape when viewed from above, and the four corners of the substantially rectangular shape may be called ends. The vehicle 1 also includes a display device, a speaker, and an operation unit, which are not shown in FIG.

[0014] A pair of bumpers 14 are provided at the front and rear ends F, R of the vehicle body 12 near the lower end of the vehicle body 12. Of the pair of bumpers 14, the front bumper 14f covers the entire front surface and part of the side surface near the lower end of the vehicle body 12. Of the pair of bumpers 14, the rear bumper 14r covers the entire rear surface and part of the side surface near the lower end of the vehicle body 12.

[0015] At predetermined ends of the vehicle body 12, wave transmitting / receiving units 15f, 15r for transmitting and receiving sound waves such as ultrasonic waves are disposed. For example, one or more wave transmitting / receiving units 15f are disposed at the front bumper 14f, and one or more wave transmitting / receiving units 15r are disposed at the rear bumper 14r. Hereinafter, when the wave transmitting / receiving units 15f, 15r are not particularly limited, they are simply referred to as the wave transmitting / receiving unit 15. In addition, the number and positions of the wave transmitting / receiving units 15 are not limited to the example shown in FIG. 1. For example, the vehicle 1 may be provided with wave transmitting / receiving units 15 on the left and right sides.

[0016] In this embodiment, a sonar using ultrasonic waves will be described as an example of the wave transmitting and receiving unit 15, but the wave transmitting and receiving unit 15 may be a radar that transmits and receives electromagnetic waves. Alternatively, the vehicle 1 may be equipped with both a sonar and a radar. Also, the wave transmitting and receiving unit 15 may simply be called a sensor.

[0017] The wave transmitting / receiving unit 15 detects obstacles around the vehicle 1 based on the results of transmitting and receiving sound waves or electromagnetic waves. In addition, the wave transmitting / receiving unit 15 measures the distance between the vehicle 1 and obstacles around the vehicle 1 based on the results of transmitting and receiving sound waves or electromagnetic waves.

[0018] Vehicle 1 also includes a first vehicle-mounted camera 16a that captures an image in front of vehicle 1, a second vehicle-mounted camera 16b that captures an image behind vehicle 1, a third vehicle-mounted camera 16c that captures an image on the left side of vehicle 1, and a fourth vehicle-mounted camera that captures an image on the right side of vehicle 1. The fourth vehicle-mounted camera is not shown in the figure.

[0019] Hereinafter, when the first vehicle-mounted camera 16a, the second vehicle-mounted camera 16b, the third vehicle-mounted camera 16c, and the fourth vehicle-mounted camera are not particularly distinguished from each other, they will be simply referred to as vehicle-mounted cameras 16. The positions and the number of the vehicle-mounted cameras are not limited to the example shown in FIG. 1. For example, the vehicle 1 may also be equipped with only two vehicles, the first vehicle-mounted camera 16a and the second vehicle-mounted camera 16b. Alternatively, the vehicle 1 may have other vehicle-mounted cameras in addition to the above-mentioned examples.

[0020] The vehicle-mounted camera 16 is capable of capturing images of the surroundings of the vehicle 1, and is, for example, a camera that captures color images. The images captured by the vehicle-mounted camera 16 may be videos or still images. The vehicle-mounted camera 16 may be a camera built into the vehicle 1, or may be a camera of a drive recorder that is retrofitted to the vehicle 1.

[0021] The vehicle 1 is also equipped with a parking assistance device 100. The parking assistance device 100 is an information processing device that can be installed in the vehicle 1, and is, for example, an ECU (Electronic Control Unit) or an OBU (On Board Unit) provided inside the vehicle 1. Alternatively, the parking assistance device 100 may be an external device installed near the dashboard of the vehicle 1. The parking assistance device 100 may also function as a car navigation device or the like. The parking assistance device 100 is an example of an in-vehicle device in this embodiment.

[0022] Next, a configuration of the vicinity of the driver's seat of the vehicle 1 according to this embodiment will be described. Fig. 2 is a diagram showing an example of a configuration of the vicinity of the driver's seat 130a of the vehicle 1 according to the first embodiment.

[0023] 2, the vehicle 1 includes a driver's seat 130a and a passenger seat 130b. In front of the driver's seat 130a, a windshield 180, a dashboard 190, a steering wheel 140, a display device 120, and operation buttons 141 are provided.

[0024] The display device 120 is a display provided on a dashboard 190 of the vehicle 1. As an example, the display device 120 is located in the center of the dashboard 190 as shown in FIG. 2. The display device 120 is, for example, a liquid crystal display or an organic EL (Electro Luminescence) display. The display device 120 may also function as a touch panel. The display device 120 is an example of a display unit in this embodiment.

[0025] The steering wheel 140 is provided in front of the driver's seat 130a and can be operated by the driver. The rotation angle of the steering wheel 140, i.e., the steering angle, is electrically or mechanically linked to the change in the direction of the front tires 13f, which are the steered wheels. The steered wheels may be the rear tires 13r, or both the front tires 13f and the rear tires 13r may be steered wheels.

[0026] The operation button 141 is a button that can accept an operation by a user. In this embodiment, the user is, for example, the driver of the vehicle 1. The operation button 141 accepts an operation from the driver to start parking assistance, for example, by accepting a press from the driver. The position of the operation button 141 is not limited to the example shown in FIG. 2, and may be provided on the steering wheel 140, for example. The operation button 141 is an example of an operation unit in this embodiment. In addition, when the display device 120 also serves as a touch panel, the display device 120 may be an example of an operation unit. In addition, an operation terminal that can transmit a signal to the vehicle 1 from outside the vehicle 1, such as a tablet terminal, a smartphone, a remote controller, or an electronic key (not shown), may be an example of an operation unit.

[0027] The parking assistance device 100 of this embodiment records a driving route based on the teacher driving by the driver, and performs parking assistance using the recorded driving route. In other words, the parking assistance method executed by the parking assistance device 100 is a method of automatically driving the vehicle 1 based on the teacher driving by the driver. This parking assistance method is effective for reducing the driver's parking effort when repeatedly parking in a fixed location, such as a garage at the driver's home, a contracted parking space at an apartment complex, or a specified parking space in a parking lot at the driver's workplace. This type of parking assistance is called home zone parking.

[0028] In this embodiment, it is assumed that there is a parking space for the vehicle 1 in a parking lot of an apartment building, a parking lot of a workplace, etc. The parking assistance device 100 of this embodiment is capable of communicating with a server device that manages such parking lots.

[0029] Fig. 3 is a diagram showing an example of a parking lot 9 in which a vehicle 1 according to the first embodiment is parked. As shown in Fig. 3, a plurality of parking spaces 901 to 904 are provided in the parking lot 9. In the example shown in Fig. 3, a parking space 903 is assigned to the vehicle 1.

[0030] 3 is a route along which the vehicle 1 is parked in the parking space 903 by the driver's manual driving. A driving area 800 shown in FIG. 3 is an area through which the body 12 of the vehicle 1 passes when the vehicle 1 travels along the driving route 8.

[0031] A fixed camera 5 is provided in the parking lot 9. The fixed camera 5 is installed in a position where it can capture at least a plurality of parking spaces 901-904 and a driving area 800 for a plurality of vehicles assigned to each of the parking spaces 901-904. The images captured by the fixed camera 5 may be videos or still images. The fixed camera 5 may also serve as a surveillance camera or the like.

[0032] The server device 2 is provided, for example, in a management room of the parking lot 9. Alternatively, the server device 2 may be provided outside the parking lot 9. The server device 2 is an example of an information processing device in this embodiment.

[0033] The parking assistance device 100 of the vehicle 1 communicates with the server device 2 via wireless communication. The wireless communication standard is Wi-Fi, Bluetooth (registered trademark), or the like, but is not particularly limited.

[0034] Furthermore, the server device 2 acquires from the fixed camera 5 images captured by the fixed camera 5. The server device 2 and the fixed camera 5 may communicate with each other by wireless communication or via a communication cable.

[0035] Furthermore, the parking assistance device 100 and the fixed camera 5 do not need to be directly connected to the server device 2, and may communicate with the server device 2 via a communication device provided in the parking lot 9, for example.

[0036] The server device 2 stores the image acquired from the fixed camera 5 before the supervised driving of the vehicle 1. Then, when the vehicle 1 subsequently automatically parks in the parking space 903 by automatic driving based on the supervised driving, the server device 2 stores the image acquired from the fixed camera 5 before the supervised driving of the vehicle 1.

[0037] The image captured by the fixed camera 5 before the vehicle 1 starts the teacher driving is an example of the first background image in this embodiment. Also, the image captured by the fixed camera 5 before the vehicle 1 starts the automatic driving is an example of the second background image in this embodiment.

[0038] The server device 2 detects the obstacles 70a, 70b based on the first background image and the second background image. The method of detecting the obstacles 70a, 70b will be described in detail later. When the detected obstacles 70a, 70b are located within the travel area 800, the server device 2 determines that the vehicle 1 cannot travel along the travel route 8. The server device 2 transmits the determination result of whether or not the vehicle 1 can travel to the vehicle 1.

[0039] Next, the functions of the parking assistance device 100 and the server device 2 of this embodiment will be described in detail.

[0040] 4 is a diagram showing an example of a parking assistance system S100 according to the first embodiment. The parking assistance system S100 of this embodiment includes an in-vehicle system S1 mounted on a vehicle 1 and an infrastructure system S2 provided in a parking lot.

[0041] The in-vehicle system S1 includes a display device 120, an in-vehicle camera 16, a parking assistance device 100, and a vehicle control system 18. The in-vehicle system S1 may further include other devices.

[0042] The vehicle control system 18 controls the driving of the vehicle 1 by controlling the steering, accelerator, brakes, and transmission gears of the vehicle 1.

[0043] The parking assistance device 100 includes a communication device 101 , a display control unit 102 , a driving information recording unit 103 , an image processing unit 104 , a correction unit 105 , and a storage unit 110 .

[0044] 4, the parking assistance device 100 is illustrated as a single device including the communication device 101, the display control unit 102, the driving information recording unit 103, the image processing unit 104, the correction unit 105, and the storage unit 110, but the configuration of the parking assistance device 100 is not limited to this. For example, the parking assistance device 100 may be configured by devices such as a plurality of ECUs. Furthermore, the vehicle control system 18 and the parking assistance device 100 may be integrated into a single ECU.

[0045] The storage unit 110 is configured by, for example, a Read Only Memory (ROM), a Random Access Memory (RAM), a flash memory, a Hard Disk Drive (HDD), etc. Note that a plurality of storage media may function as the storage unit 110.

[0046] The storage unit 110 stores programs and data used in various processes executed by the parking assistance device 100. For example, the programs executed by the parking assistance device 100 of this embodiment are modularized to include the above-mentioned functional units (display control unit 102, driving information recording unit 103, image processing unit 104, and correction unit 105), and as actual hardware, the CPU reads out and executes the programs from the storage unit 110, loading the above-mentioned units onto the RAM, and the display control unit 102, driving information recording unit 103, image processing unit 104, and correction unit 105 are generated on the RAM. The processes realized by the functional units of the parking assistance device 100 are also referred to as steps.

[0047] The program executed by the parking assistance device 100 of this embodiment is provided by being recorded in a computer-readable recording medium such as a flash memory as a file in an installable or executable format.

[0048] The program executed by the parking assistance device 100 of this embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. The program executed by the parking assistance device 100 of this embodiment may be provided or distributed via a network such as the Internet. The program executed by the parking assistance device 100 of this embodiment may be provided by being pre-installed in a ROM or the like.

[0049] The display control unit 102, the driving information recording unit 103, the image processing unit 104, and the correction unit 105 may be individual hardware circuits.

[0050] The communication device 101 is a communication device capable of wireless communication with an information processing device outside the vehicle 1. Specifically, the communication device 101 is capable of wireless communication with the server device 2 of the infrastructure system S2. As described in FIG. 3, the communication device 101 does not need to be directly connected to the server device 2, and may communicate with the server device 2 via another communication device provided in the parking lot 9, for example. The communication device 101 is an example of a communication unit of the parking assistance device 100 in this embodiment.

[0051] In this embodiment, when the vehicle 1 enters the communication range of the server device 2, the communication device 101 transmits vehicle information of the vehicle 1 to the server device 2. In addition, the communication device 101 receives information indicating whether or not the vehicle 1 can travel along the travel route 8 from the server device 2.

[0052] The vehicle information is information related to the vehicle 1, and includes at least a vehicle ID capable of identifying the vehicle 1. The vehicle ID is an example of vehicle identification information in this embodiment.

[0053] Furthermore, when a user's operation to start teacher driving is input from the driver's UI or another operation unit, the communication device 101 transmits a teacher driving start signal indicating the start of teacher driving to the server device 2. Furthermore, when a user's operation to end teacher driving is input from the driver's UI or another operation unit, the communication device 101 transmits a teacher driving end signal indicating the end of teacher driving to the server device 2. Furthermore, when a user's operation to start parking assistance by automatic driving is input from the driver's UI or another operation unit, the communication device 101 transmits a parking assistance start signal indicating the start of parking assistance to the server device 2.

[0054] The display control unit 102 controls the display device 120 to display various types of driver's UI (User Interface). For example, when the travel possibility information received by the communication device 101 from the server device 2 indicates that travel along the travel route 8 is not possible, the display control unit 102 causes the display device 120 to output a notification.

[0055] For example, the display control unit 102 causes the display device 120 in the vehicle 1 to display, as a notification, a message indicating that automatic traveling cannot be started due to the presence of an obstacle 70a. The display control unit 102 is an example of an output unit in this embodiment.

[0056] The notification output method is not limited to a screen display, and may be a voice notification output, etc. For example, the parking assistance device 100 may include a voice output unit that outputs a voice message to a speaker.

[0057] The driving information recording unit 103 records driving information of the vehicle 1 during the teacher driving. The driving information recording unit 103 acquires steering information related to the steering angle of the vehicle 1 and braking information related to the braking operation from various sensors of the vehicle 1 or from other ECUs, and registers them in the storage unit 110. In Fig. 3, the steering information and braking information are collectively referred to as steering / braking information 111.

[0058] The image processing unit 104 acquires a surrounding image 112 captured by the vehicle-mounted camera 16 during teacher driving, and registers the acquired image in the storage unit 110.

[0059] The steering / braking information 111 and the surrounding image 112 are collectively referred to as registration data for automatic parking.

[0060] Furthermore, when performing parking assistance by automatic driving, the image processing unit 104 compares the surrounding image acquired from the in-vehicle camera 16 with the surrounding image 112 during teacher driving registered in the storage unit 110, and estimates the position of the vehicle 1 based on the comparison result. For example, the image processing unit 104 extracts feature points from the surrounding image acquired when performing parking assistance and the surrounding image 112 during teacher driving, and estimates the current relative position of the vehicle 1 based on the position of the vehicle 1 during teacher driving based on the difference in the positions of the feature points.

[0061] The feature points may be extracted using a method such as FAST (Features from Accelerated Segment Test) or ORB (Oriented FAST and Rotated BRIEF). The image processing unit 104 may also perform self-location estimation using other known methods. The image processing unit 104 may define the environment around the vehicle 1 as a map based on feature points extracted from a surrounding image 112 captured during teacher driving, and store the map in the storage unit 110.

[0062] The correction unit 105 corrects the steering / braking information 111 registered in the storage unit 110 based on the relative position of the vehicle 1 estimated by the image processing unit 104 when parking assistance is performed by automatic driving.

[0063] Furthermore, when an obstacle around the vehicle 1 is detected by the wave transmitting / receiving unit 15, the correction unit 105 may correct the steering / braking information 111 according to the position of the obstacle. Furthermore, the detection of an obstacle around the vehicle 1 is not limited to being performed by the wave transmitting / receiving unit 15, and the obstacle around the vehicle 1 may be detected based on an image captured by the vehicle-mounted camera 16. For example, the vehicle-mounted camera 16 may detect an obstacle from the captured image by image recognition or the like. Alternatively, the correction unit 105 may detect an obstacle from the image captured by the vehicle-mounted camera 16 by image recognition or the like. The detection of an obstacle by the wave transmitting / receiving unit 15 and the detection of an obstacle based on an image by the vehicle-mounted camera 16 or the correction unit 105 may be used in combination, or only one of them may be used.

[0064] Furthermore, the correction unit 105 may correct the steering / braking information 111 based on various peripheral information that indicates the situation around the vehicle 1, rather than based on the result of obstacle detection.

[0065] The correction unit 105 transmits the corrected steering / braking information 111 to the vehicle control system 18. The vehicle control system 18 causes the vehicle 1 to automatically travel based on the steering / braking information 111 corrected by the correction unit 105.

[0066] The infrastructure system S2 includes a fixed camera 5 and a server device 2.

[0067] The server device 2 includes a communication unit 201 , an ID management unit 202 , a vehicle detection unit 203 , an area recognition unit 204 , an obstacle detection unit 205 , a travel feasibility determination unit 206 , a memory management unit 207 , and a storage unit 210 .

[0068] 4 illustrates the server device 2 as a single device including the communication device 201, the ID management unit 202, the vehicle detection unit 203, the area recognition unit 204, the obstacle detection unit 205, the travel possibility determination unit 206, the memory management unit 207, and the storage unit 210, but the configuration of the server device 2 is not limited to this. For example, some or all of the functions of the server device 2 may be provided in a cloud environment. Furthermore, these functions may be realized by a plurality of server devices 2.

[0069] The storage unit 210 is configured by, for example, a ROM, a RAM, a flash memory, a HDD, etc. Note that a plurality of storage media may function as the storage unit 210.

[0070] The storage unit 210 stores programs and data used in various processes executed by the server device 2. For example, the programs executed by the server device 2 in this embodiment are modularized to include the above-mentioned functional units (ID management unit 202, vehicle detection unit 203, area recognition unit 204, obstacle detection unit 205, travel feasibility determination unit 206, and memory management unit 207), and as actual hardware, the CPU reads out and executes the programs from the storage unit 210, loading the above-mentioned units onto the RAM, and the ID management unit 202, vehicle detection unit 203, area recognition unit 204, obstacle detection unit 205, travel feasibility determination unit 206, and memory management unit 207 are generated on the RAM. The processes realized by the functional units of the server device 2 are also referred to as steps.

[0071] The programs executed by the server device 2 of this embodiment are provided by being recorded in a computer-readable recording medium such as a flash memory in the form of an installable or executable file.

[0072] The program executed by the server device 2 of this embodiment may be stored on a computer connected to a network such as the Internet and provided by downloading it via the network. The program executed by the server device 2 of this embodiment may be provided or distributed via a network such as the Internet. The program executed by the server device 2 of this embodiment may be provided by being pre-installed in a ROM or the like.

[0073] The ID management unit 202, the vehicle detection unit 203, the area recognition unit 204, the obstacle detection unit 205, the travel feasibility determination unit 206, and the memory management unit 207 may be individual hardware circuits.

[0074] The communication device 201 is a communication device capable of wireless communication with the parking assistance device 100 and the fixed camera 5. The communication device 201 is an example of a communication unit of the server device 2 in this embodiment.

[0075] In this embodiment, the communication device 201 receives vehicle information from the parking assistance device 100. For example, when the vehicle 1 enters the communication range of the communication device 201 and the communication device 101 of the parking assistance device 100 and the communication device 201 of the server device 2 communicate with each other, the communication device 201 receives the vehicle information transmitted from the communication device 101.

[0076] In addition, the communication device 201 acquires a teacher driving start signal, a teacher driving end signal, and a parking support start signal from the parking support device 100.

[0077] In addition, the communication device 201 transmits to the parking assistance device 100 travel possibility information indicating the determination result as to whether the vehicle 1 is able to travel along the travel route 8 or not.

[0078] Furthermore, the communication device 201 acquires from the fixed camera 5 an image captured by the fixed camera 5. The communication device 201 may constantly acquire images captured by the fixed camera 5. Note that since the images captured by the fixed camera 5 are used in the processes executed by the vehicle detection unit 203, the area recognition unit 204, the obstacle detection unit 205, and the memory management unit 207, the fixed camera 5 is connected to the vehicle detection unit 203, the area recognition unit 204, the obstacle detection unit 205, and the memory management unit 207 by arrows in FIG. 4, but the actual transmission path of the images may go through the communication device 201.

[0079] The ID management unit 202 extracts the vehicle ID of the vehicle 1 included in the vehicle information received by the communication device 201. The ID management unit 202 sends the vehicle ID of the vehicle 1 to the memory management unit 207. The ID management unit 202 may also determine whether the vehicle ID of the vehicle 1 included in the vehicle information is included in the vehicle ID information 213 pre-registered in the storage unit 210. The vehicle ID information 213 is, for example, a table in which the vehicle ID of the vehicle 1 that has contracted for the parking lot 9 is associated with a contracted parking space ID that can identify the contracted parking space of the vehicle 1.

[0080] The vehicle detection unit 203 detects the vehicle 1 from a plurality of images taken in time series by the fixed camera 5 while the vehicle 1 is on the teacher's run. The method for detecting the vehicle 1 is not particularly limited, and for example, a known image processing technique such as object recognition can be adopted. The vehicle detection unit 203 may also detect the vehicle 1 by extracting the difference between a first background image taken before the vehicle 1 starts the teacher's run and an image taken while the vehicle 1 is on the teacher's run. The vehicle detection unit 203 sends the plurality of images and the position where the vehicle 1 is depicted on the images to the area recognition unit 204.

[0081] The area recognition unit 204 recognizes the driving area 800 of the vehicle 1 in the parking lot 9 based on the change in position of the vehicle 1 during teacher driving detected by the vehicle detection unit 203. For example, the area recognition unit 204 specifies the range of the driving area 800 that occupies an image by the coordinates on the image. The area recognition unit 204 sends the recognized driving area 800 to the memory management unit 207. The memory management unit 207 associates the driving area 800 recognized by the area recognition unit 204 with the vehicle ID specified by the ID management unit 202, and stores it in the storage unit 210 as driving area information 212. Note that this process may be executed by the area recognition unit 204.

[0082] Furthermore, the area recognition unit 204 may recognize the travel path at the center of the recognized travel area 800 in the width direction as the travel path 8.

[0083] Fig. 5 is a diagram showing an example of the traveling area information 212 according to the first embodiment. As shown in Fig. 5, the traveling area information 212 is, for example, a table in which the vehicle ID of each vehicle is associated with the traveling area.

[0084] Returning to Fig. 4, the obstacle detection unit 205 detects obstacles 70a, 70b present in the parking lot 9 based on the image captured by the fixed camera 5. More specifically, the obstacle detection unit 205 detects the obstacles 70a, 70b based on a second background image of the parking lot 9 captured at the time when the vehicle 1 starts autonomous driving, and a first background image of the parking lot 9 captured before the vehicle 1 starts supervised driving. For example, the obstacle detection unit 205 detects the obstacles 70a, 70b by extracting the difference between the first background image and the second background image.

[0085] In the example described in Fig. 3, potted plant 71 was already placed in parking lot 9 when vehicle 1 was traveling automatically. In this case, potted plant 71 is depicted at the same position in both the first background image and the second background image, so no difference occurs. In this case, obstacle detection unit 205 does not detect potted plant 71 as an obstacle.

[0086] Moreover, the obstacles 70a and 70b were not in the parking lot 9 when the vehicle 1 performed automatic driving, but were present in the parking lot 9 when the vehicle 1 started automatic driving. Therefore, the obstacles 70a and 70b are not depicted in the first background image, but are depicted only in the second background image. In this case, a difference occurs between the first background image and the second background image at the location where the obstacles 70a and 70b are depicted. In this case, the obstacle detection unit 205 detects the obstacles 70a and 70b.

[0087] The obstacle detection unit 205 may detect the obstacles 70a, 70b using other methods. For example, the obstacle detection unit 205 may detect the obstacles 70a, 70b from an image of the parking lot 9 taken at the time when the vehicle 1 starts automatic driving by image recognition. The image used for image recognition is taken by the fixed camera 5 at the same timing as the second background image, for example. When this method is adopted, the first background image during teacher driving may not be stored in the storage unit 210. The obstacle detection unit 205 is an example of a detection unit in this embodiment.

[0088] The obstacle detection unit 205 sends the positions of the detected obstacles 70a, 70b to the travel feasibility determination unit 206. The positions of the obstacles 70a, 70b may be represented by, for example, coordinates on the second background image.

[0089] The traveling feasibility determination unit 206 determines whether or not the vehicle 1 is capable of traveling along the traveling route 8, based on the positions of the obstacles 70a, 70b detected by the obstacle detection unit 205 and the traveling route 8 of the vehicle 1. The traveling feasibility determination unit 206 is an example of a determination unit in this embodiment.

[0090] More specifically, if the position of the detected obstacles 70a, 70b is within a driving area 800 that the vehicle 1 passes through when traveling along the driving route 8, the driving feasibility determination unit 206 determines that the vehicle 1 is not able to travel along the driving route 8.

[0091] In this embodiment, the positions of the obstacles 70a, 70b within the travel area 800 means that at least a part of the obstacles 70a, 70b is included in the travel area 800. In the example shown in FIG. 3, the obstacle 70b is not located within the travel area 800, but a part of the obstacle 70a is located within the travel area 800. In this case, the travel feasibility determination unit 206 determines that the vehicle 1 is not allowed to travel along the travel route 8. The travel feasibility determination unit 206 sends the travel feasibility determination result to the communication device 201.

[0092] The memory management unit 207 manages the saving of data to the storage unit 210 and the reading of data from the storage unit 210. The memory management unit 207 also records a first background image 211, which is a photograph of the parking lot 9 taken when the vehicle 1 is traveling as a teacher, in the storage unit 210 in association with the vehicle ID. The vehicle ID may be registered, for example, as supplementary information of the first background image 211, or the vehicle ID may be associated with the first background image 211 in the form of a table or the like. The memory management unit 207 is an example of a recording unit in this embodiment.

[0093] In this embodiment, the memory management unit 207 executes the process of storing various data generated by other functional units in the storage unit 210, but these processes may be executed by other functional units.

[0094] Next, a description will be given of a hardware configuration of the parking assistance device 100. Fig. 6 is a diagram showing an example of a hardware configuration of the parking assistance device 100 according to the first embodiment.

[0095] Next, a hardware configuration of the parking assistance device 100 will be described. Fig. 6 is a diagram showing an example of a hardware configuration of the parking assistance device 100 according to the first embodiment. As shown in Fig. 6, the parking assistance device 100 has a CPU (Central Processing Unit) 11A, a ROM 11B, a RAM 11C, a CAN (Controller Area Network) I / F (Interface) 11D, a NW (NetWork) I / F 11E, a HDD 11F, etc., which are connected to each other via a bus 11G, and has a hardware configuration using a normal computer.

[0096] The CPU 11A is a calculation device that controls the entire ECU. Note that the CPU 11A is an example of a processor in the parking assistance device 100 of the present embodiment, and another processor or processing circuit may be provided instead of the CPU 11A.

[0097] The ROM 11B, the RAM 11C, and the HDD 11F function as the storage unit 110. For example, the ROM 11B stores programs and the like that realize various processes by the CPU 11A. The RAM 11C is, for example, a main storage device of the parking assistance device 100, and stores data necessary for various processes by the CPU 11A.

[0098] The CAN I / F 11D is an interface for transmitting and receiving information to and from other ECUs mounted on the vehicle 1 via the CAN in the vehicle 1. Note that a standard other than CAN may be adopted. The NW I / F 11E is a communication device 101 capable of communicating with an information processing device outside the vehicle 1 via a network such as the Internet.

[0099] It should be noted that the hardware configuration of the parking assistance device 100 shown in FIG. 6 is merely an example, and the present invention is not limited to this.

[0100] Although the hardware configuration of the server device 2 is not shown in the drawings, similar to the parking assistance device 100, the hardware configuration utilizes a normal computer equipped with a processor and a storage device.

[0101] Next, a flow of a driving route recording process executed by the parking assistance device 100 and the server device 2 of this embodiment configured as above will be described.

[0102] 7 is a sequence diagram showing an example of the flow of a driving route recording process executed by the parking assistance device 100 and the server device 2 according to the first embodiment. This process is executed when the driver drives the vehicle 1 manually as a teacher. As a prerequisite for starting this process, it is assumed that the vehicle 1 has arrived at the entrance to the parking lot 9.

[0103] When the vehicle 1 arrives at the entrance of the parking lot 9, the vehicle 1 enters the communication range of the communication device 201 of the server device 2. In this case, as shown in Fig. 7, the communication device 101 of the parking assistance device 100 and the communication device 201 of the server device 2 are in communication with each other (S101).

[0104] In this case, the communication device 101 of the parking assistance device 100 transmits the vehicle information of the vehicle 1 to the server device 2 (S102). The communication device 201 of the server device 2 receives the vehicle information. Then, the ID management unit 202 extracts the vehicle ID of the vehicle 1 included in the vehicle information received by the communication device 201, and sends it to the memory management unit 207. In the subsequent processing, various data are stored in association with the vehicle ID extracted by the ID management unit 202. In this manner, the association and management of various data with the vehicle ID is called ID management (S103). At this time, the ID management unit 202 may execute authentication processing of the vehicle ID of the vehicle 1 based on the vehicle ID information 213. For example, it may be possible to register the driving area 800 in the parking lot 9 only for the vehicle ID registered in advance.

[0105] Then, the driver's UI displayed on the display device 120 or other operation units accept a parking start operation by the driver for teacher driving (S104).

[0106] In this case, the communication device 101 of the parking assistance device 100 transmits a teacher driving start signal to the server device 2 (S105). The communication device 201 of the server device 2 receives the teacher driving start signal.

[0107] When the communication device 201 receives the teacher driving start signal, the memory management unit 207 of the server device 2 stores the image captured by the fixed camera 5 at this time in the storage unit 210 as a first background image 211 in association with the vehicle ID of the vehicle 1. At this time, the vehicle 1 has not yet entered the parking lot 9, so the first background image 211 does not depict the vehicle 1 itself, but depicts the state of the parking lot 9 where the vehicle 1 is scheduled to drive.

[0108] Then, the driver of the vehicle 1 starts teacher driving (S107). In addition to the timing when the parking start operation of S104 is accepted, the communication device 101 may also transmit a signal to the server device 2 when the vehicle 1 actually starts teacher driving. While the driver is manually performing teacher driving, the driving information recording unit 103 records the driving information of the vehicle 1 during teacher driving. In addition, the image processing unit 104 acquires surrounding images from the in-vehicle camera 16.

[0109] While the vehicle 1 is performing the teacher driving, the vehicle detection unit 203 of the server device 2 detects the vehicle 1 from a plurality of images captured in time series by the fixed camera 5. In addition, the area recognition unit 204 recognizes the driving area 800 of the vehicle 1 in the parking lot 9 based on the change in the position of the vehicle 1 during the teacher driving detected by the vehicle detection unit 203 (S108).

[0110] Then, the driver's UI displayed on the display device 120 or other operation unit receives an operation by the driver to end the teacher driving (S109).

[0111] In this case, the communication device 101 of the parking assistance device 100 transmits a teacher-driving end signal to the server device 2 (S110). The communication device 201 of the server device 2 receives the teacher-driving end signal. When the communication device 201 receives the teacher-driving end signal, the detection of the vehicle 1 by the vehicle detection unit 203 and the recognition of the driving area 800 by the area recognition unit 204 are completed. The area recognition unit 204 stores the recognized driving area 800 in the storage unit 210 via the memory management unit 207. Here, the processing of this sequence diagram is completed.

[0112] Next, a flow of the parking assistance process executed by the parking assistance device 100 and the server device 2 of this embodiment configured as above will be described.

[0113] 8 is a sequence diagram showing an example of the flow of parking assistance processing executed by the parking assistance device 100 and the server device 2 according to the first embodiment. As a premise of the processing in this sequence diagram, it is assumed that the travel area 800 for the teacher travel of the vehicle 1 has already been registered in the server device 2.

[0114] When the vehicle 1 arrives at the entrance of the parking lot 9, the vehicle 1 enters the communication range of the communication device 201 of the server device 2. In this case, as shown in Fig. 7, the communication device 101 of the parking assistance device 100 and the communication device 201 of the server device 2 are in communication with each other (S101).

[0115] In this case, the communication device 101 of the parking assistance device 100 transmits the vehicle information of the vehicle 1 to the server device 2 (S201). The communication device 201 of the server device 2 receives the vehicle information.

[0116] Then, the ID management unit 202 extracts the vehicle ID of vehicle 1 included in the vehicle information received by the communication device 201, and executes authentication processing (S203). For example, the ID management unit 202 may determine whether or not the vehicle ID of vehicle 1 is registered in the vehicle ID information 213 and the driving area information 212. If the vehicle ID of vehicle 1 is not registered in the driving area information 212, the communication device 101 may transmit to vehicle 1 information indicating that teacher driving is possible, but parking assistance by automatic driving is not possible. Note that this processing is not essential.

[0117] Next, the driver's UI displayed on the display device 120 or other operation unit accepts an operation by the driver to start parking assistance for teacher driving (S204).

[0118] In this case, the communication device 101 of the parking assistance device 100 transmits a parking assistance start signal to the server device 2 (S205). The communication device 201 of the server device 2 receives the parking assistance start signal.

[0119] When the communication device 201 receives the parking assistance start signal, the obstacle detection unit 205 of the server device 2 acquires a second background image of the parking lot 9 captured by the fixed camera 5 at this time. The obstacle detection unit 205 may store the second background image in the storage unit 210. Then, the obstacle detection unit 205 detects the obstacles 70a and 70b by extracting the difference between the second background image and the first background image 211 stored in the storage unit 210 (S206).

[0120] Then, the traveling feasibility determination unit 206 determines whether the obstacles 70a, 70b have entered the traveling area 800 based on the positions of the obstacles 70a, 70b detected by the obstacle detection unit 205 and the traveling route 8 of the vehicle 1 (S207).

[0121] It should be noted that while the processes of S206 and S207 are being executed, the image processing unit 104 of the parking assistance device 100 may start the process of estimating the self-position based on the surrounding image acquired from the in-vehicle camera 16.

[0122] Next, the travel feasibility determination unit 206 determines whether the vehicle 1 is capable of traveling along the travel route 8 (S208). For example, the travel feasibility determination unit 206 determines that the vehicle 1 is not capable of traveling along the travel route 8 when at least a part of either the obstacle 70a or 70b is invading the travel area 800.

[0123] Then, the communication device 201 transmits the driving possibility information to the parking assistance device 100 (S209). The communication device 101 of the parking assistance device 100 receives the driving possibility information.

[0124] Furthermore, when the travel possibility information indicates that travel is not permitted, the communication device 201 may transmit the second background image together with the travel possibility information to the parking assistance device 100. In this case, the communication device 201 may transmit to the parking assistance device 100 information indicating the position on the second background image of the obstacle 70a that is determined by the travel possibility determination unit 206 to be located within the travel area 800, among the obstacles 70a, 70b detected by the obstacle detection unit 205.

[0125] Based on the driving feasibility information, the parking assistance device 100 starts parking assistance by automatic driving or outputs a notification to the driver that driving is not possible (S210).

[0126] For example, when the driving possibility information indicates that driving is possible, the correction unit 105 of the parking assistance device 100 starts parking assistance by automatic driving by transmitting the steering / braking information 111 corrected according to the situation around the vehicle 1 to the vehicle control system 18. In this case, the display control unit 102 may cause the display device 120 to display that automatic driving is about to start.

[0127] Furthermore, when the driving permission information indicates that driving is not permitted, the display control unit 102 of the parking assistance device 100 causes the display device 120 to display a notice to the driver that driving is not permitted.

[0128] Fig. 9 is a diagram showing an example of a notification screen indicating that driving is not possible according to the first embodiment. As shown in Fig. 9, the display control unit 102 causes the display device 120 to display a message M1 indicating that automatic driving cannot be started because an obstacle 70a exists.

[0129] 9, the display control unit 102 may display the second background image 51 transmitted from the server device 2 on the display device 120. In this case, the display control unit 102 may display a frame 700 so as to emphasize an object detected as an obstacle 70a by the obstacle detection unit 205 of the server device 2, for example. The display control unit 102 may also display an image representing the driving route 8 or a driving area 800 on the second background image 51 in a superimposed manner.

[0130] From the notification screen, the driver of the vehicle 1 understands that the vehicle 1 cannot park in the target parking space 903 on the same driving route 8 as the teacher driving. After that, for example, the driver of the vehicle 1 may switch to parking by manual driving to park the vehicle 1. Alternatively, the driver may take measures such as removing the obstacle 70a, and then start the process again from S201 in Fig. 9. Note that the display mode of the notification screen shown in Fig. 9 is an example, and is not limited thereto.

[0131] Here, the process of the sequence diagram shown in FIG. 8 ends.

[0132] In this manner, the server device 2 of the present embodiment detects the obstacles 70a, 70b in the parking lot 9, and determines whether the vehicle 1 can travel along the travel route 8 according to the positions of the obstacles 70a, 70b and the travel route 8 of the vehicle 1, and transmits the travel possibility determination result to the parking assistance device 100 of the vehicle 1. Therefore, according to the server device 2 of the present embodiment, before the vehicle 1 starts moving based on the registered travel route 8, it is possible to determine whether the vehicle 1 can travel along the travel route 8. Therefore, before the vehicle 1 starts automatic travel, the driver of the vehicle 1 can grasp whether the vehicle 1 can travel along the travel route 8, and therefore it is possible to reduce the trouble of having to redo parking because the vehicle 1 cannot travel due to the obstacle 70a after traveling partway.

[0133] Furthermore, the server device 2 of this embodiment stores a first background image 211 of the parking lot 9 taken during teacher driving of the vehicle 1 in association with the vehicle ID of the vehicle 1, and detects obstacles 70a, 70b based on the first background image 211 and a second background image 51 of the parking lot 9 taken at the time when the vehicle 1 starts automatic driving. Therefore, according to the server device 2 of this embodiment, it is possible to efficiently detect obstacles 70a, 70b that exist in the parking lot 9 at the time when the vehicle 1 starts automatic driving, based on the state of the parking lot 9 during teacher driving when the vehicle 1 was able to travel the driving route 8 without any problems.

[0134] Furthermore, the server device 2 of this embodiment detects the obstacles 70a, 70b by extracting the difference between the first background image 211 and the second background image 51. Therefore, the server device 2 of this embodiment can detect the obstacles 70a, 70b that did not exist during teacher driving but exist at the time when the vehicle 1 starts automatic driving with high accuracy. For example, the server device 2 of this embodiment does not determine that an object that exists in the same position as during teacher driving is an obstacle 70a, 70b, so that it is possible to prevent the standard for notifying the vehicle 1 from becoming unnecessarily strict.

[0135] Furthermore, the server device 2 of this embodiment detects the obstacles 70a, 70b based on the first background image 211 and the second background image 51 captured by the fixed camera 5 installed in the parking lot 9. Unlike the in-vehicle camera 16 mounted on the vehicle 1, the fixed camera 5 installed in the parking lot 9 does not move, so the capturing range and capturing angle of the first background image 211 and the second background image 51 do not change. Therefore, according to the server device 2 of this embodiment, the obstacles 70a, 70b can be detected with high accuracy from the difference between the first background image 211 and the second background image 51.

[0136] As another example of the obstacle detection means by the server device 2 of this embodiment, the obstacles 70a, 70b may be detected by image recognition from an image of the parking lot 9 taken at the time when the vehicle 1 starts automatic traveling. According to the server device 2 adopting such a configuration, the obstacles 70a, 70b existing in the parking lot 9 can be detected even if the image taken during the teacher traveling of the vehicle 1 is not saved. Furthermore, even when the above configuration is adopted, the obstacles 70a, 70b located far from the starting position of the traveling route 8 can be detected by using the image taken by the fixed camera 5 installed in the parking lot 9, and the obstacles 70a, 70b can be detected at an earlier stage than when the obstacles 70a, 70b are detected only by the on-board camera 16 of the vehicle 1.

[0137] Furthermore, when the positions of the detected obstacles 70a, 70b are within a driving area 800 that the vehicle 1 passes through when traveling along the driving route 8, the server device 2 of this embodiment determines that the vehicle 1 cannot travel along the driving route 8. Therefore, according to the server device 2 of this embodiment, it is possible to determine whether or not the vehicle 1 can travel while excluding obstacles 70b and the like that are located away from the driving area 800, and it is possible to reduce the issuance of unnecessary notifications that traveling is not possible.

[0138] Furthermore, the parking assistance device 100 of this embodiment transmits a vehicle ID capable of identifying the vehicle 1 to the server device 2, receives from the server device 2 travel propriety information indicating whether travel is possible along the travel route 8, and outputs a notification when the received travel propriety information indicates that travel along the travel route 8 is not possible. Therefore, according to the server device 2 of this embodiment, it is possible to notify the driver of whether travel is possible along the registered travel route 8 before the vehicle 1 starts moving along the travel route 8.

[0139] Furthermore, the parking assistance device 100 of this embodiment displays, as a notification, a message M1 indicating that automatic driving cannot be started due to the presence of the obstacle 70a on the display device 120 in the vehicle 1. Therefore, according to the server device 2 of this embodiment, the driver can be made aware of the presence of the obstacle 70a in the parking lot 9 before the vehicle 1 starts moving based on the driving route 8, so that the driver can consider measures such as manual driving or moving the obstacle 70a at an early stage.

[0140] Second Embodiment In the above-described first embodiment, the server device 2 records one driving area 800 for each vehicle 1. In the second embodiment, the server device 2 can record a plurality of driving areas 800 for each vehicle 1.

[0141] The parking assistance system S100 of this embodiment includes an in-vehicle system S1 and an infrastructure system S2, similar to the first embodiment.

[0142] Similar to the first embodiment, the infrastructure system S2 includes a fixed camera 5 and a server device 2. Similar to the first embodiment, the server device 2 includes a communication device 201, an ID management unit 202, a vehicle detection unit 203, an area recognition unit 204, an obstacle detection unit 205, a travel permission determination unit 206, a memory management unit 207, and a storage unit 210.

[0143] As in the first embodiment, the in-vehicle system S1 includes a display device 120, an in-vehicle camera 16, a parking assistance device 100, and a vehicle control system 18. As in the first embodiment, the parking assistance device 100 includes a communication device 101, a display control unit 102, a driving information recording unit 103, an image processing unit 104, a correction unit 105, and a storage unit 110.

[0144] The storage unit 210 of the server device 2 in this embodiment can store a plurality of travel routes 8 for one vehicle ID.

[0145] Fig. 10 is a diagram showing an example of the driving area information 212a according to the second embodiment. As shown in Fig. 10, the driving area information 212a stored in the storage unit 210 according to the present embodiment stores a vehicle ID, a driving route ID, a priority order, and a driving area 800 in association with each other.

[0146] The travel route ID is identification information capable of identifying the travel route 8 of each vehicle ID. The travel route ID may be unique for each vehicle ID. In the example shown in FIG. 10, two travel route IDs, "R001" and "R002", are associated with the vehicle ID "001". Three travel route IDs, "R001", "R002", and "R003", are associated with the vehicle ID "003". The travel route ID "R001" associated with the vehicle ID "001" and the travel route ID "R001" associated with the vehicle ID "003" are different travel routes 8. The travel route ID is an example of travel route identification information in this embodiment.

[0147] The travel route ID is shared as the same ID by the server device 2 and the parking assistance device 100 of each vehicle 1. The function of initially determining the travel route ID may be possessed by either the server device 2 or the parking assistance device 100. For example, during teacher driving of the vehicle 1, the travel information recording unit 103 of the parking assistance device 100 may assign a travel route ID to the recorded travel route 8, and the communication device 101 may transmit the travel route ID to the server device 2. Alternatively, during teacher driving of the vehicle 1, the ID management unit 202 of the server device 2 may assign a travel route ID to the travel route 8, and the communication device 201 may transmit the travel route ID to the parking assistance device 100.

[0148] Furthermore, the storage unit 210 of this embodiment can store a plurality of travel routes 8 corresponding to one vehicle ID with priority order. The priority order is the order in which the travel feasibility determination unit 206 determines whether or not the vehicle 1 can travel on the travel route IDs associated with each vehicle ID. When the travel feasibility determination unit 206 determines that the vehicle 1 can travel on any of the travel routes 8, it does not determine whether or not the vehicle 1 can travel on any of the travel routes 8 with a lower priority order than the travel route 8 that can be traveled.

[0149] The priority order may be input by the driver through a driver's UI or other operation unit when the driving route is recorded by the teacher driving, for example. Alternatively, the driving information recording unit 103 of the parking assistance device 100 or the ID management unit 202 of the server device 2 may automatically determine the priority order based on criteria such as the shortest time required for parking.

[0150] In FIG. 10, a priority order item is provided separately from the travel route ID. However, if the travel route ID includes a numerical value, the number of the travel route ID may indicate the level of priority.

[0151] 10, the travel area information 212a is illustrated in the form of one table, but a separate table may be generated for each vehicle ID. Also, the travel area information 212a may be in a form other than a table in which the vehicle ID, the travel route ID, the priority order, and the travel area 800 are associated with each other.

[0152] In the present embodiment, the storage unit 210 may store a plurality of first background images 211 for each vehicle ID and each travel route ID. For example, the storage unit 210 may store the first background image 211 in association with the vehicle ID and the travel route ID.

[0153] In addition, since the parking frame 903 assigned to the vehicle 1 does not change even if the travel route ID is different, only one first background image 211 may be registered for each vehicle ID, as in the first embodiment.

[0154] The driving feasibility determination unit 206 of this embodiment has the same functions as the first embodiment, and determines which driving routes 8 the vehicle 1 can drive from among the multiple driving routes 8 stored in the memory unit 210 and associated with the vehicle ID of the vehicle 1.

[0155] More specifically, when it is determined that the vehicle 1 cannot travel along the travel route 8 with the highest priority among the plurality of travel routes 8 associated with the vehicle ID of the vehicle 1, the travel feasibility determination unit 206 determines another travel route among the plurality of travel routes 8 that the vehicle 1 can travel along. For example, the travel feasibility determination unit 206 determines whether the travel area 800 overlaps with the positions of the obstacles 70a, 70b from the travel route ID with the highest priority among the travel route IDs associated with the vehicle ID of the vehicle 1 and registered in the travel area information 212a.

[0156] The communication device 201 of this embodiment has the same functions as those of the first embodiment, and transmits, to the parking assistance device 100, a travel route ID of a travel route 8 that is travelable among a plurality of travel routes 8 associated with the vehicle ID of the vehicle 1. More specifically, the communication device 201 transmits, to the parking assistance device 100, a travel route ID of a travel route 8 that is travelable by the vehicle 1, as determined by the travel feasibility determination unit 206.

[0157] The memory management unit 207 of this embodiment has the same functions as those of the first embodiment, and associates the traveling area 800 recognized by the area recognition unit 204 with the vehicle ID and the traveling route ID specified by the ID management unit 202, and stores them as traveling area information 212a in the storage unit 210. Note that this process may be executed by the area recognition unit 204.

[0158] In addition, the memory management unit 207 of this embodiment may record the first background image 211 in the memory unit 210 for each teacher drive of the vehicle 1, in association with the vehicle ID and the driving route ID of the driving route 8 recorded in that teacher drive.

[0159] The storage unit 110 of the parking assistance device 100 of this embodiment can store a plurality of driving routes 8 based on a plurality of teacher driving. More specifically, the storage unit 110 of this embodiment stores steering and braking information and surrounding images for each of the plurality of driving routes 8.

[0160] Fig. 11 is a diagram showing an example of steering / braking information 111a according to the second embodiment. As shown in Fig. 11, the steering / braking information 111a of this embodiment may be, for example, a table in which a travel route ID, a priority order, steering information, and braking information are associated with each other. Note that the steering / braking information 111a may be information in which a travel route ID, a priority order, steering information, and braking information are associated with each other in a format other than a table.

[0161] 12 is a diagram showing an example of the association of a surrounding image 112 with a travel route ID and a priority order according to the second embodiment. The storage unit 110 may store surrounding image information 1112 in which the surrounding image 112 is associated with a travel route ID and a priority order, as shown in FIG 12. Since the images captured by the on-board camera 16 of the vehicle 1 differ depending on the travel route 8 of the teacher travel, the surrounding images 112 associated with each travel route ID are different.

[0162] The surrounding image information 1112 may be information in which the surrounding image 112, the driving route ID, and the priority are associated with each other in a format other than a table.

[0163] The communication device 101 of the parking assistance device 100 of this embodiment has the same functions as those of the first embodiment, and receives from the server device 2 a travel ID indicating a travel route 8 on which the vehicle 1 can travel, among a plurality of travel routes 8.

[0164] The display control unit 102 of the parking assistance device 100 of this embodiment has the same functions as those of the first embodiment, and outputs a notification indicating a driving route 8 that the vehicle 1 can travel on, based on a driving route ID received by the communication device 101 from the server device 2.

[0165] Fig. 13 is a diagram showing an example of a notification screen of a possible driving route 8 according to the second embodiment. As shown in Fig. 13, the display control unit 102 causes the display device 120 to display a message M2 indicating the presence of an obstacle 70a and the possible driving route 8 on which the vehicle 1 can travel.

[0166] 13, the display control unit 102 may cause the second background image 51 transmitted from the server device 2 to be displayed on the display device 120. The display control unit 102 may also cause a frame 700 to be displayed so as to emphasize an object detected as an obstacle 70a by the obstacle detection unit 205 of the server device 2.

[0167] Furthermore, as shown in FIG. 13, the display control unit 102 of this embodiment may display, on the second background image 51, a driving route image 81 indicating a driving route 8 along which the vehicle 1 can travel.

[0168] The travel route image 81 may be an image based on the shape of the travel route 8 recognized by the area recognition unit 204 of the server device 2, for example. In this case, the communication device 201 of the server device 2 transmits the shape of the travel route 8 recognized by the area recognition unit 204 to the parking assistance device 100. Alternatively, the display control unit 102 may estimate the shape of the travel route 8 based on the steering / braking information 111a of the parking assistance device 100, and align and display the estimated shape of the travel route 8 with the second background image 51. The alignment criteria are, for example, the entrance of the parking lot 9, which is the starting position of the travel route 8, and the parking space 903, which is the ending position of the travel route 8, etc.

[0169] Furthermore, as shown in FIG. 13, the display control unit 102 of this embodiment may display an OK button 1201 and a cancel button 1202 on a notification screen of the possible driving route 8.

[0170] The OK button 1201 is an image button that can accept a driver's operation to approve the start of automatic driving along the recommended driving route shown in the message M2. When the OK button 1201 is pressed by the driver, the correction unit 105 of the parking assistance device 100 reads out the steering information and braking information corresponding to the recommended driving route "R002" shown in the message M2 from among the steering information and braking information registered in the steering / braking information 111, and corrects them according to the surrounding conditions of the vehicle 1. The correction unit 105 starts parking assistance by automatic driving by transmitting the corrected steering / braking information 111 to the vehicle control system 18.

[0171] The cancel button 1202 is an image button that can accept a driver's operation to end parking assistance by automatic driving. When the cancel button 1202 is pressed, the parking assistance ends. In this case, the driver may park the vehicle 1 in the parking space 903 by, for example, manual driving.

[0172] It should be noted that the configuration of the notification screen shown in FIG. 13 is just an example and is not limited to this.

[0173] In addition, if the driving feasibility determination unit 206 of the server device 2 determines that none of the driving routes 8 are drivable, the display control unit 102 may cause the display device 120 to display a notification screen notifying the user that there are no drivable driving routes and suggesting that the user switch to manual driving.

[0174] In this way, the server device 2 of this embodiment can store a plurality of driving routes 8 for one vehicle ID, and determines a driving route 8 that the vehicle 1 can travel from among the plurality of driving routes 8. The server device 2 of this embodiment transmits a driving route ID indicating a travelable driving route 8 from among the plurality of driving routes 8 to the parking assistance device 100. Therefore, according to the server device 2 of this embodiment, in addition to achieving the same effect as the first embodiment, it is possible to propose a travelable driving route 8 to the vehicle 1 even when an obstacle 70a is present near any of the driving routes 8.

[0175] In addition, the server device 2 of this embodiment can store a plurality of driving routes 8 corresponding to one vehicle ID with priority order, and when it is determined that the vehicle 1 cannot travel along the driving route 8 with the highest priority order among the plurality of driving routes 8, it determines another driving route 8 on which the vehicle 1 can travel among the plurality of driving routes 8. The server device 2 of this embodiment transmits a travel ID indicating a driving route determined as another driving route 8 on which the drivable vehicle 1 can travel among the plurality of driving routes 8 to the parking assistance device 100. Therefore, according to the server device 2 of this embodiment, when a plurality of driving routes 8 are registered, it is possible to efficiently identify a recommended driving route.

[0176] Furthermore, the parking assistance device 100 of this embodiment stores a plurality of driving routes 8 based on a plurality of teacher drivings, and receives a driving route ID representing a driving route 8 that the vehicle 1 can travel among the plurality of driving routes 8 from the server device 2. The server device 2 of this embodiment outputs a notification representing a driving route 8 that the vehicle 1 can travel based on the received driving route ID. Therefore, according to the parking assistance device 100 of this embodiment, it is possible to suggest a drivable driving route 8 to the driver depending on the situation of the parking lot 9 at the start of parking assistance.

[0177] (Variation 1) In each of the above-described embodiments, the case where the number of fixed cameras 5 installed in the parking lot 9 is one has been described as an example, but multiple fixed cameras 5 may be installed in the parking lot 9. For example, multiple fixed cameras 5 may be installed in one parking lot 9 when the parking lot 9 is larger than the range that can be captured by one fixed camera 5 or when the parking lot 9 is divided into multiple floors.

[0178] Fig. 14 is a diagram showing an example of the relationship between the fixed cameras 5a to 5c and the server device 2 according to the first modification. As shown in Fig. 14, each of the fixed cameras 5a to 5c is capable of communicating with the server device 2 and transmits captured images to the server device 2. The fixed cameras 5a to 5c are assumed to be identifiable by identification information such as a camera ID.

[0179] In this modified example, the vehicle detection unit 203, the area recognition unit 204, the obstacle detection unit 205, and the memory management unit 207 use images captured by the fixed cameras 5a to 5c for processing.

[0180] For example, the vehicle detection unit 203 of this modified example detects the vehicle 1 from a plurality of images taken in time series by the plurality of fixed cameras 5a to 5c while the vehicle 1 is traveling as a teacher.

[0181] Moreover, the area recognition unit 204 of this modified example recognizes a driving area 800 of the vehicle 1 in the parking lot 9 based on a change in the position of the vehicle 1 during teacher driving detected by the vehicle detection unit 203.

[0182] Moreover, the obstacle detection unit 205 of this modification detects obstacles 70a, 70 based on the first background image 211 and the second background image 51 captured by each of the fixed cameras 5a to 5c. The first background image 211 and the second background image 51 to be compared are images captured by the same fixed camera 5.

[0183] For example, the obstacle detection unit 205 detects the obstacles 70a and 70b by extracting the difference between the first background image 211 captured by the fixed camera 5a and the second background image 51 captured by the fixed camera 5a. The obstacle detection unit 205 also detects the obstacles 70a and 70b by extracting the difference between the first background image 211 captured by the fixed camera 5b and the second background image 51 captured by the fixed camera 5b. The obstacle detection unit 205 detects the obstacles 70a and 70b by extracting the difference between the first background image 211 captured by the fixed camera 5c and the second background image 51 captured by the fixed camera 5c.

[0184] In addition, the memory management unit 207 of this modified example associates the camera ID capable of identifying each fixed camera 5a to 5c, the driving area 800 recognized by the area recognition unit 204 for each fixed camera 5a to 5c, and the vehicle ID identified by the ID management unit 202, and stores them in the memory unit 210 as driving area information 212.

[0185] Furthermore, the memory management unit 207 of this modified example causes the first background image 211 captured by each of the fixed cameras 5a to 5c to be recorded in the storage unit 210 in association with the camera ID and the vehicle ID.

[0186] The vehicle detection unit 203, the area recognition unit 204, the obstacle detection unit 205, and the memory management unit 207 may be provided separately for each of the fixed cameras 5a to 5c. For example, the server device 2 may be provided with three vehicle detection units 203, three area recognition units 204, three obstacle detection units 205, and three memory management units 207, the number of which is the same as the number of the fixed cameras 5a to 5c. Alternatively, one vehicle detection unit 203, area recognition unit 204, obstacle detection unit 205, and memory management unit 207 may execute processing for each of the fixed cameras 5a to 5c.

[0187] According to the parking assistance device 100 and the server device 2 of this modification, even if the parking lot 9 is larger than the range that can be captured by one fixed camera 5 or if the parking lot 9 is divided into multiple floors, it is possible to notify the driver of the vehicle 1 of whether or not the vehicle 1 can travel along the travel route 8 before starting automatic travel of the vehicle 1. Note that the number of the fixed cameras 5a to 5c is not limited to the example shown in FIG.

[0188] (Variation 2) The fixed camera 5 may be provided with the functions described as the functions of the server device 2 in each of the above-mentioned embodiments. That is, the ID management unit 202, the vehicle detection unit 203, the area recognition unit 204, the obstacle detection unit 205, the travel possibility determination unit 206, the memory management unit 207, and the storage unit 210 may be provided inside the fixed camera 5. When this configuration is adopted, the fixed camera 5 is an example of an information processing device.

[0189] Furthermore, as described in the first variant example, when multiple fixed cameras 5a to 5c are installed in the parking lot 9, each of the fixed cameras 5a to 5c may include an ID management unit 202, a vehicle detection unit 203, an area recognition unit 204, an obstacle detection unit 205, a driving feasibility determination unit 206, a memory management unit 207, and a memory unit 210.

[0190] (Variation 3) In each of the above-described embodiments, the timing for capturing the first background image is before the start of the teacher run, but the timing for capturing the first background image may be after the teacher run ends.

[0191] (Variation 4) In each of the above-described embodiments, the travel feasibility determination unit 206 determines that the vehicle 1 cannot travel along the travel route 8 when the positions of the detected obstacles 70a, 70b are within the travel area 800, but the determination criterion is not limited to this. For example, even if the obstacles 70a, 70b are not located within the travel area 800, the travel feasibility determination unit 206 may determine that the vehicle 1 cannot travel along the travel route 8 when the distance between the outline of the travel area 800 and the obstacles 70a, 70b is equal to or less than a specified distance. The specified distance is not particularly limited, and may be set in advance by, for example, an administrator of the infrastructure system S2.

[0192] (Variation 5) In the above-described second embodiment, the storage unit 210 has been described as storing a plurality of driving routes 8 corresponding to one vehicle ID with priority order, but the storage unit 210 may store a plurality of driving routes 8 without prioritizing them. In this case, the travel possibility determination unit 206 of the server device 2 may determine whether or not it is possible to travel for all of the driving routes 8 associated with the vehicle ID of the vehicle 1 that has transmitted the parking support start signal.

[0193] (Variation 6) In each of the above-described embodiments, the display device 120 and the vehicle-mounted camera 16 are not included in the parking assistance device 100. However, a configuration in which the parking assistance device 100 includes the display device 120 and the vehicle-mounted camera 16 may be adopted.

[0194] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents described in the claims, as well as in the scope and spirit of the invention. [Explanation of symbols]

[0195] 1 vehicle 2. Server equipment 5. Fixed Camera 8. Travel route 9. Parking 12 Body 16,16a~16c Vehicle-mounted camera 18 Vehicle Control System 51 Second background image 70a, 70b Obstacles 81 Route image 100 Parking assistance device 101 Communication equipment 102 Display control unit 103 Driving information recording unit 104 Image processing section 105 Correction section 110 Storage section 111,111a Steering and braking information 112 Nearby Images 120 Display device 201 Communication equipment 202 ID Management Department 203 Vehicle detection unit 204 Area recognition part 205 Obstacle detection unit 206 Driving Property Judgment Unit 207 Memory Management Unit 210 Storage section 211 First background image 212 Driving area information 212a Driving area information 213 Vehicle ID Information 800 Travel area Parking spaces 901~904 1112 Nearby image information 1201 OK button 1202 Cancel button S1 In-Vehicle System S2 Infrastructure System S100 Parking Assist System

Claims

1. a communication unit that receives vehicle identification information capable of identifying a vehicle from a plurality of vehicles that are autonomously traveling in a parking lot along a travel route recorded based on a teacher driving by a driver; a storage unit that stores the vehicle identification information in association with a travel area that is an area through which a body of the vehicle passes when the vehicle travels along the travel route; A detection unit that detects an obstacle; a determination unit that determines whether or not the vehicle is capable of traveling along the travel route according to the travel area of ​​the vehicle identified among the plurality of vehicles based on the received vehicle identification information and the position of the detected obstacle, The communication unit transmits a determination result of whether or not the vehicle is capable of traveling made by the determination unit to the specified vehicle among the plurality of vehicles. Information processing device.

2. a recording unit that causes a first background image of the parking lot taken during the teacher driving of the vehicle to be recorded in the storage unit in association with the vehicle identification information; the detection unit detects the obstacle based on a second background image in which the parking lot is captured at a time when the vehicle starts the automatic traveling and the first background image; The information processing device according to claim 1 .

3. the detection unit detects the obstacle by extracting a difference between the first background image and the second background image. The information processing device according to claim 2 .

4. The detection unit detects the obstacle from an image of the parking lot captured at the time when the vehicle starts the automatic traveling by image recognition. The information processing device according to claim 1 .

5. The determination unit determines that the vehicle cannot travel along the travel route when the position of the detected obstacle is within the travel area. The information processing device according to claim 1 .

6. The storage unit is capable of storing a plurality of driving routes for one of the vehicle identification information, The determination unit determines a travel route that the vehicle can travel on from among the plurality of travel routes; The communication unit transmits travel route identification information indicating a travelable travel route among the plurality of travel routes to the identified vehicle among the plurality of vehicles. The information processing device according to claim 2 .

7. The storage unit is capable of storing the plurality of travel routes corresponding to one of the vehicle identification information in order of priority; when it is determined that the vehicle cannot travel along the travel route with the highest priority among the plurality of travel routes, the determination unit determines another travel route among the plurality of travel routes on which the vehicle can travel; The communication unit transmits, to the identified vehicle among the plurality of vehicles, travel route identification information indicating a travel route determined by the determination unit as another travel route on which the vehicle can travel, among the plurality of travel routes. The information processing device according to claim 6.

8. The detection unit detects the obstacle based on an image captured by a fixed camera installed in the parking lot. The information processing device according to claim 1 .

9. An ID management unit that determines whether the received vehicle identification information is included in the vehicle identification information pre-registered in the storage unit. The information processing device according to claim 1 .

10. A first communication step in which a communication device of the information processing device receives vehicle identification information capable of identifying a vehicle from a plurality of vehicles that are autonomously driving in a parking lot along a driving route recorded based on a teacher driving by a driver; a detection step in which the information processing device detects an obstacle; a determination step in which the information processing device determines whether or not the vehicle can travel along the travel route in accordance with the travel area of ​​the plurality of vehicles based on a travel area that is an area through which the vehicle body passes when the vehicle travels along the travel route and that is stored in advance in association with the vehicle identification information in a storage unit of the information processing device, and a position of the detected obstacle; A second communication step in which the information processing device transmits a determination result of whether or not the vehicle can travel to the specified vehicle among the plurality of vehicles; The method includes:

Images