Management system and automated valet parking system

Abstract

This makes it easier for users to recognize that their request has been received by the automated valet parking system. [Solution] The automated valet parking system includes a vehicle to be automated valet parked in a parking lot and a terminal. The terminal includes at least one of a user terminal for the vehicle's user and an infrastructure terminal installed in the parking lot. In response to a request from the user regarding automated valet parking of the vehicle, the vehicle performs a first reaction and the terminal performs a second reaction.

Description

Technical Field

[0001] This disclosure relates to Automated Valet Parking (AVP) of vehicles in a parking lot.

Background Art

[0002] Patent Document 1 discloses automated valet parking in a parking lot. A vehicle corresponding to automated valet parking acquires route information from a parking lot system and autonomously travels along the acquired route.

Prior Art Document

Patent Document

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Consider automated valet parking in a parking lot. A request may be sent from a user of automated valet parking to an automated valet parking system. If the user does not know whether the request has reached the automated valet parking system, the user may feel anxious.

Means for Solving the Problems

[0005] A first aspect relates to a management system for managing automated valet parking of vehicles in a parking lot. The terminal includes at least one of a user terminal of a vehicle user and an infrastructure terminal installed in the parking lot. The management system includes one or more processors. The one or more processors receive a request regarding automated valet parking of a vehicle from the user terminal. In response to the request, one or more processors instruct the vehicle to perform a first reaction and the terminal to perform a second reaction.

[0006] The second point concerns automated valet parking systems. An automated valet parking system includes a vehicle that is subject to automated valet parking in a parking lot, and a terminal. The terminal includes at least one of the vehicle user's terminal and the infrastructure terminal installed in the parking lot. In response to a user's request for automated valet parking of their vehicle, the vehicle performs a first reaction, and the terminal performs a second reaction. [Effects of the Invention]

[0007] According to this disclosure, in response to a user request, the vehicle performs a first reaction, and the terminal performs a second reaction. If the user is looking at the vehicle, the user can notice the vehicle's first reaction. Even if the user is not looking at the vehicle, there is a high probability that the user will notice the terminal's second reaction. In other words, the certainty of the user recognizing the reaction is improved. By recognizing the reaction, the user can recognize that their request has reached the automated valet parking system. As a result, the user feels reassured and their trust in the automated valet parking system increases. [Brief explanation of the drawing]

[0008] [Figure 1] This is a conceptual diagram illustrating the overview of an automated valet parking system (AVP system). [Figure 2] This is a conceptual diagram illustrating an example of automated valet parking. [Figure 3] This is a conceptual diagram illustrating examples of user requests and responses in AVP. [Figure 4] This is a conceptual diagram illustrating examples of a first reaction by a vehicle and a second reaction by a terminal. [Figure 5] It is a conceptual diagram for explaining various examples of the linkage between the first reaction by a vehicle and the second reaction by a terminal. [Figure 6] It is a conceptual diagram for explaining the first example of supplementary information. [Figure 7] It is a conceptual diagram for explaining the second example of supplementary information. [Figure 8] It is a block diagram showing a configuration example of a vehicle. [Figure 9] It is a block diagram showing a configuration example of a user terminal. [Figure 10] It is a block diagram showing a configuration example of a backend system. [Figure 11] It is a block diagram showing a configuration example of a parking lot system.

Embodiments for Carrying Out the Invention

[0009] Embodiments of the present disclosure will be described with reference to the accompanying drawings. In the following description, automatic valet parking may be referred to as "AVP".

[0010] 1. Automatic Valet Parking System (AVP System) FIG. 1 is a conceptual diagram showing an overview of the AVP system 10 according to the present embodiment. The AVP system 10 is a system for AVP in a parking lot. The AVP system 10 includes a vehicle 100, a user terminal 200, a backend system 300, and a parking lot system 400.

[0011] The vehicle 100 is an object of AVP in the parking lot. The vehicle 100 is equipped with at least a function of autonomously traveling in the parking lot.

[0012] The user terminal 200 is a terminal operated by a user of the AVP service, that is, a user of the vehicle 100. Examples of the user terminal 200 include a smartphone and a PC.

[0013] The backend system 300 manages the AVP in one or more parking lots, users of the AVP service, etc. The parking lot system 400 is an infrastructure system installed in a certain parking lot and manages the AVP in that parking lot. The backend system 300 and the parking lot system 400 can also be collectively referred to as the "management system". The management system manages the AVP in the parking lot.

[0014] The vehicle 100 and the backend system 300 can communicate with each other. For example, the vehicle 100 and the backend system 300 communicate with each other using a mobile communication service. Also, in the parking lot, the vehicle 100 and the parking lot system 400 can communicate wirelessly with each other. For example, the vehicle 100 and the parking lot system 400 communicate wirelessly with each other using a wireless LAN. Also, the user terminal 200 and the backend system 300 can communicate with each other. For example, the user terminal 200 and the backend system 300 communicate with each other using a mobile communication service. Furthermore, the backend system 300 and the parking lot system 400 can communicate with each other by wire or wirelessly.

[0015] An example of the process of reserving the AVP service is as follows. Assume that the user's membership information is registered in the backend system 300 in advance. First, the user makes a reservation for the AVP. For example, the user operates the user terminal 200 and inputs the user's ID information, the desired parking lot, the desired usage date, the desired usage time (scheduled arrival time and scheduled departure time), etc. The user terminal 200 sends reservation request information including the input information to the backend system 300. The backend system 300 performs reservation processing based on the reservation request information and sends a reservation completion notification to the user terminal 200. Also, the backend system 300 provides the reservation information to the parking lot system 400 of the reserved parking lot.

[0016] Figure 2 is a conceptual diagram for explaining an example of AVP in a parking lot.

[0017] Vehicle 100 recognizes its surroundings using recognition sensors (e.g., cameras) mounted on it. Vehicle 100 drives safely while recognizing its surroundings. Multiple markers M (landmarks) may be placed within the parking lot. Markers M are used to guide vehicle 100 within the parking lot. For example, vehicle 100 acquires images of its surroundings using its camera and recognizes markers M based on the images. Then, based on the recognition results of markers M, vehicle 100 performs localization to estimate its position in the parking lot with high accuracy. Based on the estimated vehicle position, vehicle 100 drives automatically within the parking lot.

[0018] One or more infrastructure cameras (CAMs) may be installed in the parking lot. The infrastructure camera (CAM) photographs the parking lot and acquires images showing the conditions of the parking lot. The parking lot system 400 communicates with the infrastructure camera (CAM) and acquires the images taken by the infrastructure camera (CAM). The parking lot system 400 detects vehicles (100) that are shown in the images by analyzing the images. The parking lot system 400 also estimates the position of the vehicles (100) shown in the images. Furthermore, the parking lot system 400 manages each vehicle (100) in the parking lot based on its position. The parking lot system 400 may provide the vehicle (100) with its position information. The vehicle (100) may automatically drive within the parking lot based on the position information provided by the parking lot system 400.

[0019] An example of the parking process (check-in) is as follows: Vehicle 100 stops in the parking area. In the parking area, the user gets out of vehicle 100 and requests to park using a user terminal 200 or the like. The management system (at least one of the backend system 300 and the parking system 400) authenticates the user and vehicle 100. Once authentication is complete, the control of vehicle 100 is transferred from the user to the management system. The management system also communicates with vehicle 100 and starts it up. The parking system 400 then assigns an available parking space to vehicle 100. The assigned available parking space becomes the target parking space, or destination, for vehicle 100 at the time of parking. Furthermore, the parking system 400 sets a driving route TP (target trajectory) from the parking area to the target parking space in the parking lot. The parking system 400 sends a parking instruction to vehicle 100. The parking instruction includes information on the target parking space and the driving route TP. In response to the parking instruction, vehicle 100 automatically drives to the target parking space according to the driving path TP. In other words, vehicle 100 automatically drives to follow the driving path TP based on its position. Then, vehicle 100 automatically parks in the target parking space. After parking is complete, the management system instructs vehicle 100 to stop its operation.

[0020] An example of the checkout process is as follows: The user requests checkout using a user terminal 200 or the like. The management system communicates with vehicle 100 and starts vehicle 100. When checking out, the designated checkout area becomes the destination for vehicle 100. The parking system 400 sets the driving route TP (target trajectory) from the parking space in the parking lot to the checkout area. The parking system 400 sends a checkout instruction to vehicle 100. The checkout instruction includes information on the designated checkout area and driving route TP. In response to the checkout instruction, vehicle 100 automatically drives to the checkout area according to the driving route TP. In other words, vehicle 100 automatically drives to follow the driving route TP based on its position. Then, vehicle 100 automatically stops at the checkout area. Operational authority for vehicle 100 is transferred from the management system to the user. The user gets into vehicle 100. Vehicle 100 starts moving towards the next destination.

[0021] 2. Reactions to user requests AVP users may submit requests regarding AVP to the AVP system 10. If users are unsure whether their requests have been successfully received by the AVP system 10, they may feel anxious. In other words, if there is no reaction to a user's request, the user may feel anxious. Therefore, the AVP system 10 according to this embodiment is configured to respond to user requests regarding AVP by providing an answer back.

[0022] Figure 3 is a conceptual diagram illustrating examples of user requests and responses in AVP.

[0023] For example, in the parking area, the user gets out of vehicle 100 and requests AVP activation using a user terminal 200. For example, the user launches the AVP app on user terminal 200. The display on user terminal 200 shows an "AVP Activation" button. When the user taps the "AVP Activation" button, an AVP activation request is sent from user terminal 200 to backend system 300. Upon receiving the AVP activation request, backend system 300 communicates with vehicle 100 and checks whether the vehicle 100's status meets the AVP activation conditions. The AVP activation conditions include, for example, that vehicle 100 is in the ignition-off state, that the windows are closed, that the doors are closed and locked, etc. If the vehicle 100's status does not meet the AVP activation conditions, backend system 300 rejects the AVP activation request. On the other hand, if the vehicle 100's status meets the AVP activation conditions, backend system 300 accepts the AVP activation request and proceeds with AVP activation.

[0024] Here, if the user does not know whether the AVP start request was successfully received by the AVP system 10, they may feel uneasy. In other words, if there is no reaction after sending the AVP start request, the user may feel uneasy. Also, if the AVP of vehicle 100 starts abruptly without any reaction after sending the AVP start request, the user may be surprised by the sudden start of the AVP.

[0025] Therefore, when the backend system 300 receives an AVP start request, it communicates with the vehicle 100 and instructs the vehicle 100 to take a predetermined action. In other words, in response to the AVP start request, the backend system 300 communicates with the vehicle 100 and instructs the vehicle 100 to take a predetermined action.

[0026] A reaction is defined by a combination of the device that performs the reaction, the operating pattern of that device, and the duration of its operation. Examples of devices that perform a reaction include lights, actuators, and horns. Examples of lights include turn signals, headlights, brake lights, and fog lights. Examples of actuators include wiper actuators that operate the wipers and actuators that automatically open and close the door mirrors.

[0027] Examples of visible reactions include flashing lights, operating wipers, and opening and closing door mirrors. Examples of audible reactions include sounding the horn. For example, the lights flash in a predetermined pattern for a predetermined period (e.g., a few seconds). Another example is that the door mirrors open and close in a predetermined pattern for a predetermined period. Yet another example is that the horn sounds in a predetermined pattern for a predetermined period.

[0028] The reaction performed by vehicle 100 is hereinafter referred to as the "first reaction." In response to the AVP start request from the user, the backend system 300 communicates with vehicle 100 and instructs vehicle 100 to perform the first reaction. Specifically, the backend system 300 sends first reaction information RAX1 to vehicle 100, which instructs the first reaction to be performed. The first reaction information RAX1 may include the content of the first reaction (type, pattern, duration). Typically, the content of the first reaction is predetermined. Vehicle 100 receives the first reaction information RAX1 from the backend system 300. Vehicle 100 performs the first reaction according to the first reaction information RAX1. That is, vehicle 100 performs the first reaction in response to the AVP start request from the user. This allows the user to recognize that the AVP start request has reached the AVP system 10. As a result, the user feels reassured and their trust in the AVP system 10 increases.

[0029] The content of the first reaction may be set to differ depending on whether the AVP start request is accepted or not. For example, the first reaction when the AVP start request is accepted may be flashing lights, and the first reaction when the AVP start request is rejected may be wiper activation. As an example, the first reaction may be flashing lights, and the flashing pattern of those lights may differ depending on whether the AVP start request is accepted or rejected.

[0030] The above explanation provided an example of an AVP initiation request and its first reaction. However, requests related to AVP are not limited to AVP initiation requests. Furthermore, the entity instructing vehicle 100 to perform the first reaction is not limited to the backend system 300. The parking system 400 may also instruct vehicle 100 to perform the first reaction.

[0031] In general, in response to a user request regarding the AVP, the management system (at least one of the backend system 300 and the parking system 400) communicates with the vehicle 100 and instructs the vehicle 100 to perform a first reaction. Specifically, the management system sends first reaction information RAX1 to the vehicle 100, instructing it to perform the first reaction. The vehicle 100 performs the first reaction according to the first reaction information RAX1. That is, the vehicle 100 performs the first reaction in response to the user's request regarding the AVP. This allows the user to recognize that their request has reached the AVP system 10. As a result, the user feels reassured and their trust in the AVP system 10 increases.

[0032] 3. Linking of vehicles and terminals As described above, vehicle 100 performs a first reaction in response to a user's request regarding AVP. However, if the user is not looking at vehicle 100, the user may not notice the first reaction by vehicle 100. Therefore, vehicle 100 may perform the first reaction, and another terminal (e.g., user terminal 200) may perform a second reaction. In other words, vehicle 100 and the terminal may work together to perform the first and second reactions, respectively.

[0033] Figure 4 is a conceptual diagram illustrating an example of a first reaction by a vehicle 100 and a second reaction by a terminal. The terminal is, for example, a user terminal 200. As another example, the terminal may be an infrastructure terminal 500 installed in a parking lot. The infrastructure terminal 500 is an AVP-dedicated terminal included in the parking lot system 400. The infrastructure terminal 500 has the same functions as the user terminal 200. The terminal may include both the user terminal 200 and the infrastructure terminal 500. Examples of second reactions performed by the terminal include flashing lights (e.g., displays), vibration, sound output, etc.

[0034] For example, in the receiving area, a user requests AVP (Automatic Vehicle Programming) to start using a user terminal 200 or the like. The AVP start request is sent from the user terminal 200 to the backend system 300.

[0035] In response to an AVP start request from the user, the backend system 300 obtains information on the content (type, pattern, duration) of the first and second reactions. For example, the first reaction includes at least one of the following: flashing the lights of vehicle 100, activating the wipers of vehicle 100, opening and closing the door mirrors of vehicle 100, and sounding the horn of vehicle 100. For example, the second reaction includes at least one of the following: flashing the lights (e.g., display) of a terminal, vibrating the terminal, and emitting sound from the terminal. The first and second reactions are associated with each other. For example, the first and second reactions may be predetermined and pre-associated. Alternatively, the backend system 300 may determine the content of the first and second reactions and associate them.

[0036] The backend system 300 communicates with the vehicle 100 and instructs the vehicle 100 to perform a first reaction. More specifically, the backend system 300 sends first reaction information RAX1 to the vehicle 100, which instructs the first reaction. The first reaction information RAX1 may include the content of the first reaction (type, pattern, duration). The vehicle 100 receives the first reaction information RAX1 from the backend system 300. The vehicle 100 performs the first reaction according to the first reaction information RAX1. That is, the vehicle 100 performs the first reaction in response to the AVP start request from the user.

[0037] Furthermore, the backend system 300 communicates with the terminal (at least one of the user terminal 200 and the infrastructure terminal 500) and instructs the terminal to perform a second reaction. More specifically, the backend system 300 sends second reaction information RAX2 to the terminal instructing it to perform a second reaction. The second reaction information RAX2 may include the details of the second reaction (type, pattern, duration). The terminal receives the second reaction information RAX2 from the backend system 300. The terminal performs a second reaction according to the second reaction information RAX2. That is, the terminal performs a second reaction in response to the AVP start request from the user.

[0038] In this way, in response to the AVP start request from the user, the backend system 300 instructs the vehicle 100 to perform the first reaction and the terminal to perform the second reaction. As a result, the vehicle 100 performs the first reaction, and in conjunction with it, the terminal performs the second reaction. In other words, the vehicle 100 and the terminal work together to perform the first and second reactions, respectively.

[0039] Figure 5 is a conceptual diagram illustrating various examples of the coordination between the first reaction by vehicle 100 and the second reaction by the terminal.

[0040] In example (A) in Figure 5, the first and second reactions are performed synchronously. Here, "synchronous" does not necessarily mean perfect synchronization, and may include errors due to communication delays or errors that are imperceptible to humans. The duration of the first reaction and the duration of the second reaction are set to be the same. The backend system 300 simultaneously transmits the first reaction information RAX1 and the second reaction information RAX2 to the vehicle 100 and the terminal, respectively. As a result, the vehicle 100 performs the first reaction, and in synchronous motion, the terminal performs the second reaction.

[0041] In examples (B) and (C) in Figure 5, the duration of the first reaction and the duration of the second reaction partially overlap. The first reaction may start before the second reaction or start after the second reaction. The first reaction may end before the second reaction or end after the second reaction. These cases are also included in the linkage between the first and second reactions.

[0042] In example (D) in Figure 5, the durations of the first reaction and the second reaction do not overlap, but the first and second reactions occur consecutively within a short period (e.g., 10 seconds). This case is also included in the sequence of the first and second reactions.

[0043] The type of the first reaction and the type of the second reaction may be the same. For example, the first reaction may include flashing the lights of vehicle 100, and the second reaction may include flashing the lights (e.g., display) of the terminal. Another example is that the first reaction may include sounding the horn of vehicle 100, and the second reaction may include outputting sound from the terminal. Matching the types of the first and second reactions in this way increases the sense of coordination (coordination) between the first reaction of vehicle 100 and the second reaction of the terminal.

[0044] As shown in example (E) in Figure 5, the patterns of the first reaction and the second reaction may be the same. For example, the first reaction includes turning the lights of the vehicle 100 ON / OFF in a predetermined pattern. The second reaction includes at least one of the following: turning the lights of the terminal ON / OFF in the same predetermined pattern, turning the vibration of the terminal ON / OFF in the same predetermined pattern, and turning the sound output from the terminal ON / OFF in the same predetermined pattern. By having the patterns of the first reaction and the second reaction match in this way, the sense of coordination (coordination) between the first reaction of the vehicle 100 and the second reaction of the terminal is increased.

[0045] The type and pattern of the first reaction may match. For example, the first reaction includes turning the lights of vehicle 100 ON / OFF in a predetermined pattern. The second reaction includes turning the lights of the terminal ON / OFF in the same predetermined pattern. By matching both the type and pattern of the first and second reactions, the sense of coordination (coordination) between the first reaction of vehicle 100 and the second reaction of the terminal is further enhanced.

[0046] The above explanation provided an example of an AVP initiation request and its first and second reactions. However, requests related to AVP are not limited to AVP initiation requests. Furthermore, the entity issuing reaction instructions is not limited to the backend system 300. The parking system 400 may also issue reaction instructions.

[0047] In general terms, in response to a user request regarding the AVP, the management system (at least one of the backend system 300 and the parking system 400) instructs the vehicle 100 to perform a first reaction and the terminal to perform a second reaction. In response to the user request regarding the AVP, the vehicle 100 performs the first reaction and the terminal performs the second reaction. In other words, the vehicle 100 performs the first reaction, and in conjunction with that, the terminal performs the second reaction.

[0048] If the user is looking at vehicle 100, the user can notice the vehicle's first reaction. Even if the user is not looking at vehicle 100, there is a high probability that the user will notice the terminal's second reaction. In other words, the certainty of the user recognizing the reaction is improved. By recognizing the reaction, the user can recognize that the request has reached the AVP system 10. As a result, the user feels reassured and their trust in the AVP system 10 increases.

[0049] 4. Notification of supplementary information The management system may notify the terminal of supplementary information. In this case, the second reaction information RAX2 includes the supplementary information. The second reaction by the terminal may include displaying the supplementary information on the terminal's screen.

[0050] 4-1. Example 1 Figure 6 is a conceptual diagram showing a first example of supplementary information. Even if vehicle 100 performs the first reaction described in Sections 2 and 3 above, the meaning of that first reaction may not necessarily be clear to the user. Therefore, in the first example, the supplementary information includes information explaining the meaning of the vehicle's first reaction. The terminal's second reaction includes displaying information explaining the meaning of vehicle 100's first reaction on the terminal's screen.

[0051] For example, in response to an AVP start request from a user, the backend system 300 communicates with the vehicle 100 and checks whether the state of the vehicle 100 meets the AVP start conditions. The AVP start conditions include, for example, that the vehicle 100 is in the ignition-off state, that the windows are closed, that the doors are closed and locked, etc. If the state of the vehicle 100 meets the AVP start conditions, the backend system 300 accepts the AVP start request and proceeds with AVP start. In this case, the first reaction is set to mean "AVP start request accepted," and the supplementary information is set to include information indicating "AVP start request accepted." On the other hand, if the state of the vehicle 100 does not meet the AVP start conditions, the backend system 300 rejects the AVP start request. In this case, the first reaction is set to mean "AVP start request rejected," and the supplementary information is set to include information indicating "the reason why the AVP start request was rejected" or "the operation required to accept the AVP start request."

[0052] As described above, according to the first example, the terminal's second reaction includes displaying information on the terminal's screen that explains the meaning of the vehicle 100's first reaction. This allows the user to accurately understand the meaning of the vehicle's first reaction.

[0053] 4-2. Second Example Figure 7 is a conceptual diagram showing a second example of supplementary information. The supplementary information may include an animation representing the vehicle 100 that performs the first reaction. In this case, the terminal's second reaction includes displaying the animation representing the vehicle 100 that performs the first reaction on the terminal's screen. The display of such an animation on the terminal increases the sense of interaction (cooperation) between the vehicle 100 and the terminal.

[0054] The supplementary information may include an animation that represents the interaction between vehicle 100 and the terminal. In this case, the terminal's second reaction includes displaying the animation representing the interaction between vehicle 100 and the terminal on the terminal's screen. The display of such an animation on the terminal enhances the sense of interaction (cooperation) between vehicle 100 and the terminal.

[0055] A combination of the first and second examples above is also possible.

[0056] 5. Combinations The above combinations of Section 2 and Section 4 are also possible. The above combinations of Section 3 and Section 4 are also possible.

[0057] 6. Example Configuration 6-1. Example of Vehicle Configuration Figure 8 is a block diagram showing an example configuration of a vehicle 100 according to this embodiment. The vehicle 100 includes a communication device 110, a sensor group 120, a running device 130, lights 140, an actuator 150, a horn 160, and a control device 170.

[0058] The communication device 110 communicates with the outside world via a communication network. For example, the communication device 110 communicates with the backend system 300. It also communicates with the parking system 400 via a wireless LAN.

[0059] The sensor group 120 includes recognition sensors, vehicle condition sensors, etc. Recognition sensors are used to recognize (detect) the surrounding conditions of the vehicle 100. Examples of recognition sensors include cameras, LiDAR (Laser Imaging Detection and Ranging), radar, etc. Vehicle condition sensors include speed sensors, acceleration sensors, yaw rate sensors, steering angle sensors, etc.

[0060] The running gear 130 includes a steering gear, a drive gear, and a braking gear. The steering gear steers the wheels. For example, the steering gear includes an electric power steering (EPS) system. The drive gear is a power source that generates driving force. Examples of drive gears include an engine, an electric motor, an in-wheel motor, etc. The braking gear generates braking force.

[0061] Light 140 includes turn signals (indicators), headlights, brake lights, fog lights, etc.

[0062] Examples of actuators 150 include wiper actuators for operating the wipers, actuators for automatically opening and closing the door mirrors, and so on.

[0063] The Horn 160 emits sound.

[0064] The control device 170 is a computer that controls the vehicle 100. The control device 170 includes one or more processors 171 (hereinafter simply referred to as processor 171) and one or more storage devices 172 (hereinafter simply referred to as storage devices 172). The processor 171 performs various processes. Examples of processors 171 include general-purpose processors, application-specific processors, CPUs (Central Processing Units), GPUs (Graphics Processing Units), ASICs (Application Specific Integrated Circuits), FPGAs (Field-Programmable Gate Arrays), integrated circuits, and / or combinations thereof. The processor 171 can also be called a processing circuitry. The storage devices 172 store various information. Examples of storage devices 172 include volatile memory, non-volatile memory, HDDs (Hard Disk Drives), SSDs (Solid State Drives), etc.

[0065] The vehicle control program 180 is a computer program for controlling the vehicle 100. The functions of the control device 170 may be realized through the cooperation of a processor 171 that executes the vehicle control program 180 and a storage device 172. The vehicle control program 180 is stored in the storage device 172. Alternatively, the vehicle control program 180 may be recorded on a computer-readable recording medium.

[0066] The control device 170 performs vehicle driving control to control the movement of the vehicle 100. Vehicle driving control includes steering control, acceleration control, and deceleration control. The control device 170 performs vehicle driving control by controlling the driving equipment 130 (steering equipment, drive equipment, braking equipment).

[0067] The control device 170 communicates with the backend system 300 and the parking system 400 via the communication device 110.

[0068] The control device 170 acquires driving environment information 190 that indicates the driving environment of the vehicle 100. The driving environment information 190 is stored in the storage device 172. For example, the driving environment information 190 includes surrounding conditions information, vehicle status information, map information, location information, etc.

[0069] The surrounding environment information indicates the recognition results from the recognition sensor. The surrounding environment information may include object information about objects recognized by the recognition sensor. Examples of objects around vehicle 100 include obstacles, white lines, marker M, etc. Examples of obstacles include walls, pillars, other vehicles, etc. The object information indicates the relative position and relative velocity of the object with respect to vehicle 100.

[0070] Vehicle status information indicates the vehicle status detected by vehicle status sensors. Examples of vehicle status include speed, acceleration, yaw rate, steering angle, etc.

[0071] The map information is map information of the parking lot in which vehicle 100 travels. The map information shows the layout of roads within the parking lot. The map information also shows the layout of stationary obstacles (e.g., walls, pillars) within the parking lot. Furthermore, the map information shows the layout of markers M within the parking lot. For example, the map information is provided by the parking lot system 400 that manages the parking lot. The control device 170 acquires the map information from the parking lot system 400 via the communication device 110.

[0072] The location information indicates the current position of vehicle 100 in the parking lot. For example, the control device 170 acquires highly accurate location information through localization. Specifically, the control device 170 calculates the approximate position of vehicle 100 in the parking lot based on vehicle status information (steering angle and speed). The control device 170 also recognizes markers M around vehicle 100 using a recognition sensor. Furthermore, the control device 170 acquires information on the placement of markers M around vehicle 100 from map information. The control device 170 corrects the position of vehicle 100 by matching the recognition results of markers M with their placement. This results in highly accurate location information.

[0073] Alternatively, the location information of the vehicle 100 may be estimated by the parking system 400 based on images captured by the infrastructure camera CAM. In this case, the control device 170 may obtain the location information from the parking system 400 via the communication device 110.

[0074] Furthermore, the control device 170 acquires information on the driving route TP in the parking lot. For example, the driving route TP is determined by the parking system 400, and the control device 170 acquires information on the driving route TP from the parking system 400 via the communication device 110. In another example, the control device 170 may determine the driving route TP based on map information and location information. Then, based on the location information, the control device 170 performs vehicle driving control so that the vehicle 100 travels according to the driving route TP.

[0075] Furthermore, the control device 170 obtains first reaction information RAX1 from the backend system 300 or the parking system 400 via the communication device 110. The first reaction information RAX1 may indicate the content of the first reaction. For example, the first reaction includes at least one of flashing the lights 140, activating the wipers, opening and closing the door mirrors, and sounding the horn 160. The control device 170 executes the first reaction according to the first reaction information RAX1.

[0076] 6-2. Example of User Terminal Configuration Figure 9 is a block diagram showing an example configuration of a user terminal 200 according to this embodiment. The user terminal 200 includes a communication device 210, an input device 220, a display device 230, a light 240, a vibrator 250, a speaker 260, and a control device 270.

[0077] The communication device 210 communicates with the outside world via a communication network. For example, the communication device 210 communicates with the backend system 300.

[0078] Examples of input devices 220 include touch panels, buttons, microphones, and the like.

[0079] Examples of the display device 230 include touch panels, displays, etc. The display device 230 and the input device 220 may be the same touch panel.

[0080] Examples of the light 240 include touch panels, displays, etc. The light 240 may be the same as the display device 230.

[0081] The vibrator 250 vibrates the user terminal 200.

[0082] Speaker 260 outputs sound.

[0083] The control device 270 is a computer that controls the user terminal 200. The control device 270 includes one or more processors 271 (hereinafter simply referred to as processor 271) and one or more storage devices 272 (hereinafter simply referred to as storage devices 272). The processors 271 perform various processes. Examples of processors 271 include general-purpose processors, application-specific processors, CPUs, GPUs, ASICs, FPGAs, integrated circuits, and / or combinations thereof. The processors 271 can also be called processing circuitry. The storage devices 272 store various information. Examples of storage devices 272 include volatile memory, non-volatile memory, HDDs, SSDs, etc.

[0084] The terminal control program 280 is a computer program for controlling the user terminal 200. The terminal control program 280 includes an AVP application. The functions of the control device 270 may be realized through the cooperation of the processor 271 that executes the terminal control program 280 and the storage device 272. The terminal control program 280 is stored in the storage device 272. Alternatively, the terminal control program 280 may be recorded on a computer-readable recording medium.

[0085] The user can input a request regarding AVP using the input device 220. The request information REQ indicates the request entered by the user. For example, the input device 220 and the display device 230 are configured as a touch panel. In the receiving area, the user launches the AVP app. The touch panel displays an "AVP Start" button. When the user taps the "AVP Start" button, request information REQ, which includes an AVP start request, is generated. The control device 270 transmits the request information REQ to the management system via the communication device 210.

[0086] Furthermore, the control device 270 obtains second reaction information RAX2 from the backend system 300 or the parking system 400 via the communication device 210. The second reaction information RAX2 may indicate the content of the second reaction. For example, the second reaction includes at least one of the following: flashing the light 240, activating the vibrator 250 to vibrate the user terminal 200, and outputting sound from the speaker 260. The control device 270 executes the second reaction according to the second reaction information RAX2.

[0087] The second reaction information RAX2 may include the supplementary information described in Section 4 above. In this case, the control device 270 displays the supplementary information on the display device 230.

[0088] Furthermore, the infrastructure terminal 500 has the same configuration as the user terminal 200.

[0089] 6-3. Example of a backend system configuration Figure 10 is a block diagram showing an example configuration of the backend system 300 according to this embodiment. The backend system 300 includes a communication device 310, one or more processors 320 (hereinafter simply referred to as processor 320), and one or more storage devices 330 (hereinafter simply referred to as storage devices 330).

[0090] The communication device 310 communicates with each vehicle 100. The communication device 310 also communicates with each user's user terminal 200. Furthermore, the communication device 310 communicates with the parking system 400 of each parking lot. Additionally, the communication device 310 may communicate with the infrastructure terminal 500 via the parking system 400.

[0091] The processor 320 performs various processes. Examples of the processor 320 include general-purpose processors, application-specific processors, CPUs, GPUs, ASICs, FPGAs, integrated circuits, and / or combinations thereof. The processor 320 can also be called processing circuitry. The storage device 330 stores various information. Examples of storage devices 330 include volatile memory, non-volatile memory, HDDs, SSDs, etc.

[0092] The management program 340 is a computer program for managing the AVP in the parking lot. The functions of the backend system 300 may be realized through the cooperation of the processor 320, which executes the management program 340, and the storage device 330. The management program 340 is stored in the storage device 330. The management program 340 may be recorded on a computer-readable recording medium.

[0093] The storage device 330 stores management information 350. The management information 350 may include user information and reservation information for each user. The management information 350 may also include facility information and reservation status information for each parking lot. When the processor 320 receives reservation request information from a user, it may perform reservation processing based on the management information 350.

[0094] The processor 320 receives request information REQ from the user terminal 200 via the communication device 310. The request information REQ is stored in the storage device 330.

[0095] In response to the request information REQ, the processor 320 obtains the first reaction information RAX1 and the second reaction information RAX2. The first reaction information RAX1 instructs the vehicle 100 to perform a first reaction. The first reaction information RAX1 may include the content of the first reaction. The second reaction information RAX2 instructs the terminal (at least one of the user terminal 200 and the infrastructure terminal 500) to perform a second reaction. The second reaction information RAX2 may include the content of the second reaction. The second reaction information RAX2 may include the supplementary information described in Section 4 above. The processor 320 transmits the first reaction information RAX1 to the vehicle 100 and the second reaction information RAX2 to the terminal via the communication device 310.

[0096] 6-4. Example of a Parking System Configuration Figure 11 is a block diagram showing an example configuration of a parking system 400 according to this embodiment. The parking system 400 includes a communication device 410, one or more processors 420 (hereinafter simply referred to as processor 420), and one or more storage devices 430 (hereinafter simply referred to as storage devices 430).

[0097] The communication device 410 communicates with each vehicle 100. The communication device 410 also communicates with the backend system 300. Furthermore, the communication device 410 may communicate with the infrastructure camera CAM installed in the parking lot.

[0098] The processor 420 performs various processes. Examples of the processor 420 include general-purpose processors, application-specific processors, CPUs, GPUs, ASICs, FPGAs, integrated circuits, and / or combinations thereof. The processor 420 can also be called processing circuitry. The storage device 430 stores various information. Examples of storage devices 430 include volatile memory, non-volatile memory, HDDs, SSDs, etc.

[0099] The management program 440 is a computer program for managing the parking lot. The functions of the parking system 400 may be realized through the cooperation of the processor 420 that executes the management program 440 and the storage device 430. The management program 440 is stored in the storage device 430. The management program 440 may be recorded on a computer-readable recording medium.

[0100] The processor 420 communicates with the vehicle 100 and the backend system 300 via the communication device 410.

[0101] Furthermore, the storage device 430 stores management information 450 for managing the parking lot. The management information 450 includes map information of the parking lot. The processor 420 may provide the map information to the vehicle 100 via the communication device 410. The management information 450 also indicates the usage status (availability) of parking spaces within the parking lot. Based on the management information 450, the processor 420 can assign an available parking space (destination) to the vehicle 100.

[0102] The management information 450 may include vehicle management information. The vehicle management information includes location information for each vehicle 100 in the parking lot. The processor 420 may communicate with each vehicle 100 via the communication device 410 and collect location information from each vehicle 100. Alternatively, the processor 420 may acquire images taken by infrastructure cameras CAM installed in the parking lot and estimate the location of each vehicle 100 based on those images. The vehicle management information may include a travel route TP assigned to each vehicle 100. The processor 420 can determine the travel route TP assigned to each vehicle 100 based on the location information, destination, and map information of the vehicle 100. The processor 420 may provide the travel route TP information to the vehicle 100 via the communication device 410.

[0103] The processor 420 may receive request information REQ from the user terminal 200 via the backend system 300 and the communication device 410. The request information REQ is stored in the storage device 430.

[0104] In response to the request information REQ, the processor 420 obtains the first reaction information RAX1 and the second reaction information RAX2. The first reaction information RAX1 instructs the vehicle 100 to perform the first reaction. The first reaction information RAX1 may include the content of the first reaction. The second reaction information RAX2 instructs the terminal (at least one of the user terminal 200 and the infrastructure terminal 500) to perform the second reaction. The second reaction information RAX2 may include the content of the second reaction. The second reaction information RAX2 may include the supplementary information described in Section 4 above. The processor 420 transmits the first reaction information RAX1 to the vehicle 100 via the communication device 410. The processor 420 also transmits the second reaction information RAX2 to the user terminal 200 via the communication device 410 and the backend system 300. The processor 420 may send the second reaction information RAX2 to the infrastructure terminal 500. [Explanation of symbols]

[0105] 10. Automatic Valet Parking (AVP) System 100 vehicles 200 user terminals 300 backend systems 400 Parking System 500 infrastructure terminals RAX1 First Reaction Information RAX2 2nd Reaction Information

Claims

1. A management system for managing automated valet parking of vehicles in a parking lot, Equipped with one or more processors, The terminal includes at least one of the user terminal of the vehicle's user and the infrastructure terminal installed in the parking lot. The one or more processors described above are: The user terminal receives a request regarding the automated valet parking of the vehicle. In response to the aforementioned request, the vehicle is instructed to perform a first reaction, and the terminal is instructed to perform a second reaction. It is configured in such a way Management system.

2. A management system according to claim 1, The first reaction includes at least one of the following: flashing the vehicle's lights, activating the vehicle's wipers, opening and closing the vehicle's door mirrors, and sounding the vehicle's horn. The second reaction includes at least one of the following: flashing the light of the terminal, vibrating the terminal, and emitting sound from the terminal. Management system.

3. A management system according to claim 1, The first reaction includes flashing the lights of the vehicle, The second reaction includes flashing the light on the terminal. Management system.

4. A management system according to claim 1, The first reaction includes turning the vehicle's lights ON / OFF in a predetermined pattern. The second reaction includes at least one of the following: turning the terminal's light ON / OFF in the predetermined pattern, turning the terminal's vibration ON / OFF in the predetermined pattern, and turning the sound output from the terminal ON / OFF in the predetermined pattern. Management system.

5. A management system according to claim 1, The first reaction includes turning the vehicle's lights ON / OFF in a predetermined pattern. The second reaction includes turning the terminal's light ON / OFF in the predetermined pattern. Management system.

6. A management system according to claim 1, The second reaction includes displaying an animation on the terminal screen representing the vehicle performing the first reaction. Management system.

7. A management system according to claim 1, The second reaction includes displaying an animation on the terminal's screen that represents the interaction between the vehicle and the terminal. Management system.

8. A management system according to any one of claims 1 to 7, The second reaction includes displaying information on the terminal screen that explains the meaning of the first reaction of the vehicle. Management system.

9. A management system according to any one of claims 1 to 7, The aforementioned request is to initiate the automated valet parking of the vehicle. Management system.

10. Vehicles that are eligible for automated valet parking in a parking lot, A terminal including at least one of the user terminal of the vehicle user and the infrastructure terminal installed in the parking lot. Includes, In response to the user's request regarding the automatic valet parking of the vehicle, the vehicle performs a first reaction, and the terminal performs a second reaction. Automatic valet parking system.

Images