Management System
The management system increases the likelihood of identifying target vehicles by having them perform multiple actions in a set order, using sensors to ensure accurate recognition despite environmental conditions.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- TOYOTA JIDOSHA KK
- Filing Date
- 2024-11-07
- Publication Date
- 2026-05-19
AI Technical Summary
Existing vehicle identification technologies face challenges in accurately recognizing vehicle actions due to environmental conditions, such as lighting issues leading to whiteout in camera images, which can hinder the correct identification of target vehicles.
A management system that instructs target vehicles to perform multiple types of actions in a predetermined order, using sensors to recognize these actions and identify the vehicle based on successful performance of at least one action within a determination period.
Enhances the probability of identifying target vehicles by ensuring at least one action is recognizable despite environmental challenges, thereby improving accuracy and reliability.
Smart Images
Figure 2026082408000001_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to a technology for managing vehicles in a predetermined area.
Background Art
[0002] Patent Document 1 discloses a technology related to Automated Valet Parking (AVP) in a parking lot. The target vehicle is a vehicle that uses AVP in the parking lot. The system transmits information instructing the execution of a predetermined action to the target vehicle. The predetermined action is, for example, the blinking of the front light. The system uses a camera installed at the drop-off position in the parking lot to check whether the vehicle parked at the drop-off position has executed the predetermined action. The system identifies the vehicle that has executed the predetermined action as the target vehicle.
Prior Art Documents
Patent Documents
[0003]
Patent Document 1
Summary of the Invention
Problems to be Solved by the Invention
[0004] Consider a vehicle identification technology that identifies a vehicle that has executed a predetermined action in a predetermined area as the target vehicle. Regarding such a vehicle identification technology, the inventor of the present application has recognized the following problem. That is, depending on the environment, there may be cases where the action performed by the vehicle cannot be recognized well. For example, when the action is the blinking of a light, depending on the lighting conditions, there is a possibility that "whiteout" occurs in the image captured by the camera. If whiteout occurs in the image, even if the vehicle blinks the light, it may not be correctly detected. If the action performed by the vehicle cannot be recognized well, the target vehicle cannot be identified.
[0005] One objective of this disclosure is to provide a technology that can increase the probability of identifying a target vehicle. [Means for solving the problem]
[0006] One aspect of this disclosure relates to a management system for managing vehicles in a designated area. The management system comprises one or more processors. One or more processors instruct the target vehicle to perform multiple types of actions in a predetermined order. One or more processors recognize actions performed by vehicles within a predetermined area by using sensors installed in that area. One or more processors identify a vehicle as a target vehicle if it has performed one of several types of actions within a determination period following an instruction to that vehicle. [Effects of the Invention]
[0007] According to this disclosure, the target vehicle is instructed to perform multiple types of actions in a predetermined order. The vehicle that performs any of these multiple types of actions is then identified as the target vehicle. Even if one action is difficult to recognize in a given environment, another action is likely to be easily recognized. In other words, in any environment, it is highly likely that at least one of the multiple types of actions will be recognized. Therefore, the probability of identifying the target vehicle increases. [Brief explanation of the drawing]
[0008] [Figure 1] This is a conceptual diagram illustrating an example of vehicle control within a designated area. [Figure 2] This is a conceptual diagram to explain the basic vehicle identification process. [Figure 3] This is a conceptual diagram illustrating the overview of vehicle identification processing based on multiple types of actions. [Figure 4]This is a conceptual diagram illustrating one example of how to determine multiple types of actions. [Figure 5] This is a conceptual diagram illustrating an example of the sequence of multiple types of actions. [Figure 6] This is a block diagram showing an example of a management system configuration. [Modes for carrying out the invention]
[0009] Embodiments of this disclosure will be described with reference to the attached drawings.
[0010] 1. Vehicle control in a designated area Let's consider the control of Vehicle 1 in a designated area of augmented reality (AR). Examples of a designated area of augmented reality include parking lots, factories, facility grounds, and even a small city (smart city). In the designated area of augmented reality, Vehicle 1 is controlled to travel to a set destination. Vehicle 1 may be an autonomous vehicle.
[0011] Figure 1 is a conceptual diagram illustrating an example of vehicle 1 control in a predetermined area AR. In the example shown in Figure 1, the predetermined area AR is a parking lot PL. This parking lot PL provides an automated valet parking (AVP) service. Vehicle 1 is equipped with an automated valet parking function and can drive autonomously at least within the parking lot PL.
[0012] The in-vehicle system 100 is mounted on vehicle 1 and controls vehicle 1. Specifically, the in-vehicle system 100 recognizes the surrounding environment of vehicle 1 using recognition sensors (e.g., cameras) mounted on vehicle 1. The in-vehicle system 100 safely drives vehicle 1 while recognizing the surrounding environment of vehicle 1. Multiple markers M (landmarks) may be placed within the parking lot PL. The markers M are used to guide vehicle 1 within the parking lot PL. For example, the in-vehicle system 100 acquires images of the surrounding environment using a camera and recognizes the markers M based on the images. Then, based on the recognition results of the markers M, the in-vehicle system 100 performs localization processing to estimate the position of vehicle 1 in the parking lot PL with high accuracy. Based on the estimated vehicle position, the in-vehicle system 100 automatically drives vehicle 1 within the parking lot PL.
[0013] The management system 200 is a system that manages the parking area PL (designated area AR) and automated valet parking, and is located outside of the vehicle 1. The management system 200 can communicate with each vehicle 1 within the parking area PL. For example, the management system 200 communicates with each vehicle 1 within the parking area PL via wireless LAN. The management system 200 may also remotely control each vehicle 1 within the parking area PL.
[0014] One or more infrastructure cameras (CAMs) may be installed within the parking area (PL). The infrastructure cameras (CAMs) photograph the parking area (PL) and acquire images showing the conditions of the parking area (PL). The management system 200 communicates with the infrastructure cameras (CAMs) and acquires the images captured by the infrastructure cameras (CAMs). The management system 200 detects vehicle 1 in the images by analyzing them. The management system 200 also estimates the position of vehicle 1 in the images. Furthermore, the management system 200 manages vehicle 1 within the parking area (PL) based on its position. The management system 200 may provide vehicle 1 with its position information. The onboard system 100 of vehicle 1 may automatically drive vehicle 1 within the parking area (PL) based on the position information provided by the management system 200.
[0015] The warehousing process is as follows. Vehicle 1 stops in the warehousing area. The management system 200 assigns an empty parking space to Vehicle 1. The assigned empty parking space becomes the target parking space, i.e., the destination, for Vehicle 1 upon warehousing. Furthermore, the management system 200 sets a target trajectory (travel route TP) from the warehousing area to the target parking space in the parking lot PL. The in-vehicle system 100 acquires information on the target trajectory to the target parking space. The management system 200 issues a warehousing instruction to the in-vehicle system 100. In response to the warehousing instruction, the in-vehicle system 100 drives Vehicle 1 along the target trajectory to the target parking space. That is, the in-vehicle system 100 controls Vehicle 1 to follow the target trajectory based on the vehicle position. Then, the in-vehicle system 100 parks Vehicle 1 in the target parking space.
[0016] The出库process is as follows. Upon出库, the designated出库area becomes the destination for Vehicle 1. The management system 200 sets a target trajectory (travel route TP) from the parking space in the parking lot PL to the出库area. The in-vehicle system 100 acquires information on the target trajectory to the出库area. The management system 200 issues an出库instruction to the in-vehicle system 100. In response to the出库instruction, the in-vehicle system 100 drives Vehicle 1 along the target trajectory to the出库area. That is, the in-vehicle system 100 controls Vehicle 1 to follow the target trajectory based on the vehicle position. Then, the in-vehicle system 100 stops Vehicle 1 in the出库area.
[0017] 2. Vehicle Identification Process Consider identifying a specific Vehicle 1 in a predetermined area AR. The Vehicle 1 to be identified is hereinafter referred to as "target Vehicle 1T". Also, the process of identifying the target Vehicle 1T in the predetermined area AR is hereinafter referred to as the "vehicle identification process".
[0018] For example, when the predetermined area AR is a parking lot PL that provides an AVP service as shown in FIG. 1, the target vehicle 1T is an incoming vehicle that attempts to enter the parking lot using the AVP service. For example, when the first vehicle of the first user attempts to enter the parking lot using the AVP service, that first vehicle is the target vehicle 1T. More specifically, the first vehicle of the first user parks in the incoming area, and the first user gets out of the first vehicle. The operation authority of the first vehicle is transferred from the first user to the management system 200 (referred to as handover). The management system 200 communicates with the first vehicle and starts the first vehicle. Here, when starting the AVP for the first vehicle, it is desirable for the management system 200 to accurately recognize which vehicle 1 in the incoming area is the first vehicle. That is, it is desirable for the management system 200 to identify the first vehicle (target vehicle 1T). By identifying the first vehicle, it becomes possible to accurately recognize the position of the first vehicle, that is, the starting position of autonomous driving. Also, it becomes possible to prevent a vehicle 1 that is not the first vehicle from mistakenly entering the parking lot.
[0019] FIG. 2 is a conceptual diagram for explaining basic vehicle identification processing. In the present embodiment, the "action" executed by the target vehicle 1T is used for vehicle identification processing.
[0020] An action is defined by a combination of the device that executes the action and the operation pattern of that device. Examples of the device that executes the action include lights, wipers, actuators, engines, horns, etc. Examples of lights include headlights, brake lights, fog lights, etc. Examples of actuators include a door actuator for automatically opening and closing a door, a window actuator for automatically opening and closing a window, a mirror actuator for automatically opening and closing a door mirror, a bonnet actuator for automatically opening and closing a bonnet, etc.
[0021] Examples of visible actions include turning lights on or on, flashing turn signals, operating wipers, opening and closing doors, opening and closing windows, opening and closing side mirrors, opening and closing the hood, etc. Examples of audible actions include sounding the horn, starting the engine, etc. For example, the headlights or turn signals flash in a predetermined pattern for a predetermined period (e.g., a few seconds). Another example is that the side mirrors open and close in a predetermined pattern for a predetermined period. Yet another example is that the horn sounds in a predetermined pattern for a predetermined period. The predetermined pattern of actions may be repeated over time.
[0022] The management system 200 communicates with the target vehicle 1T within a predetermined area AR. The management system 200 then instructs the target vehicle 1T to perform a "specified action." More specifically, the management system 200 sends instruction information INS to the target vehicle 1T, instructing it to perform the specified action. For example, the instruction information INS contains information about the content and pattern of the specified action assigned to the target vehicle 1T. Different actions may be assigned as specified actions for each target vehicle 1T (user). As another example, the content and pattern of the specified action may be predetermined and shared in advance by the target vehicle 1T.
[0023] The onboard system 100 of the target vehicle 1T receives instruction information INS from the management system 200, which instructs it to perform a specified action. In response to the instruction information INS, the onboard system 100 of the target vehicle 1T performs the specified action.
[0024] One or more sensors 10 are placed in the designated AR area to recognize (detect) actions. For example, sensor 10 includes a camera to recognize (detect) visible actions. As another example, sensor 10 may include a microphone to recognize (detect) audible actions.
[0025] The management system 200 can recognize (detect) actions performed by vehicle 1 within a predetermined area AR by using sensor 10. At this stage, since the target vehicle 1T has not yet been identified, the management system 200 recognizes (detects) actions performed by any vehicle 1 within the predetermined area AR. More specifically, the management system 200 acquires sensor detection information SEN detected by sensor 10. Then, based on the sensor detection information SEN, the management system 200 recognizes actions performed by vehicle 1 within the predetermined area AR. For example, sensor detection information SEN may include images captured by a camera. In this case, the management system 200 can recognize visible actions performed by vehicle 1 based on the images. As another example, sensor detection information SEN may include audio information detected by a microphone. In this case, the management system 200 can recognize audible actions performed by vehicle 1 based on the audio information.
[0026] After instructing target vehicle 1T to perform a specified action, a predetermined judgment period is established. The predetermined judgment period is set considering factors such as communication delay and the duration of the specified action. During the predetermined judgment period, the management system 200 determines, based on sensor detection information SEN, whether any vehicle 1 within the predetermined area AR has performed the specified action. If it is recognized (detected) that a vehicle 1 has performed the specified action within the predetermined judgment period, the management system 200 identifies that vehicle 1 as target vehicle 1T. In other words, the management system 200 identifies a vehicle 1 that has performed the specified action within the predetermined judgment period as target vehicle 1T.
[0027] 3. Vehicle identification process based on multiple types of actions Depending on the environment, it may not be possible to properly recognize the actions performed by vehicle 1. For example, if the action is flashing lights, depending on the lighting, the image captured by the camera may be overexposed. If overexposure occurs in the image, it may not be possible to correctly detect that vehicle 1 is flashing its lights. If the actions performed by the vehicle cannot be properly recognized, it will not be possible to identify the target vehicle 1T.
[0028] Therefore, this embodiment proposes a technique that can increase the probability of identifying the target vehicle 1T.
[0029] 3-1. Overview Figure 3 is a conceptual diagram illustrating the outline of the vehicle identification process according to this embodiment. According to this embodiment, the management system 200 instructs the target vehicle 1T to perform multiple types of actions in a predetermined order. For example, the instruction information INS sent from the management system 200 to the target vehicle 1T includes information specifying the content of the multiple types of actions and their order. The management system 200 uses sensors 10 installed in a predetermined area AR to recognize the actions to be performed by vehicles 1 within the predetermined area AR. The management system 200 determines whether any of the vehicles 1 within the predetermined area AR have performed any of the multiple types of actions. The management system 200 then identifies the vehicle 1 that performed any of the multiple types of actions within the determination period after the instruction to the target vehicle 1T as the target vehicle 1T.
[0030] For simplicity, let's consider the case where multiple types of actions include a first action and a second action. The same applies when the number of multiple types of actions is three or more. The types of the first action and the second action are different from each other. Also, the first action is executed first, and the second action is executed after the first action. In other words, the management system 200 instructs the target vehicle 1T to execute the second action after the first action.
[0031] The first determination period (t0~t1) is the determination period immediately following an instruction being given to target vehicle 1T, and is the period for recognizing the first action. The first determination period is set considering communication delays, the duration of the first action, etc. During the first determination period, the management system 200 determines, based on sensor detection information SEN, whether any vehicle 1 within a predetermined area AR has performed the first action. If it is recognized (detected) that a vehicle 1 within the first determination period has performed the first action, the management system 200 identifies that vehicle 1 as target vehicle 1T. In other words, the management system 200 identifies a vehicle 1 that performed the first action within the first determination period as target vehicle 1T. If the identification of target vehicle 1T is successful, target vehicle 1T does not need to perform the subsequent second action. Therefore, the management system 200 may notify target vehicle 1T that no further action is required. In other words, if the identification of target vehicle 1T is successful, the management system 200 may instruct target vehicle 1T to stop performing the specified action. This prevents unnecessary actions from being taken.
[0032] On the other hand, if the system fails to recognize a vehicle 1 that performs the first action within the first determination period, the management system 200 performs the following process. The second determination period (t1~t2) is a determination period that follows the first determination period and is a period for recognizing the second action. The second determination period is set considering the duration of the second action, etc. During the second determination period, the management system 200 determines, based on the sensor detection information SEN, whether any vehicle 1 within the predetermined area AR has performed the second action. If it is recognized (detected) that a vehicle 1 performing the second action is within the second determination period, the management system 200 identifies that vehicle 1 as target vehicle 1T. In other words, the management system 200 identifies a vehicle 1 that performed the second action within the second determination period as target vehicle 1T.
[0033] As described above, according to this embodiment, the target vehicle 1T is instructed to perform multiple types of actions in a predetermined order. Then, the vehicle 1 that performs any of the multiple types of actions is identified as the target vehicle 1T. Even if one action is difficult to recognize in a given environment, another action is likely to be easily recognized. In other words, in any environment, it is highly likely that at least one of the multiple types of actions will be recognized. Therefore, the probability of being able to identify the target vehicle increases.
[0034] 3-2. Examples of methods for determining multiple types of actions Multiple types of actions may be predetermined by the system designer. Alternatively, multiple types of actions may be flexibly determined considering the equipment of the target vehicle 1T.
[0035] Figure 4 is a conceptual diagram illustrating an example of how multiple types of actions are determined. The onboard system 100 of the target vehicle 1T provides the management system 200 with vehicle information VCL related to the target vehicle 1T. The vehicle information VCL includes vehicle equipment information indicating the equipment of the target vehicle 1T. The equipment includes the lights and actuators mentioned above. The vehicle equipment information indicates the types of lights and actuators that the target vehicle 1T is equipped with. Based on the vehicle equipment information of the target vehicle 1T, the management system 200 recognizes which actions can and cannot be performed on the target vehicle 1T. The management system 200 then excludes the unexecutable actions from the list of candidates for multiple types of actions and constructs the list of multiple types of actions using only the executable actions.
[0036] 3-3. Examples of the order of multiple types of actions Figure 5 is a conceptual diagram illustrating an example of the sequence of multiple types of actions.
[0037] In the first example, the order of multiple types of actions is set from the perspective of the time required. More specifically, the first time required to perform the first action is shorter than the second time required to perform the second action. For example, since the time required to open and close the door mirrors is longer than the time required to flash the lights, flashing the lights is set as the first action and opening and closing the door mirrors is set as the second action. In another example, since the time required to open and close the doors is longer than the time required to open and close the door mirrors, opening and closing the door mirrors is set as the first action and opening and closing the doors is set as the second action. In the first example, since actions are executed in order from the shortest time required, it is expected that the total time required to identify the target vehicle 1T will be shortened.
[0038] In the second example, the order of multiple actions is determined from the perspective of power consumption. More specifically, the power consumption required for the first action is lower than the power consumption required for the second action. For example, since the power consumption required for opening and closing the door mirrors is higher than the power consumption required for flashing the lights, flashing the lights is set as the first action and opening and closing the door mirrors is set as the second action. As another example, since the power consumption required for opening and closing the doors is higher than the power consumption required for opening and closing the door mirrors, opening and closing the door mirrors is set as the first action and opening and closing the doors is set as the second action. In the second example, since actions are executed in order from the lowest power consumption, it is expected that the total power consumption required to identify the target vehicle 1T will be reduced.
[0039] In the third example, the order of multiple actions is determined from the perspective of changes in vehicle size. Vehicle size refers to the external size of vehicle 1. More specifically, the change in vehicle size caused by the execution of the first action is smaller than the change in vehicle size caused by the execution of the second action. For example, since the change in vehicle size caused by opening and closing the door mirrors is larger than the change in vehicle size caused by flashing the lights, flashing the lights is set as the first action and opening and closing the door mirrors is set as the second action. As another example, since the change in vehicle size caused by opening and closing the doors is larger than the change in vehicle size caused by opening and closing the door mirrors, opening and closing the door mirrors is set as the first action and opening and closing the doors is set as the second action. In the third example, actions are executed in order from those that cause the smallest change in vehicle size, thus minimizing the impact on the surroundings of the target vehicle 1T.
[0040] 3-4. Effects As described above, according to this embodiment, the target vehicle 1T is instructed to perform multiple types of actions in a predetermined order. Then, the vehicle 1 that performs any of the multiple types of actions is identified as the target vehicle 1T. Even if one action is difficult to recognize in a given environment, another action is likely to be easily recognized. In other words, in any environment, it is highly likely that at least one of the multiple types of actions will be recognized. Therefore, the probability of being able to identify the target vehicle increases.
[0041] 4. Example of a Management System Configuration Figure 6 is a block diagram showing an example configuration of the management system 200 according to this embodiment. The management system 200 includes a communication device 210, one or more processors 220 (hereinafter simply referred to as processor 220), and one or more storage devices 230 (hereinafter simply referred to as storage devices 230).
[0042] The communication device 210 communicates with the in-vehicle system 100 of each vehicle 1. The communication device 210 also communicates with the sensor 10 installed in a predetermined area AR. Furthermore, the communication device 210 may also communicate with the infrastructure camera CAM installed in a predetermined area AR.
[0043] The processor 220 performs various processes. Examples of the processor 220 include general-purpose processors, application-specific processors, CPUs (Central Processing Units), GPUs (Graphics Processing Units), ASICs (Application Specific Integrated Circuits), FPGAs (Field-Programmable Gate Arrays), integrated circuits, and / or combinations thereof. The processor 220 can also be called processing circuitry. The storage device 230 stores various information. Examples of storage devices 230 include volatile memory, non-volatile memory, HDDs (Hard Disk Drives), SSDs (Solid State Drives), etc.
[0044] The management program 240 is a computer program for managing a predetermined area AR. The functions of the management system 200 may be realized through the cooperation of the processor 220 that executes the management program 240 and the storage device 230. The management program 240 is stored in the storage device 230. The management program 240 may be recorded on a computer-readable recording medium.
[0045] Management information 250 is information for managing a predetermined area AR. Management information 250 is stored in the storage device 230. For example, management information 250 includes map information of the predetermined area AR. If the predetermined area AR is a parking lot PL, management information 250 may also include usage status information indicating the usage status (availability) of parking spaces within the parking lot PL. Based on the management information 250, the processor 220 can assign an available parking space (destination) to the target vehicle 1T.
[0046] As yet another example, the management information 250 may include vehicle management information for managing each vehicle 1 within a predetermined area AR. The vehicle management information includes the location of each vehicle 1 within the predetermined area AR. The processor 220 may communicate with each vehicle 1 via the communication device 210 and collect location information from each vehicle 1. Alternatively, the processor 220 may acquire images captured by infrastructure cameras CAM installed in the predetermined area AR and estimate the location of each vehicle 1 based on those images. The vehicle management information may include a travel route TP assigned to each vehicle 1. The processor 220 can determine the travel route TP assigned to each vehicle 1 based on the location information, destination, and map information of each vehicle 1.
[0047] One or more sensors 10 are placed in a designated area AR to recognize (detect) actions. The processor 220 communicates with each sensor 10 via the communication device 210 and acquires sensor detection information SEN detected by each sensor 10. The sensor detection information SEN is stored in the storage device 230.
[0048] The processor 220 may communicate with each vehicle 1 via the communication device 210 and collect vehicle information VCL from each vehicle 1. The vehicle information VCL includes vehicle equipment information indicating the equipment of vehicle 1. The vehicle information VCL is stored in the storage device 230. The processor 220 may determine multiple types of actions to assign to target vehicle 1T based on the vehicle equipment information of the target vehicle 1T.
[0049] Furthermore, the processor 220 generates instruction information INS for the target vehicle 1T. The instruction information INS includes information specifying the content and order of multiple types of actions to be assigned to the target vehicle 1T. The processor 220 communicates with the target vehicle 1T via the communication device 210 and transmits the instruction information INS to the target vehicle 1T. Based on the sensor detection information SEN, the processor 220 determines whether any vehicle 1 within a predetermined area AR has performed one of the multiple types of actions. The processor 220 identifies the vehicle 1 that performed one of the multiple types of actions within the determination period after the instruction to the target vehicle 1T as the target vehicle 1T. [Explanation of symbols]
[0050] 1 vehicle 10 sensors 100 In-vehicle systems 200 Management Systems
Claims
1. A management system for managing vehicles in a designated area, Equipped with one or more processors, The one or more processors described above are: The system instructs the target vehicle to perform multiple types of actions in a predetermined order. By using sensors installed in the predetermined area, the system recognizes the actions performed by vehicles within the predetermined area. A vehicle that performs any of the above-mentioned multiple types of actions within the judgment period following the instruction given to the aforementioned target vehicle is identified as the aforementioned target vehicle. It is configured in such a way Management system.
2. A management system according to claim 1, The aforementioned multiple types of actions include a first action and a second action that is different from the first action. The one or more processors are configured to instruct the target vehicle to perform the second action after the first action. Management system.
3. A management system according to claim 2, The first time required to perform the first action is shorter than the second time required to perform the second action. Management system.
4. A management system according to claim 2, The first power consumption required to perform the first action is lower than the second power consumption required to perform the second action. Management system.
5. A management system according to claim 2, The change in vehicle size resulting from the execution of the first action is smaller than the change in vehicle size resulting from the execution of the second action. Management system.