Parking lot management device

Abstract

To provide a parking lot management apparatus that is less affected by restrictions on the position and quantity of cameras.SOLUTION: A parking lot management apparatus comprises: an imaging apparatus for acquiring image information of a parking lot; a vehicle identification apparatus for identifying a vehicle included in the image information including vehicle exterior shape information that can identify the vehicle and acquired by the imaging apparatus; and an information processing apparatus for setting coordinates to an area of a parking slot of the parking lot included in the image information, predicting that the vehicle is using the parking slot in the image information based on information on the coordinates and the vehicle exterior shape information, and determining that the vehicle is using the parking slot according to the reliability of the prediction.SELECTED DRAWING: Figure 14

Description

[Technical Field]

[0001] The present application relates to a parking lot management device. [Background technology]

[0002] Deep learning, an artificial intelligence (AI) technology, is being used to extract predetermined objects from two-dimensional images. For example, it has been proposed to use AI to determine whether a vehicle is parked in a parking space based on video footage of a parking lot captured by a surveillance camera.

[0003] As a system for detecting the usage status of parking spaces in a parking lot, it has been proposed to detect vacant parking spaces by detecting images of parking spaces from images taken by a surveillance video camera in the parking lot and determining whether the images of the parking spaces satisfy the characteristics of vacant parking spaces (Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-84364 Summary of the Invention [Problem to be solved by the invention]

[0005] As shown in Patent Document 1, in order to determine the state of an object (empty space) by predetermining the object from an image, images taken by a camera corresponding to each parking space are required. Therefore, it is necessary to install the camera in a high position or to provide a large number of cameras.

[0006] The present application aims to provide a parking lot management device that requires cameras but is less affected by restrictions on the location and number of cameras. [Means for solving the problem]

[0007] The parking lot management device of the present application includes an imaging device for acquiring image information of the parking lot. , vehicle Deep learning is used to learn the external shape in advance. storing identifiable features of the vehicle; The image information acquired by the imaging device is divided into a plurality of regions, and the image information stored in each of the divided regions is By determining whether the image information includes a characteristic part of the vehicle, The aforementioned a vehicle identification device that identifies the inclusion of a vehicle, and a parking space of the parking lot included in the image information; Represents Set the coordinates and the information of the coordinates , the identified vehicle Information on the external shape of the Deep learning is performed on data related to the parking space, including at least the installation height and installation angle of the imaging device; The parking space in the image information The aforementioned Predicting whether a vehicle is using the parking space and reserving the parking space according to the reliability of the prediction. The aforementioned The vehicle is characterized by being equipped with an information processing device that determines whether the vehicle is in use. [Effects of the Invention]

[0008] According to the parking lot management device of the present invention, it is possible to obtain a parking lot management device that is not subject to many restrictions on the position and number of cameras. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a schematic configuration diagram showing an object identification device according to a first embodiment. [Figure 2] FIG. 2 is a diagram illustrating a hardware configuration for realizing the block diagram according to the embodiment. [Figure 3] FIG. 10 is an explanatory diagram showing a flow of the second embodiment. [Figure 4] FIG. 10 is an explanatory diagram showing the relationship between an obstruction and an object to be identified. [Figure 5] FIG. 10 is a flowchart showing a process for counting the number of vehicles when the presence of the vehicles is unknown due to an obstruction. [Figure 6] FIG. 11 is an explanatory diagram showing the flow of the parking lot management device of the third embodiment. [Figure 7]FIG. 10 is an explanatory diagram showing the flow of the parking lot management device of the fourth embodiment. [Figure 8] FIG. 10 is an explanatory diagram showing the flow of the parking lot management device of the fifth embodiment. [Figure 9] FIG. 13 is an explanatory diagram showing the flow of the parking lot management device of the seventh embodiment. [Figure 10] FIG. 13 is an explanatory diagram showing the flow of the parking lot management device of the eighth embodiment. [Figure 11] FIG. 13 is an explanatory diagram showing the flow of the parking lot management device of the ninth embodiment. [Figure 12] FIG. 20 is an explanatory diagram showing the flow of the parking lot management device of the tenth embodiment. [Figure 13] FIG. 20 is an explanatory diagram showing the flow of the parking lot management device of the eleventh embodiment. [Figure 14] FIG. 22 is an explanatory diagram showing the flow of the parking lot management device of the twelfth embodiment. [Figure 15] FIG. 23 is an explanatory diagram showing the flow of the vehicle control device of the thirteenth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] The following describes embodiments of an object identification device, a parking lot management device, and a vehicle management device using the same according to the present application, with reference to the drawings. Note that identical or corresponding parts in each drawing are designated by the same reference numerals, and redundant explanations will be omitted.

[0011] Embodiment 1 FIG. 1 is a schematic diagram showing the configuration of an object identification device according to a first embodiment of the present invention. As shown in FIG. 1, the object identification device 100 of the first embodiment includes an imaging device 1 that acquires image information, a first storage device 2 that stores information on characteristic parts of an object to be identified, a second storage device 3 that accumulates the image information acquired by the imaging device 1, and an information processing device 4 that divides the image information of the second storage device 3 into a plurality of regions and determines whether the image information includes an object to be identified based on the image information of each divided region and the information on characteristic parts of the object to be identified stored in the first storage device 2.

[0012] Based on the determination result by the information processing device 4, the identification device 5 identifies whether the subject captured by the imaging device 1 is an object to be identified. The identification result by the identification device 5 is displayed on the display device 6. The object to be identified is set in an identification object setting unit 7. The characteristic parts of the object to be identified stored in the first storage device 2 are generated by image recognition using a deep learning technique using training data of the object to be identified. That is, the information on the characteristic parts stored in the first storage device 2 is obtained by learning through comparison between the training data and similar object data. For the object to be identified that is set in advance in the identification object setting unit 7, training data to be used for deep learning is created in a training data creation unit 8, and similar object data is created in a similar object data creation unit 9.

[0013] The training data is information on the external shape of the object to be identified, obtained by changing the drawing parameters of the front view, side view, and three-dimensional structure images of the object to be identified. The similar object data is information on the external shape of a different structure that is similar to the external shape of the object to be identified. The training data created by the training data creation unit 8 and the similar object data created by the similar object data creation unit 9 are used to learn in the learning processing device 10 how the object to be identified is similar to other similar objects and what the distinguishing feature parts are. The information on the feature parts of the object to be identified learned by the learning processing device 10 is stored in the first storage device 2.

[0014] That is, the learning processing means 11 includes a recognition target object setting unit 7, a teacher data creation unit 8, an analog data creation unit 9, and a learning processing device 10. The object recognition means 12 includes a first storage device 2, a second storage device 3, an information processing device 4, and a recognition device 5.

[0015] The learning processing means 11 and the object identification means 12 are configured with a processor 200 and a storage device 201, as shown in FIG. 2, which is an example of hardware. Although the storage device is not shown, it includes a volatile storage device such as a random access memory and a non-volatile auxiliary storage device such as a flash memory. Also, instead of the flash memory, a hard disk auxiliary storage device may be provided. The processor 200 executes a program input from the storage device 201. In this case, the program is input to the processor 200 from the auxiliary storage device via the volatile storage device. Also, the processor 200 may output data such as calculation results to the volatile storage device of the storage device 201, or may store the data in the auxiliary storage device via the volatile storage device.

[0016] Here, the identification target refers to an object to be identified in an image acquired by an imaging device 1 such as a camera. There are no particular limitations on the identification target, and it can be set appropriately depending on the purpose. For example, it can be an object that can be detected by human vision, such as a small vehicle, a large vehicle, a specific model of vehicle, a license plate with a specific local name, people entering a facility, men and women, adults, children, or animals, which are included in an image (video) of a parking lot.

[0017] Training data is a pair of "input data" and "correct label" used in supervised deep learning. Here, deep learning is performed by inputting the input data into a neural network with many parameters, and the frequency of correct answers is increased by repeatedly updating the difference between the inferred label and the correct label. In addition, here, the inclusion of an object to be identified in an image is identified by recognizing part of the object's appearance, rather than making a judgment based on the entire image of the object to be identified.

[0018] To achieve this, the image information acquired by the imaging device 1 is divided into multiple regions, and it is determined whether or not the image information of each of the multiple divided regions contains a characteristic part of the object to be identified. This performs the same function as when, for example, a manager visually checks the number of small vehicles in a parking lot, and counts the small vehicles by checking only the parts of the vehicle bodies that are only slightly visible, using image recognition. Image recognition is a technology that analyzes the image content of image data to recognize its shape and appearance. In this case, image recognition involves extracting the outline of an object to be identified from the image data, separating it from the background of the image, and then analyzing what the object is.

[0019] Next, detection of an object to be identified using deep learning will be described. The algorithm used here uses a one-stage object detection method. The steps for detecting objects using this algorithm are as follows: First, an image captured by the image capture device 1 is divided into a grid of a predetermined size (for example, grid regions of equal size). A predetermined number of bounding boxes of a predefined shape centered at the center of the grid are predicted within the divided grid regions. Each prediction is associated with a class probability and object confidence (whether the region contains an object or is only background). Next, the bounding box associated with the highest object confidence and class probability is selected. The object class with the highest class probability becomes the object's category (class). Then, only default boxes with high scores are extracted. Next, the overlap rate of the default boxes for each class is calculated, and if the overlap rate is high, default boxes with low scores are removed. Finally, a bounding box representing the object's position is output.

[0020] Embodiment 2 Next, a case where the object identification device is applied to a parking lot management device will be described. FIG. 3 is an explanatory diagram showing the configuration and flow of a parking lot management device using an object identification device. As shown in Fig. 3, when used in a parking lot management system, multiple types of vehicles are set as identification targets. For example, passenger cars, trucks, buses, motorbikes, bicycles, etc. are set as identification targets. Then, an imaging device 1 (camera) takes images of the parking lot at predetermined time intervals (step S200). An edge computer 202 performs the functions of the learning processing means 11 and the object identification means 12 shown in Fig. 1.

[0021] That is, a camera image is acquired (step S210), and bounding boxes are predicted for object detection (step S220). Image recognition is performed, and objects (passenger cars, trucks, buses, motorbikes, bicycles) with a high degree of identification reliability are selected (step S230). Objects with a reliability higher than a preset threshold are selected as processing targets (step S240). The number of objects to be identified as processing targets is then counted (step S250). The counting of the number of vehicles here is performed by the object identification device, as described in the first embodiment, by dividing the image information stored in the second storage device into multiple regions and determining whether the image information contains the object to be identified based on the image information of each divided region and information on the characteristic parts of the object to be identified stored in the first storage device.

[0022] The status of the parking lot is determined as full, empty, or crowded (step S260). The determination result is displayed on a display panel as full, empty, or crowded (step S270). In addition, the cloud server displays full, empty, or crowded on a homepage (step S280). In addition, the number of vehicles counted is tallied by vehicle type and time period (step S290). In step S240, it is shown that the object with the highest degree of identification reliability is the object to be processed. This is because, just as in a situation where a manager is standing at a predetermined location (monitoring location) in the parking lot and visually observing the parking lot, there may be cases where the object cannot be confirmed at all due to the presence of another vehicle between the manager and the object. In other words, there may be cases where the object is not counted even though it exists due to the presence of a truck between the object (vehicle) and the imaging device.

[0023] When an obstruction is present in this way, if a person (manager) is looking at the parking lot, the hidden object cannot be seen until the obstruction moves away. However, in the parking lot management device of the second embodiment, the imaging device 1 is set to acquire images of a predetermined area of ​​the parking lot, so special management is performed when an obstruction enters the parking lot area.

[0024] That is, Fig. 4 is an explanatory diagram showing the relationship between an obstructing object and an object to be identified. Fig. 4(a) shows a state in which a vehicle 30 to be identified is completely hidden by a truck 20. Fig. 4(b) shows a state in which the truck 20, which is the obstructing object, is about to park after the vehicle 30 to be identified has parked. Fig. 4(c) shows a state in which no vehicle 30 is parked in the area obstructed by the truck 20, which is the obstructing object, when the truck 20 parks. Fig. 4(d) shows a state in which a vehicle 30 is about to be parked in the area obstructed by the truck 20, which is the obstructing object.

[0025] 4(a), there are cases where even a part of the vehicle 30, which is the object to be identified, cannot be confirmed. When the parking lot is confirmed, a truck 20 is parked there, acting as an obstruction, so it is unclear whether the vehicle 30 is already present in the obstructed area or whether the vehicle 30 is not present at all.

[0026] When there is an area where it is determined that the presence of an object to be identified is unknown due to an obstruction as shown in Fig. 4(a), processing is performed as shown in Fig. 5. That is, Fig. 5 is a flow chart showing the processing procedure for counting the number of vehicles when the presence of a vehicle, which is an object to be identified, is unknown due to an obstruction. In managing a parking lot, the state of vehicles entering and exiting is stored as image information in a second storage device 3, and the presence of a vehicle as an object to be identified is confirmed using this stored image information.

[0027] First, as shown in Fig. 5, it is determined whether or not there is a possibility that a vehicle is completely obscured (step S10). If a vehicle parked in a parking lot can be confirmed in some way (if only a part of its outline can be confirmed, such as if it is partially protruding), it is determined that there is no obscured area, and the number of confirmed vehicles is tallied (step S11), completing the process. If there is a possibility that a vehicle is completely obscured, image information from before it was hidden by an obscurant is confirmed (step S12). Then, it is confirmed whether the vehicle was parked before the obscurant was present (step S13).

[0028] It is checked whether there is information that a vehicle that was already present has left after the obstruction entered. If the vehicle was not parked before the obstruction came into existence, it is checked whether a vehicle was parked behind the obstruction (the area obstructed by the obstruction) after the obstruction came into existence (step S14). In other words, if a vehicle was parked in the obstructed area, the number of vehicles is added (step S15). If there continues to be no vehicles behind the obstruction, it is determined that the number of vehicles has not increased or decreased (step 16).

[0029] If a vehicle was present in the hidden area before the presence of the obstructing object, the number of hidden vehicles is incremented (step S17). Furthermore, it is checked whether the vehicle that was present behind the obstructing object has moved since the image information indicating the presence of the vehicle was obtained (step S18). If the vehicle has moved after counting, the number of vehicles is decremented (step S19). Furthermore, it is checked in step S14 whether a new vehicle has parked after the vehicle moved. As described above, even if an obstruction is present, image information from the time the parking lot is managed is stored in the second storage device, so by rewinding the time of the image information and playing it back, it is possible to accurately determine whether a vehicle is parked behind the obstruction.

[0030] In the parking lot management device of the second embodiment, the presence of an object to be identified is estimated from partial information on the characteristic part. Furthermore, if there is an object that completely obscures the object to be identified, the presence of the object to be identified can be confirmed by checking image information before and after the presence of the obscurant. In other words, by checking past image information for the area of ​​the parking lot to be monitored where the object to be identified may be present and keeping a record of the confirmation status, management information that could not be obtained by human management can be obtained, and more reliable management can be achieved.

[0031] Furthermore, in this second embodiment, vehicle detection is possible not only during the day, but also at night, in bad weather conditions such as snow and rain. That is, with vehicle detection technology using clustering, vehicle detection is stable during the daytime when images are clear, but the detection accuracy is unstable in bad weather conditions such as at night, in snow and rain. In contrast, by utilizing deep learning object detection technology and training in advance with images taken at night, in snow and rain, vehicle detection can be performed stably. Stable vehicle detection even at night and in bad weather conditions such as snow and rain makes it possible to provide stable services.

[0032] Embodiment 3 Next, a case where the object identification device is applied to a parking lot management device and the attributes of users of the parking lot are totaled will be described. FIG. 6 is an explanatory diagram showing the configuration and flow of a parking lot management device using an object identification device. In the second embodiment (FIG. 3), the object to be identified was a vehicle. In the third embodiment, however, the object to be identified is a person or a pet. Image information from an imaging device installed in a parking lot to detect whether a parking lot is full or empty is used to understand the attributes of people entering the facility, which can be used for marketing and to communicate the facility's congestion status. Here, deep learning-based object recognition technology is used to detect people and count the number of people entering the facility. It is also possible to detect gender (male / female), age, and pets. Furthermore, it is possible to read emotions from human facial expressions.

[0033] In this third embodiment, step S330 includes a person as an object to be identified in step S230 of the second embodiment. Then, a "person" is set as an image segmentation (step S350). Then, as image recognition using deep learning, discrimination of gender, age, and facial expression is performed (step S351), and the acquired gender, age, and facial expression data is counted (step S352). In parallel, the number of dogs and cats is counted (step S360). Then, the data is aggregated on a cloud server (step S370), and the facility's congestion status is displayed on a homepage (step S380). Furthermore, information is categorized by time of day, day of the week, holidays, weather, etc., and the relationship between facility users and sales performance is analyzed (step S390). The results of this analysis can be reflected in the product lineup for sale (step S391).

[0034] As mentioned above, by classifying information by time of day, day of the week, holidays, weather, etc. and analyzing sales performance, it is possible to adjust the product lineup at the kiosk and the menu at the cafeteria to suit the customers who visit, which can help increase sales. Furthermore, by displaying the congestion status of the kiosk, it is possible to communicate the level of congestion, which can lead to improved satisfaction among users of the service area.

[0035] Embodiment 4 Next, a case where the object identification device is applied to a parking lot management device and demand forecasting is performed based on the area name and vehicle type on the license plate will be described. FIG. 7 is an explanatory diagram showing the configuration and flow of a parking lot management device using an object identification device. In this fourth embodiment, the area names on the license plates of facility users' vehicles are read and tallied using deep learning image recognition. Deep learning is also used to recognize vehicle types (passenger cars, trucks, buses), and the area names and vehicle types are tallied to determine customer demographics and perform demand forecasts for commercial facilities with parking lots.

[0036] In the second embodiment (Figure 3), the object to be identified is a vehicle, but in this fourth embodiment, the object to be identified is a license plate, and image information from an imaging device installed in a parking lot to detect whether a parking lot is full or empty is used to compile data (area name and vehicle type) from the license plates of vehicles entering the facility, thereby predicting demand for commercial facilities and tailoring the products and services sold to meet that demand, leading to increased sales and improved service.

[0037] In this fourth embodiment, step S430 includes the license plate as an object to be identified in step S230 of the second embodiment. Then, "license plate" is set in the image cutout (step S450). Then, as image recognition using deep learning, character and number discrimination is performed (step S451), and the acquired license plate data is counted (step S452). Then, the data is aggregated in a cloud server (step S490).

[0038] Embodiment 5. Next, a case will be described in which the object identification device is applied to a parking lot management device, making it possible to detect full / empty parking lot information according to the layout of the parking lot (layout of vehicle parking spaces). FIG. 8 is an explanatory diagram showing the configuration and flow of a parking lot management device using an object identification device. In this fifth embodiment, when it is not possible to determine whether a parking lot is full or empty based on the number of vehicles alone due to differences in operation, such as parking multiple standard vehicles in a space for a large vehicle, object recognition using deep learning is used to classify vehicles into standard vehicles, large vehicles, and motorcycles, and even if a standard vehicle is parked in a parking space for a large vehicle, if there is space to park another standard vehicle, it is determined that there is an empty vehicle, thereby enabling efficient use of parking spaces.

[0039] Similarly, because multiple motorcycles may be parked in a vehicle space, even if a motorcycle is parked in a parking space for a large vehicle or a standard vehicle, the system will determine whether there is an empty space for a standard vehicle. Also, if a vehicle is parked outside a parking space, the system will not reduce the count of empty vehicles and will count it as "parking outside a parking space," making it possible to recognize parking in an inappropriate parking space. Furthermore, classification boundaries are created to classify parking spaces into multiple groups, and can be changed as desired, allowing for efficient use.

[0040] In the second embodiment (Fig. 3), the object to be identified was a vehicle, but in this fifth embodiment, the object to be identified is classified as a passenger car, truck, bus, motorbike, or bicycle, and improper parking can be recognized by classifying the object into a passenger car, a large vehicle, or a two-wheeled vehicle.

[0041] In this fifth embodiment, step S530 is configured to allow identification of objects such as passenger cars, trucks, buses, motorcycles, and bicycles, and to aggregate the identified objects (step S530). A full / empty / congested parking lot determination is performed to determine whether the appropriate parking location for each vehicle type is being used, and vehicles parked in inappropriate areas are recognized (step S550). The results are then displayed on a display panel to alert users to improper parking (step S570). Furthermore, the cloud server displays information on whether the parking lot is full, empty, or congested on a homepage (step S580). Various data are also aggregated based on the vehicle count (step S590).

[0042] Embodiment 6 Next, we will explain a case where the object identification device is applied to a parking lot management device, where vehicles are recognized by type, and optimal vehicle allocation is calculated, not limited to parking spaces in parking areas. The configuration and flow are the same as those of the fifth embodiment. In this sixth embodiment, a parking lot is shown that is divided into parking areas for large vehicles and parking areas for standard / small vehicles, and the parking area for standard / small vehicles is congested, but a small / standard vehicle is parked in the large vehicle area during a time period when the parking area for large vehicles is empty. In other words, when parking two small vehicles in a parking area for large vehicles, it is possible to park two small vehicles, but with conventional vehicle detection methods, it is difficult to determine whether a small vehicle is parked in the large vehicle area. Therefore, by performing vehicle detection using deep learning, large / standard cars, small vehicles, and motorcycles are recognized separately.

[0043] This makes it possible to determine how many small vehicles are parked in a parking area for large vehicles and how many more can be parked.Furthermore, by analyzing the number of large, standard, and small vehicles parked where and how crowded they are depending on the time of day, day of the week, and weather, it is possible to calculate the ratio of more efficient parking lots. By setting the parking lot occupancy area for each vehicle class (large, standard, small, motorcycle, etc.) in advance and comparing it with the actual vehicle class parked there, it is possible to detect and report violations of occupancy. By classifying and determining whether a vehicle is large, small, standard, or two-wheeled, it is possible to measure parking lot congestion with a higher degree of accuracy. Furthermore, by analyzing parking lot usage by time of day, day of the week, and weather, it is effective in formulating parking lot improvement plans.

[0044] Embodiment 7 Next, a case where the object identification device is applied to a parking lot management device, which makes it possible to grasp the availability of parking lanes for the disabled, will be described. 9 is an explanatory diagram showing the configuration and flow of a parking lot management system using an object identification device. This seventh embodiment addresses the problem that the availability of handicapped parking lanes cannot be known until the user reaches the parking spot.

[0045] Parking lots have parking areas for the disabled, but parking lot occupancy monitoring systems that count entry and exit are unable to grasp the parking location of each vehicle. Therefore, by using deep learning to detect vehicles, it is possible to determine whether a parking space for the disabled is vacant or full. This makes it possible to notify users, security guards, and maintenance personnel via a display board at the parking lot entrance or via a distribution method such as the web.

[0046] This improves convenience by allowing users to determine in advance whether they can park in a disabled parking space. In this embodiment 7, when determining whether a parking area for disabled persons is full, empty, or crowded, the full or empty status of the parking area for disabled persons is determined (step S351), and the full or empty status of the parking area for disabled persons is displayed on the parking lot display panel (step S670).

[0047] Embodiment 8 Next, a case will be described in which the object identification device is applied to a parking lot management device, and image processing is performed in real time to deliver the congestion status of the parking lot to the outside. FIG. 10 is an explanatory diagram showing the configuration and flow of a parking lot management device using an object identification device. In this eighth embodiment, a case will be described in which, when information relating to a parking lot is distributed to an external party, appropriate measures are taken to respect the personal information and portrait rights contained therein.

[0048] To check the congestion status of a parking lot, images can be viewed in real time via the Internet. However, since the images captured by cameras clearly show vehicles, license plates, and people's faces, it is necessary to respect personal information and portrait rights. Therefore, the video cannot be distributed as is. To address this, deep learning-based object detection is used to detect vehicle license plates, faces of people in the video, and other distinctive features, and those parts are automatically blurred or mosaicked to make them less clear.

[0049] Alternatively, humans can be recognized and removed from the image in real time, creating a video that can be posted online. For this reason, the predicted bounding boxes are classified by selecting those with high reliability as the objects to be detected (people, license plates) (step S730), and the people and license plates in the information to be used for display are pixelated (step S750).Then, the cloud server makes the pixelated images available on the Internet in real time (step S780).

[0050] Embodiment 9 Next, the object identification device is applied to a parking lot management system, which prevents accidents by detecting vehicles parked in places other than designated parking lots and vehicles driving in the wrong direction on aisles. FIG. 11 is an explanatory diagram showing the configuration and flow of a parking lot management device using an object identification device. In this ninth embodiment, accidents may occur in a parking lot due to vehicles being parked in places other than designated parking spaces and vehicles not traveling in the designated direction.

[0051] In this ninth embodiment, deep learning object detection technology is used to detect vehicles parked on roads and taxiways, making it possible to detect whether a vehicle is parked illegally on roads and taxiways other than parking lots. Furthermore, a case will be described in which a wrong-way vehicle is detected by comparing the travel direction of a vehicle with the travel direction of a pre-set passage and taxiway.

[0052] As shown in FIG. 11, the edge computer 202 determines the driving direction of vehicles in the parking lot and vehicles entering and exiting (step S850). Reverse phase is detected based on the relationship between the vehicle's driving direction and the lane it is traveling in (step S851). Information about the wrong-way driving vehicle is then displayed on the display panel to the target vehicle and surrounding vehicles (step S870). Furthermore, the security room notifies maintenance and security personnel of the wrong-way driving vehicle (step S880). By notifying the target vehicle, surrounding vehicles, and maintenance and security personnel of illegal parking and wrong-way driving information, accidents can be prevented.

[0053] Embodiment 10 Next, the object identification device is applied to a parking lot management device, which detects a queue of vehicles entering a parking lot and provides information on the queue of vehicles outside the parking lot. FIG. 12 is an explanatory diagram showing the configuration and flow of a parking lot management device using an object identification device. In this embodiment 10, cameras are used to monitor not only the inside of the parking lot but also the line of vehicles outside the parking lot, and when the parking lot becomes full, the video footage from the camera monitoring the line of vehicles outside the parking lot is distributed externally so that it can be viewed on the Internet.

[0054] As shown in FIG. 12, the imaging device 1 (camera) captures an image of a vehicle waiting to enter (step S900). Inside the edge computer 202, a determination is made as to whether the vehicle is full, empty, or crowded, and the presence or absence of a vehicle waiting to enter is displayed (step S950). The number of vehicles waiting to enter is also counted (step S960). Then, the display panel displays the full, empty, or crowded status, and the presence or absence of waiting vehicles (step S970). The cloud server also displays information on the homepage as to whether the vehicle is full, empty, or crowded, and the presence or absence of waiting vehicles (step S980). Data is also collected (step S980). This allows vehicles entering the parking lot to be informed of the parking lot's status.

[0055] Embodiment 11 Next, when the object identification device is applied to a parking lot management device, in a parking lot that is divided into multiple areas such as a multi-story parking lot, the device prevents congestion by notifying the degree of fullness or vacancy for each of the multiple parking areas. FIG. 13 is an explanatory diagram showing the configuration and flow of a parking lot management device using an object identification device.

[0056] In this eleventh embodiment, the full / empty / crowded state of each of the multiple parking areas (multiple floors) is determined (step S1050). The number of vehicles on each floor is counted (step S1060), and the full / empty / crowded state of each floor is displayed on a display panel (step S1070). The cloud server displays the full / empty / crowded state of each floor on a homepage (step S1080). In addition, the data for each floor is compiled (step S1090). This allows users to see which floors and parking lots have fewer vehicles than the parking spaces by looking at the occupancy rate on the display board, thereby eliminating congestion on one floor or parking lot.

[0057] Embodiment 12 In this embodiment, an object identification device is applied to a parking lot management device to achieve the objective of obtaining information on the usage status of parking lots. In this twelfth embodiment, the parking space occupied by a vehicle is determined based on image information of the vehicle parked in the parking space in the parking lot. To make this determination, the object identification device has a function of recognizing the vehicle as an identification target through deep learning, as described in the first embodiment. In addition, coordinates are set by dividing the area of ​​the parking space in the parking lot, and it is possible to determine whether the vehicle is within the parking space or has parked outside the parking space. This determination can be learned and the reliability of the determination can be improved by repeatedly learning using the coordinate system representing the parking space and the vehicle's appearance information.

[0058] FIG. 14 is an explanatory diagram showing the configuration flow of a parking lot management device using an object recognition device. As shown in FIG. 14, the imaging device 1 (camera) captures an image of a parking lot. This captures images of parking spaces in the parking lot and the vehicles using them (step S200). Inside the edge computer 202, deep learning is used to repeatedly predict the use of parking spaces, thereby predicting the reliability that the detected vehicle is parked in a specific parking space (step S1000). If the reliability of the prediction that the vehicle is using the parking space is higher than a predetermined threshold, it is determined that the vehicle is parked in the parking space (step S1010). In addition, the edge computer 202 calculates the optimal height and installation angle (depression angle) of the imaging device 1 (camera) to acquire data related to the parking space (step S1020). Then, the display panel displays whether or not there are any parked vehicles (step S1170), and the cloud server displays whether or not there are any parked vehicles on the homepage (step S1180).In addition, the parking lot manager is notified of information about vehicles parked in the vehicle spaces (step S1030).

[0059] Embodiment 13 When an object identification device is applied to a vehicle management device, it uses deep learning object detection technology to process images of the road to determine whether or not there are stopped vehicles, making it possible to detect stopped vehicles faster than a human could. FIG. 15 is an explanatory diagram showing the configuration and flow of a vehicle management device using an object identification device. In this embodiment 13, the road conditions are monitored by a camera, and vehicles are detected by their appearance or part of their outline, and parked vehicles are detected using the deep learning technology of embodiment 1.

[0060] As shown in FIG. 15, the imaging device 1 (camera) captures an image of the road (step S1100). Inside the edge computer 202, a determination is made as to whether the road is full, empty, or congested, and the traveling direction and speed of vehicles traveling on the road are determined (step S1150). The presence or absence of stopped vehicles is also determined taking into account the speed difference with surrounding vehicles (step S1160). Then, the presence or absence of stopped vehicles is displayed on a display panel (step S1170), and the cloud server displays the presence or absence of stopped vehicles on a homepage (step S1180). The road administrator is also notified of the information about the stopped vehicles (step S1190). This will reduce the time it takes for road administrators to arrive after a stopped vehicle, thereby reducing the occurrence of rear-end collisions.

[0061] Although the present application describes various exemplary embodiments and examples, the various features, aspects, and functions described in one or more embodiments are not limited to application to a particular embodiment, but may be applied to the embodiments alone or in various combinations. Therefore, countless variations not illustrated are conceivable within the scope of the technology disclosed in the present specification, including, for example, cases where at least one component is modified, added, or omitted, and cases where at least one component is extracted and combined with components of another embodiment. [Explanation of symbols]

[0062] REFERENCE SIGNS LIST 1 Imaging device, 2 First storage device, 3 Second storage device, 4 Information processing device, 5 Identification device, 6 Display device, 7 Identification object setting unit, 8 Teacher data creation unit, 9 Similar object data creation unit, 10 Learning processing device, 11 Learning processing means, 12 Object identification means, 20 Truck, 30 Vehicle, 100 Object identification device, 200 Processor, 201 Storage device, 202 Edge computer

Claims

[Claim 1] a vehicle identification device that identifies whether the vehicle is included in the image information by dividing the image information acquired by the imaging device into multiple regions and determining whether the image information of each divided region contains the stored characteristic features of the vehicle; and an information processing device that sets coordinates representing parking spaces in the parking lot included in the image information, performs deep learning on the coordinate information, information on the identified vehicle's external shape, and data related to the parking space including at least the installation height and installation angle of the imaging device, predicts whether the parking space in the image information is being used by the vehicle, and determines whether the parking space is being used by the vehicle depending on the reliability of the prediction.

Images