Method for checking parking occupation status and apparatus for the same

Abstract

A method for checking the parking occupancy status is disclosed for verifying whether a parking space is occupied in a parking lot. The parking occupancy status checking method according to the present invention acquires a parking space image in which a parking space is captured, detects a vehicle in the acquired parking space image, extracts a detected vehicle image for the vehicle from the parking space image, applies a pre-trained artificial intelligence model to the detected vehicle image to derive the vehicle direction and floor point height of the vehicle, and matches the derived vehicle direction and floor point height to the specific parking space.

Description

Technology Field

[0001] The present invention relates to a method and apparatus for checking parking occupancy status, and more specifically, to a method and apparatus for checking parking occupancy status using artificial intelligence. Background Technology

[0002] As the number of automobiles increases, the need for efficient parking management in various public and private buildings is growing. In particular, large parking lots at airports, major shopping centers, and entertainment facilities are increasingly equipped with technology that notifies users of available parking spaces.

[0003] The technology for checking the occupancy status of a vehicle parking area using conventional technology (hereinafter referred to as "parking occupancy status checking") simply determines whether a vehicle is parked in a parking area where the sensor is installed by using ultrasonic or geomagnetic sensors, thereby managing the number of parking areas occupied and unoccupied by vehicles in the entire parking area, or checks and manages the occupancy status of a parking area by using images acquired through cameras such as CCTV.

[0004] However, while using cameras can resolve many of the problems associated with sensor-based methods, there is a disadvantage in that many cameras must be used to overcome this, as the accuracy regarding whether a vehicle is parked is relatively lower compared to those methods.

[0005] In other words, the parking occupancy status verification technology using cameras according to the prior art has a problem in that the complexity and maintenance costs of the parking status management device increase significantly because many cameras must be installed in proportion to the size of the parking lot in order for the cameras to verify the parking status without distortion. The problem to be solved

[0006] The objective of the present invention, which aims to solve the aforementioned problems, is to provide a method and apparatus that enable accurate verification of parking occupancy status using a small number of cameras, even if the size of the parking lot increases.

[0007] The objectives of the present invention are not limited to those mentioned above, and other unmentioned objectives will be clearly understood by those skilled in the art from the description below. means of solving the problem

[0008] A method for verifying a vehicle occupancy status according to an embodiment of the present invention for achieving the above objective acquires a parking area image in which a parking area is captured, detects a vehicle in the acquired parking area image, extracts a detected vehicle image for the vehicle from the parking area image, applies a pre-trained artificial intelligence model to the detected vehicle image to derive the vehicle parking direction and floor point height of the vehicle, and matches the derived vehicle parking direction and floor point height to the specific parking area. Effects of the invention

[0009] According to the present invention, the parking occupancy status of a parking lot can be checked and managed even in a large parking lot using a small number of cameras. In addition, by applying multi-object tracking technology, objects can be efficiently tracked in camera images containing multiple objects, such as vehicles. Brief explanation of the drawing

[0010] FIG. 1 is a configuration diagram showing a parking occupancy status checking device according to one embodiment of the present invention. FIG. 2 is a conceptual diagram showing a parking occupancy status checking device according to the prior art. Figure 3 is a conceptual diagram showing a method for checking parking occupancy status according to the prior art. FIG. 4 is a conceptual diagram illustrating a vehicle classification method in a parking occupancy status checking device according to one embodiment of the present invention. FIG. 5 is a conceptual diagram illustrating a method for checking the parking occupancy status in a parking occupancy status checking device according to one embodiment of the present invention. FIG. 6 is a conceptual diagram illustrating a method for checking parking occupancy in a parking occupancy status checking device according to another embodiment of the present invention. FIG. 7 is a flowchart illustrating a method for checking parking occupancy status according to an embodiment of the present invention. FIG. 8 is a flowchart illustrating a method for checking parking occupancy status according to another embodiment of the present invention. Specific details for implementing the invention

[0011] The present invention is capable of various modifications and may have various embodiments, and some embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to specific embodiments, and the scope of the present invention should be understood to include all modifications, variations, equivalents, and substitutions that fall within the technical spirit of the present invention related to the embodiments.

[0012] Terms such as "first," "second," etc., may be used to describe various components, but said components should not be limited by said terms. Such terms are used solely for the purpose of distinguishing one component from another. The term "and / or" includes a combination of multiple related described items or any of the multiple related described items.

[0013] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. It should be understood that no other components exist in between only when it is stated that one component is "directly connected" or "directly connected" to another component.

[0014] The terms used in this application are used merely to describe the embodiments presented and are not intended to limit the invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "having" are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of other features, numbers, steps, actions, components, parts, or combinations thereof.

[0015] As used in this application, "the above" and similar designations may indicate both singular and plural forms. Furthermore, unless there is a description explicitly specifying the order of steps describing the method according to this disclosure, the described steps may be performed in the order intended to achieve the purpose. This disclosure is not limited by the order in which the described steps are described.

[0017] All terms used in this specification, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which the present invention pertains, and should not be interpreted in an ideal or overly narrow formal sense; and where the meaning of any term is defined in this specification, that term shall be interpreted as defined.

[0018] Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the attached drawings. In order to facilitate an overall understanding of the present invention, the same reference numerals are used for identical components in the drawings, and redundant descriptions of identical components are omitted.

[0020] The present invention is characterized by a configuration that enables efficient management of a parking lot by identifying the parking status of a parking lot using only a small number of cameras, by utilizing artificial intelligence modeling technology that has been learned from images of vehicles captured from various angles and heights in advance. In other words, the present invention uses deep learning to classify the types of detected vehicles according to direction and floor point height, thereby measuring the precise location of the vehicle and significantly increasing the accuracy of matching with the parking area. Furthermore, it can distinguish special situations such as double parking, and the number of vehicles or parking areas that can be processed by a single camera can be increased due to the improved accuracy of vehicle location, which can significantly reduce costs in the parking management system.

[0022] FIG. 1 is a configuration diagram showing a parking occupancy status checking device according to one embodiment of the present invention.

[0023] Referring to FIG. 1, a parking occupancy status checking device (100) according to one embodiment of the present invention comprises a parking lot shooting unit (110) composed of one or more cameras, a vehicle image extraction unit (130) that detects a parked vehicle in a parking area shooting image captured by the parking lot shooting unit (110) and extracts a detected vehicle image for the vehicle, an artificial intelligence modeling unit (140) that learns photos of various vehicles according to shooting angles and heights and derives information on the vehicle parking direction and floor point height when the vehicle is parked extracted by the vehicle image extraction unit (130) therefrom through artificial intelligence modeling, a parking area matching unit (150) that matches a vehicle with a specific parking area of ​​the parking lot based on the information on the vehicle parking direction and floor point height for the vehicle parked in the parking area derived by the artificial intelligence modeling unit (140), and a control unit (120) that controls the operation of the parking lot shooting unit (110), the vehicle image extraction unit (130), the artificial intelligence modeling unit (140), and the parking area matching unit (150).

[0024] The parking lot shooting unit (110) of the parking occupancy status checking device (100) according to one embodiment of the present invention may include one or more cameras as cameras for shooting vehicles parked in the parking lot. As shown in FIGS. 2 and 3, in the case of the prior art, the frontal image captured by the camera can clearly confirm whether a vehicle is parked in a specific parking area, but the side image makes it difficult to clearly confirm which parking area the vehicle is parked in. That is, in the frontal view of the camera, the camera can clearly identify that the vehicle (301, 302) is parked in the parking area (303, 304), but in the case of the parking area (306, 307) on the side of the camera, it is difficult to clearly identify which parking area the vehicle (305) is parked in.

[0025] In particular, as shown in Fig. 1, when a single camera covers multiple parking areas, it becomes difficult to check the parking occupancy status of the parking area, especially for the side parking areas.

[0026] A vehicle image extraction unit (130) of a parking occupancy status verification device (100) according to one embodiment of the present invention detects a vehicle in a parking area captured image obtained from a parking lot capturing unit (110), and if a vehicle is detected, extracts a detected vehicle image (meaning an image of a vehicle detected in a parking area captured image) for the vehicle from the parking area captured image.

[0027] According to one embodiment of the present invention, the artificial intelligence modeling unit (140) of the parking occupancy status verification device (100) applies a pre-learned artificial intelligence model to the detected vehicle image to derive the vehicle parking direction and floor point height of the vehicle. The artificial intelligence modeling unit (140) has pre-learned images of various shooting directions and various shooting heights of various vehicles, and uses the results of this learning to perform artificial intelligence modeling on the vehicle in the detected vehicle image to derive the vehicle parking direction and floor point height of the vehicle. Meanwhile, the artificial intelligence modeling in the embodiments of the present invention may be implemented using deep learning, but is not necessarily limited thereto and may be implemented through various artificial intelligence techniques for image identification and detection.

[0028] As shown in FIG. 4, which illustrates a method for classifying vehicles in a parking occupancy status checking device according to one embodiment of the present invention, the vehicles extracted by the vehicle image extraction unit (130) have various appearances depending on the vehicle parking direction and floor point height. The artificial intelligence modeling unit (140) derives values ​​for the vehicle parking direction and floor point height of the vehicles extracted by the vehicle image extraction unit (130) through deep learning based on images of various vehicle types with various vehicle parking directions and vertical heights of floor points that have been pre-learned.

[0029] The number of classifications for the vehicle's parking direction and floor point height classified in the artificial intelligence modeling unit (140) can be applied in various ways depending on the application method. Meanwhile, while the vehicle's parking direction and floor point height are continuous values, the classification in FIG. 3 is a discontinuous value and may not match exactly. However, due to the nature of deep learning, in the case of ambiguous classification values, similar nearby values ​​are output together. Therefore, continuous values ​​for the vehicle's parking direction (dir) and floor point height (ht) can be obtained by averaging and using values ​​that consider not only the highest value but also the second and third ranks, as in Equations 1 and 2.

[0030]

[0032]

[0033] is the value of the artificial intelligence output for the vehicle direction, and in the case of Fig. 3, it can be one of the values ​​0 degrees, 45 degrees, 90 degrees, 135 degrees, 180 degrees, 225 degrees, 270 degrees, and 315 degrees. is output It is the reliability of, and the values ​​can be sorted in descending order. is a user-selectable value and is selected as a value greater than 1 and less than or equal to the total number of direction values. The floor point height (ht) is also in the case of Fig. 3. The value of can be implemented as one of the values ​​0, 0.25, or 0.5. is output It is the reliability and can be sorted in descending order. Likewise, it must be a value greater than 1 and equal to or less than the total number of vehicle heights.

[0034] For example, In the case where the number is 4 and one of the 4 directions (0 degrees, 90 degrees, 180 degrees, 270 degrees) is determined, the artificial intelligence modeling unit (140) for the 4 directions outputs a reliability value, and in the case where the target vehicle is in the 0-degree direction, the artificial intelligence modeling unit (140) can output a reliability for each direction as follows.

[0035] Possible directions (di) Reliability (ci) 0 degrees 0.9 90 degrees 0.05 180 degrees 0.0 270 degrees 0.05

[0036] The vehicle parking direction (dir) and floor point height (ht) derived in the manner described above are derived using an artificial intelligence method. A parking area matching unit (150) according to one embodiment of the present invention matches a vehicle with a specific parking area of ​​a parking lot based on the vehicle parking direction value and floor point height value received from the artificial intelligence modeling unit (140).

[0037] Looking at the example of Table 1 described above, the final vehicle parking direction value is derived as 13.5 (= 0*0.9 + 90 * 0.05 + 180*0 + 270*0.05). Alternatively, if the direction is 270 degrees, applying -90 degrees instead of 270 degrees results in a final vehicle parking direction value of 0 (0*0.9 + 90 * 0.05 + 180*0 + (-90)*0.05 = 0). In other words, in such a case, the parking area matching unit (150) can determine that the vehicle is parked in the corresponding parking area (the closer the vehicle parking direction value is to 0 degrees or 180 degrees, the more likely it is to be determined that the vehicle is parked in the parking area). At this time, it is not necessary to add up all the values ​​of kd (4 in this case), and only one or two to three values ​​can be added and used. Likewise, if the floor point height value is also derived from the artificial intelligence modeling unit (140), it can be determined whether the vehicle is located in the center of the parking area.

[0038] As shown in the matching between the vehicle and the parking area for checking the parking occupancy status in the parking occupancy status checking device according to one embodiment of the present invention in FIG. 5, since the vehicle is generally symmetrical, if the height of the vehicle's bottom point is known, the approximate location (501) of the vehicle's bottom point in the image can be obtained. Since the parking area (504) is shaped like a polygon such as a rectangle or a parallelogram, if an algorithm is used to determine whether a specific point exists inside or outside the polygon, it is easy to determine whether the vehicle's bottom point (501) is inside the parking area (504). If the vehicle's bottom point (501) is inside the parking area (504), the vehicle is considered to have been parked in the parking area.

[0039] Generally, most vehicles are rectangular in shape, long in one direction, and accordingly, the parking area (504) is also rectangular or parallelogram-shaped, so if the parking direction (502) of the vehicle is known, it is possible to identify whether the direction matches the shape of the parking area. That is, if the parking area direction (503) and the parking direction (502) of the vehicle are close to 0 degrees or 180 degrees, the probability of matching the parked vehicle and the parking area increases, and if they are close to 90 degrees, the probability of matching decreases, so it is possible to determine which parking area the vehicle is parked in.

[0040] The present invention can also be applied to determining whether a vehicle is double-parked, as in the parking occupancy status check device according to another embodiment of the present invention of FIG. 6. That is, in the conventional case, the rectangular position (602) of the vehicle overlaps significantly with the parking area (601), so there is a high possibility of incorrectly determining that a double-parked vehicle is parked in the parking area (601). However, in the case of the present invention, since the direction (604) of the vehicle and the direction (605) of the parking area (606) are close to 90 degrees, it can be easily determined that the vehicle is not parked in the parking area.

[0041] A control unit (120) of a parking occupancy status checking device (100) according to one embodiment of the present invention controls the operation of the aforementioned parking lot shooting unit (110), vehicle image extraction unit (130), artificial intelligence modeling unit (140), and parking area matching unit (150). The control unit (120) controls data input and output between each component module of the parking occupancy status checking device (100) and can determine which parking area a vehicle is parked in based on the determination result of the parking area matching unit (150). When a parking lot shooting unit (110) of a parking lot includes a plurality of cameras and uses them to check the parking occupancy status of the parking lot, a specific parking area may be captured by the plurality of cameras, and a captured image of the corresponding parking area may be obtained from each camera and a parking determination value may be derived. At this time, the control unit (120) can more reliably determine whether a vehicle occupies a parking area by controlling the parking area matching unit (150) by considering the vehicle images from the plurality of cameras and the value derived from the artificial intelligence modeling unit (140). At this time, the control unit (120) can check the parking area occupancy status by integrating images of the same parking area that overlap between multiple cameras.

[0043] FIG. 7 is a flowchart illustrating a method for checking parking occupancy status according to an embodiment of the present invention.

[0044] Referring to FIG. 7, a parking occupancy status checking method according to one embodiment of the present invention is an example of an operation to determine whether a vehicle is parked in a parking area of ​​a parking lot using a parking occupancy status checking device according to one embodiment of the present invention of FIG. 1.

[0045] First, the parking area of ​​the parking lot is photographed by the parking lot camera (110) (S710).

[0046] By the vehicle image extraction unit (130), a parking area image is obtained, and a vehicle is detected therefrom (S720).

[0047] If a vehicle is detected in the captured image of the parking area (e.g., S730), the vehicle image extraction unit extracts the detected vehicle image, which is an image of the vehicle detected in the captured image of the parking area (S740). If no vehicle is detected in the captured image of the parking area (e.g., S730), the vehicle is detected again in the captured parking area to determine whether a vehicle is detected.

[0048] By the artificial intelligence modeling unit (140), a pre-trained artificial intelligence model is applied to the detected vehicle image (S750) to derive the vehicle parking direction and floor point height of the detected vehicle (S760).

[0049] The vehicle parking direction and floor point height derived by the parking area matching unit (150) are matched to the parking area (S770). Through this, it is determined which parking area the detected vehicle is parked in (S780). Determining which parking area the detected vehicle is parked in may be performed by the parking area matching unit (S150) or the control unit (120) depending on the implementation situation.

[0050] Meanwhile, as described above, when the parking areas captured by multiple cameras of the parking lot camera unit (110) overlap, the images captured by each camera can be integrated to more accurately determine whether a vehicle occupies a specific parking area.

[0051] In addition, the parking occupancy status check method according to one embodiment of the present invention can be applied to MOT (multi-object tracking). That is, the parking occupancy status check method according to another embodiment of the present invention of FIG. 8 (based on the embodiments of the present invention of FIG. 1 and FIG. 7) detects a vehicle, which is an object, for each frame image (meaning a frame image of a parking area captured by a camera of a parking lot capturing unit (110)) that is captured and acquired (S810) in MOT, which is a technology for tracking multiple objects (vehicles) in a time flow, and generates a path for each object (S850) by matching the object detected in the previous frame with the vehicle detected in the current frame (S830) in a one-to-one manner.

[0052] At this time, since the vehicle image varies slightly depending on the frame image, various feature values ​​(e.g., vehicle size, color, type, model, direction, etc.) are compared to match similar ones in order to prevent matching different objects (vehicles). That is, when the parking direction and floor point height of the vehicle are derived and identified through the parking occupancy status verification method according to another embodiment of the present invention of FIG. 8 (S840), the vehicle detected in the previous frame image is matched with the vehicle detected in the current frame image (S870), and the information thus matched is used to create and store a path (a set of matching results) (S860), and then matched with the result of the next frame image (here, the path refers to a set of vehicle information (locations) detected in the still image).

[0053] At this time, when a specific parking area is photographed by multiple cameras, the results of confirming the parking area occupancy status matched to each camera (referred to as parking information) are summed (S880) to more accurately confirm the parking area occupancy status.

[0055] The above embodiments may be implemented using various forms of computing means including one or more processors, memory, and storage means. Additionally, a network interface connected to a wired or wireless network may be included. Each of the aforementioned components communicates data through a data communication bus. The processor may be a central processing unit or a semiconductor device that executes processing instructions stored in memory and / or storage units. The memory and storage units may include volatile storage media or non-volatile storage media. For example, the memory may include ROM and RAM.

[0056] Additionally, the steps of the method or algorithm described in connection with the embodiments disclosed herein may be directly implemented by hardware, software modules, or a combination of both, executed by a processor. The software modules may reside in storage media such as RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disks, removable disks, and CD-ROMs.

[0057] An exemplary storage medium is coupled to a processor, and the processor can read information from the storage medium and write information to the storage medium. Alternatively, the storage medium may be integrated with the processor. The processor and the storage medium may reside within an Application-Specific Integrated Circuit (ASIC). The ASIC may reside within a user terminal. Alternatively, the processor and the storage medium may reside as separate components within the user terminal.

[0059] The configuration of the present invention has been described in detail above through preferred embodiments. However, the aforementioned embodiments are merely examples and do not limit the scope of the rights of the present invention. A person skilled in the art will be able to make various modifications and changes within the scope of the technical concept of the present invention from the teachings and suggestions of this specification. For example, the control unit (120) and the parking area matching unit (150) may be implemented as a single integrated module or divided into two or more devices. Accordingly, the scope of protection of the present invention should be determined by the description in the following claims.

Claims

Claim 1 A parking occupancy status verification device comprising: a vehicle image extraction unit that extracts a detected vehicle image corresponding to a vehicle by detecting a vehicle parked in the parking area from a parking area captured by a camera; an artificial intelligence modeling unit that derives the vehicle parking direction and floor point height of the vehicle by applying the detected vehicle image to a pre-trained artificial intelligence model; and a parking area matching unit that matches the vehicle with a specific parking area of ​​a parking lot based on the vehicle parking direction and floor point height of the vehicle, wherein the floor point height refers to the ratio of the distance from the floor line segment of the vehicle's bounding box generated as a result of the detection to the floor point, which is the center point of the floor surface forming the boundary of the lower part of the vehicle, to the height of the bounding box. Claim 2 A parking occupancy status verification device according to claim 1, wherein the artificial intelligence model is a model that outputs reliability for each of a plurality of pre-set discontinuous vehicle parking directions and a plurality of discontinuous floor point heights, and the artificial intelligence modeling unit applies the detected vehicle image to the artificial intelligence model, extracts a first number and a second number of values ​​from the plurality of discontinuous vehicle parking directions and the plurality of discontinuous floor point heights, respectively, according to the order of high reliability, and calculates the average of the first number of values ​​and the average of the second number of values ​​using the reliability to derive the vehicle parking direction and floor point height of the vehicle. Claim 3 A parking occupancy status checking device according to paragraph 2, wherein the first number and the second number are selected by the user. Claim 4 A parking occupancy status confirmation device according to paragraph 2, wherein the parking area matching unit identifies whether the parking direction of the vehicle matches the direction corresponding to the shape of the parking area obtained from the parking area shooting image and determines that the vehicle is parked in the corresponding parking area. Claim 5 A parking occupancy status confirmation device according to claim 4, wherein the parking area matching unit determines that the vehicle is parked in the corresponding parking area as the angle formed by the vehicle parking direction and the direction corresponding to the shape of the parking area approaches 0 degrees or 180 degrees. Claim 6 A parking occupancy status verification device according to claim 4, wherein the parking area matching unit determines that the vehicle is a double-parked vehicle not in the corresponding parking area as the direction corresponding to the shape of the parking area and the vehicle parking direction of the vehicle approaches 90 degrees. Claim 7 A parking occupancy status verification device according to paragraph 2, wherein the height of the plurality of discontinuous floor points is set to a value of 0.5 or less. Claim 8 A parking occupancy status verification device according to claim 1, wherein the parking area matching unit determines whether the floor point exists inside a polygon corresponding to the parking area obtained from the parking area shooting image and determines that the vehicle is parked in the corresponding parking area. Claim 9 A parking occupancy status verification device according to claim 1, wherein the parking area matching unit tracks the vehicle from the previous frame to the current frame of the parking area captured video using the vehicle parking direction and floor point height of the vehicle, and generates the tracking result of the vehicle as a single path to verify the occupancy status of the vehicle in a specific parking area.

Images