Parking management system and computing device for accurately identifying vehicles without stagnation phenomenon or using identifiers

Abstract

A parking management system capable of accurately identifying vehicles without the stagnation phenomenon is disclosed. The parking management system includes a first image capture device for recognizing the vehicle number of a vehicle at the entrance or inside of a parking lot, a second image capture device installed inside the parking lot, and a computing device communicatively connected to the first image capture device or the second image capture device. The computing device identifies a vehicle by considering detected vehicle information using RE-ID (re-identification) technology and vehicle information obtained by analyzing an image captured by at least one of the image capture devices, and the re-identification technology determines whether the vehicle is the same based on vehicle exterior information included in an image captured by the first image capture device and vehicle exterior information included in an image captured by the second image capture device.

Description

[Technical field]

[0001] The present invention relates to a parking management system that accurately identifies vehicles or utilizes identifiers without the stagnation phenomenon. [Background technology]

[0002] Parking sharing services often run into problems when they arrive at the parking lot due to illegally parked vehicles. For this reason, various methods have been developed to check whether a vehicle is in the parking lot. For example, as shown in Figure 1, there are methods such as checking the presence of a vehicle through 1:1 matching using IoT sensors, checking through CCTV cameras, and checking through GPS signals received by mobile phones.

[0003] However, such parking control is usually implemented to separate the outside space from the controlled inside space by a barrier and to prevent vehicles from leaving without paying the fee, but such barriers cause many inconveniences: they can break down, they are expensive to install, and the barriers can cause vehicles to get stuck when entering the parking lot.

[0004] In a conventional parking management system as shown in Figure 2, the barrier opens after the vehicle number is recognized through an ALPR camera and the vehicle is parked in the parking lot. When the vehicle leaves, the ALPR camera recognizes the vehicle number again, and once payment is made at the KIOSK, the barrier rises. In this type of parking management system, the barrier can cause vehicle entry to be stalled as shown in Figure 3, and the barrier is expensive, which increases the construction cost of the parking management system.

[0005] Also, as shown in Figure 4, errors may occur in the recognition of vehicle number plates when the ALPR camera that recognizes the vehicle number plates takes pictures at an oblique angle. Therefore, most parking lots have narrow entrance roads, and inexperienced drivers experience the inconvenience of their wheels or car bodies scraping against the road.

[0006] In addition, the concrete structures for installing ALPR cameras in the entrance areas take up a certain amount of space, which may cause vehicle scraping more frequently for inexperienced drivers. In other words, in order to take measurements with an ALPR camera, the road must be narrowed and the ALPR camera must be installed above the road, which could result in damage to the vehicle. Summary of the Invention [Problem to be solved by the invention]

[0007] The present invention provides a parking management system that can accurately identify vehicles without the stagnation phenomenon. The present invention also provides an RE-ID technology that can accurately identify vehicles. Additionally, the present invention provides a tracking technique that can determine when a vehicle is parked or exiting. Furthermore, the present invention provides a parking management system that can settle parking fees by using the identifier. [Means for solving the problem]

[0008] In order to achieve the above object, a parking management system according to an embodiment of the present invention includes a first image acquisition device for recognizing a vehicle number of a vehicle at the entrance or inside of a parking lot, a second image acquisition device installed inside the parking lot, and a computing device communicatively connected to the first image acquisition device or the second image acquisition device. Here, the computing device identifies the vehicle by considering vehicle information detected using a RE-ID (re-identification) technology and vehicle information obtained by analyzing an image acquired by at least one of the first and second image acquisition devices, and the re-identification technology is a technology for determining whether the vehicle is the same vehicle based on vehicle exterior information included in an image acquired by the first image acquisition device and vehicle exterior information included in an image acquired by the second image acquisition device.

[0009] A parking management system according to another embodiment of the present invention includes a first image acquisition device for recognizing a vehicle number of a vehicle, a second image acquisition device, an identifier, and a computing device communicatively connected to the first image acquisition device or the second image acquisition device, wherein the computing device identifies the vehicle using at least one of vehicle information obtained by analyzing an image acquired by at least one of the first and second image acquisition devices and vehicle information detected using a re-identification (RE-ID) technology, and when a user of the vehicle inputs user information through recognition of the identifier, the computing device bills the user for a parking fee based on the input user information or automatically settles the parking fee, and the re-identification technology is a technology for determining whether the vehicle is the same vehicle based on vehicle exterior information included in an image acquired by the first image acquisition device and vehicle exterior information included in an image acquired by the second image acquisition device.

[0010] A computing device used in a parking management system according to an embodiment of the present invention includes a communication unit which is a communication connection path with a first image acquisition device or a second image acquisition device, and a re-identification unit which identifies a vehicle that has entered a parking lot by considering all of vehicle information detected using a RE-ID (re-identification) technology and vehicle information obtained by analyzing an image acquired by at least one of the first and second image acquisition devices. Here, the re-identification technology is a technology for determining whether a vehicle is the same vehicle based on vehicle external shape information included in an image acquired by the first image acquisition device and vehicle external shape information included in an image acquired by the second image acquisition device.

[0011] A computing device used in a parking management system according to another embodiment of the present invention includes a communication unit which is a communication connection path with a first image acquisition device or a second image acquisition device, an identifier unit which receives input of user information through identifier recognition, a billing unit, and a re-identification unit which identifies a vehicle that has entered a parking lot using at least one of vehicle information detected using a RE-ID (re-identification) technology and vehicle information obtained by analyzing an image acquired by at least one of the first and second image acquisition devices. Here, when a user of the vehicle inputs user information through the identifier recognition, the billing unit bills the user for a parking fee based on the input user information or automatically settles the parking fee, and the re-identification technology is a technology for determining whether the vehicle is the same vehicle based on vehicle exterior information included in an image acquired by the first image acquisition device and vehicle exterior information included in an image acquired by the second image acquisition device. Effect of the Invention

[0012] The parking management system of the present invention identifies vehicles by using the vehicle number recognized by the camera together with RE-ID (re-identification) technology, so that the vehicle can be accurately identified regardless of the camera position, recognition errors, etc. In addition, the parking management system does not require barriers, so it does not require expensive installation costs, and it can prevent parking problems. Furthermore, it can significantly reduce friction between vehicles when they enter and exit the parking lot. Furthermore, when parking fees are paid using an identifier in the parking management system, parking fees can be paid even for vehicles that leave without paying the parking fee, and it is possible to eliminate the need to install payment machines and exit barriers. [Brief description of the drawings]

[0013] [Figure 1] 1 is a diagram illustrating a conventional parking surface checking method. [Diagram 2] 1 is a diagram illustrating a configuration of a conventional parking management system. [Diagram 3] 1 is a diagram illustrating an example of a traffic jam occurring when a vehicle enters a road; [Figure 4] 13 is a diagram illustrating a case where a vehicle number plate is not recognized by diagonal photography. [Diagram 5] 1 is a block diagram illustrating a schematic configuration of a parking management system according to an embodiment of the present invention. [Figure 6] 1 is a diagram illustrating an example of a Re-ID technique according to an embodiment of the present invention. [Figure 7] 1 is a diagram illustrating an example of a method for identifying differences between vehicles of the same model according to an embodiment of the present invention; [Figure 8] 1 is a diagram illustrating a learning process of an RE-ID deep learning model using Triplet loss according to an embodiment of the present invention. [Figure 9] FIG. 11 is a block diagram illustrating a schematic configuration of a parking management system according to another embodiment of the present invention. [Figure 10] FIG. 11 is a block diagram illustrating a parking management system according to another embodiment of the present invention. [Figure 11] 1 is a diagram illustrating a tracking process according to an embodiment of the present invention; [Figure 12] 1 is a diagram illustrating a tracking process according to an embodiment of the present invention; [Figure 13] 1 is a diagram illustrating a tracking process according to an embodiment of the present invention; [Figure 14] 1 is a diagram illustrating a tracking process according to an embodiment of the present invention; [Figure 15] 1 is a diagram illustrating a QR code recognition process according to an embodiment of the present invention. [Figure 16] 1 is a diagram illustrating a QR code processing process according to an embodiment of the present invention. [Figure 17] 1 is a diagram illustrating a re-identification technique according to another embodiment of the present invention. [Figure 18] 1 is a diagram illustrating a re-identification technique according to another embodiment of the present invention. [Figure 19] 1 is a diagram illustrating a re-identification technique according to another embodiment of the present invention. [Figure 20] 1 is a diagram illustrating a re-identification technique according to another embodiment of the present invention. [Figure 21] 1 is a diagram illustrating a re-identification technique according to another embodiment of the present invention. [Figure 22] 1 is a diagram illustrating a re-identification technique according to another embodiment of the present invention. [Figure 23] 1 is a diagram illustrating a re-identification technique according to another embodiment of the present invention. [Figure 24] 1 is a diagram illustrating a re-identification technique according to another embodiment of the present invention. [Diagram 25] 4 is a flowchart illustrating a parking space guidance process according to an embodiment of the present invention. [Figure 26] FIG. 1 is a block diagram illustrating a computing device in accordance with one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] As used herein, singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as "comprise" or "include" should not be interpreted as including all of the components or steps described in the specification, and should be interpreted as including some components or steps not included, or including additional components or steps. In addition, terms such as "... unit" and "module" described in the specification mean a unit that processes at least one function or operation, which may be embodied in hardware or software, or a combination of hardware and software.

[0015] The present invention relates to a parking management system, and has a structure that does not cause stagnation. In particular, the parking management system can be efficiently operated and can be significantly cheaper than conventional systems in terms of cost.

[0016] Conventional parking management systems basically use barriers. This is because in order to accurately recognize the vehicle number, the vehicle must be photographed by a camera after it is parked. In particular, in order to accurately identify the vehicle, the vehicle entrance road must be narrowed and barriers must be used. However, when the camera photographs the vehicle at an oblique angle, the vehicle number cannot be accurately identified.

[0017] To solve these problems, the parking control system of the present invention uses RE-ID (Re-identification) technology that it has developed independently. That is, even if the vehicle number cannot be accurately recognized when the vehicle enters, the parking control system can accurately identify the vehicle by utilizing RE-ID technology. Therefore, there is no need to narrow the vehicle entrance road, and it is not necessary to use a barrier when the vehicle enters. However, the barrier can be used if it is necessary for the safety of the vehicle.

[0018] Of course, even if the barrier is used, if a vehicle is expected to be parked, or if the barrier is broken and the barrier is left open, the vehicle can be accurately identified, so the parking management system can charge the vehicle an accurate fee (parking fee).

[0019] According to another embodiment, the parking management system can register entry using an identifier (e.g., a QR code) installed in the parking area, so that even if payment is not made through a payment machine when a vehicle leaves the parking lot, payment can be automatically made through the information registered with the identifier. Therefore, payment can be made even if there is no barrier in the exit area.

[0020] Various embodiments of the present invention will now be described in detail with reference to the accompanying drawings. Fig. 5 is a block diagram illustrating a schematic configuration of a parking management system according to an embodiment of the present invention, Fig. 6 is a diagram illustrating an example of a Re-ID technology according to an embodiment of the present invention, Fig. 7 is a diagram illustrating an example of a method for identifying differences between the same vehicle type according to an embodiment of the present invention, and Fig. 8 is a diagram illustrating a learning process of an RE-ID deep learning model using triplet loss according to an embodiment of the present invention.

[0021] 5, the parking management system of the present embodiment may include an entrance camera and a barrier installed at the entrance to the parking lot, an exit camera installed at the exit of the parking lot, a payment machine and a barrier, and a computing device (not shown, for example, a server). Here, when entry and exit are performed on the same passage, there may be only one barrier for the entry and exit. In addition, a camera (e.g., an ALPR camera) is one example of an image capturing device that can recognize a vehicle number plate, and various devices can be used as long as they can recognize the vehicle number plate. However, for convenience of explanation, it is assumed that the image capturing device is a camera.

[0022] According to one embodiment, the parking management system of this embodiment can identify vehicles, i.e., recognize vehicle numbers, by using both the images captured by the entrance camera and RE-ID technology to recognize entering vehicles.

[0023] Of course, the image captured by the camera and the RE-ID technology may be applied according to a priority order. For example, the image captured by the camera may be applied with priority over the RE-ID technology. In this case, if the vehicle number is clearly displayed in the image captured by the camera, the RE-ID technology may not be applied, and the RE-ID technology may be applied only if the vehicle number is not clearly displayed in the image.

[0024] However, when no barrier is used, the image captured by the camera alone has limitations, so the parking management system can apply the image captured by the camera and the RE-ID technology to all entering vehicles to identify the vehicles. In this case, the vehicles can be clearly identified regardless of the camera position, shooting angle, vehicle number recognition rate, etc.

[0025] The RE-ID technology is a technology that recognizes an object photographed by one camera as the same object when it is captured by another camera. According to one embodiment, the computing device can analyze the image captured by the entrance camera to detect the vehicle number, and determine whether it is the same vehicle by applying the RE-ID technology as shown in Figures 6 and 7, and identify (recognize) the vehicle by taking into consideration both the detected vehicle number and the results of applying the RE-ID technology.

[0026] For example, the RE-ID technology (re-identification technology) can identify the vehicle model by comparing the vehicle image included in the image captured by the entrance camera with the vehicle image stored in an internal DB or external DB as shown in FIG. 6, and based on the identified vehicle model, can compare the vehicle image included in the image captured by the entrance camera with the vehicle image included in the image captured by an internal camera in the parking lot to determine whether they are the same vehicle.

[0027] That is, the re-identification technology can determine whether the vehicle is the same by comparing the vehicle's exterior shape (exterior information) contained in the image captured by the entrance camera with the vehicle's exterior shape (exterior information) contained in the image captured by the internal camera.

[0028] In particular, the re-identification technique does not only look at the overall shape of the vehicle, but can determine whether it is the same vehicle by comparing the details as shown in Figure 7. Here, the details can be a part of the vehicle itself, but can also be equipment (e.g., a high-pass terminal or a driving recorder) installed or disposed on the vehicle.

[0029] According to an embodiment, the re-identification technique may be learned through machine learning, for example, a deep learning model. For example, the deep learning model may use a triplet loss, which is a loss function. Specifically, when a plurality of images including vehicles are input, as shown in FIG. 8, using the triplet loss, images of the same vehicle may be induced to be located close to each other in an N (an integer equal to or greater than 2) dimensional feature space, and images of different vehicles may be induced to be located far away in the feature space so that they are easily distinguishable. As a result, the image of the vehicle included in the image captured by the entrance camera and the image of the vehicle stored in the DB may be compared quickly and efficiently.

[0030] Although the re-identification technology has been described above as comparing the image of a vehicle contained in the image captured by the entrance camera with the image of a vehicle in the DB, it can also compare the image of a vehicle contained in the image captured by a camera inside the parking lot with the image of a vehicle in the DB.

[0031] When a vehicle leaves, the exit camera recognizes at least one of the vehicle number and the image of the vehicle, and the parking fee can be charged for the parking time of the vehicle. Therefore, the user can leave the vehicle after paying the parking fee through the payment machine, and the barrier on the exit side opens when the parking fee is paid.

[0032] If the entrances are on one passage, there may be only one barrier for entering / exiting. Of course, there may be only an exit camera and a fare machine on the exit side without the barrier. Also, the payment of the fee may not be processed through the payment machine, but may be charged separately to the user later.

[0033] Meanwhile, if there is an internal camera in the parking lot, the vehicle number can be recognized using the internal camera even if the vehicle number cannot be recognized accurately when entering the parking lot. In this case, the vehicle can be identified by considering the time when it arrived at the internal camera after entering and the type of vehicle, and the vehicle number of the identified vehicle can be obtained through the internal camera. In this case, the fee can be settled only with the vehicle number when the vehicle leaves.

[0034] In summary, the parking management system of the present embodiment can accurately identify the entered vehicle by using the intersection of vehicle information (e.g., vehicle type recognition) detected through RE-ID technology as well as vehicle information detected at the entrance. Therefore, even if the vehicle number is not accurately recognized at the entrance, the vehicle can be accurately identified, making it possible to charge parking fees. In this case, the barrier does not need to be present, but can be present for safety reasons.

[0035] FIG. 9 is a block diagram illustrating a schematic configuration of a parking management system according to another embodiment of the present invention, FIG. 10 is a block diagram illustrating a parking management system according to another embodiment of the present invention, and FIGS. 11 to 14 are diagrams illustrating a tracking process according to one embodiment of the present invention.

[0036] 9, the parking management system of this embodiment may include at least one internal camera, an exit camera, a payment machine, and a computing device, in which case the barrier may or may not be present.

[0037] That is, the parking management system may include an internal camera installed inside the parking lot without including an entrance camera, where the internal camera may be installed at a position capable of capturing an image of the parking area. The internal camera recognizes the vehicle number when the vehicle is parked within the parking line (parking area) and counts the parking time from the time the parking is completed.

[0038] According to one embodiment, the computing device can identify a vehicle by applying the intersection of vehicle information detected through the internal camera and vehicle information (e.g., vehicle type) detected using RE-ID technology. Therefore, even when the vehicle is photographed obliquely by the internal camera and the vehicle number is not 100% recognized, the computing device can identify the vehicle, i.e., clearly identify it.

[0039] According to another embodiment, the vehicle number of the vehicle can be recognized using the first internal camera, and the RE-ID technology using the first and second internal cameras can be applied to the vehicle that has entered the parking lot to clearly identify the vehicle.

[0040] According to yet another embodiment, the computing device can use tracking technology to detect the parking and exit of a vehicle and can also recognize the vehicle number. That is, even if the vehicle number is not accurately recognized by the internal camera when parking, the computing device can use the tracking technology to clearly identify and track the vehicle. In this case, there may not be an exit camera as shown in FIG. 10.

[0041] The same camera vehicle tracking technology can determine whether a vehicle has entered or exited the area by comparing the vehicle's position information acquired in the previous frame with the vehicle's position information acquired in the current frame.

[0042] For example, if a vehicle that was present in a previous frame is not found in the current frame, the computing device may determine that the vehicle has left the parking area monitored by the camera.

[0043] As another example, if a vehicle present in the current frame is not present in the previous frame, the computing device may determine that the vehicle has newly entered or parked in the parking area. For example, as shown in FIG 11, if other vehicles are located in the same position in the previous frame and the current frame, and vehicle number 4 is detected in the parking area only in the current frame, the computing device may determine that vehicle number 4 has newly entered and parked in the parking area.

[0044] The computing device can apply such a tracking method to the internal camera, thereby tracking the movement of the vehicle. The computing device can also obtain a vehicle number during the vehicle tracking process and assign a specific number to the vehicle.

[0045] Looking at the flow of such a tracking technology in detail with reference to FIG. 12, the computing device can detect a vehicle by comparing a previous frame with a current frame (detection result), and track the vehicle by performing IoU (Intersection over Union) matching as follows to determine whether the vehicle has parked in the parking area or has left the parking area.

[0046] That is, the computing device can detect the extent to which the vehicle area (e.g., 6435, the vehicle's rectangular box) detected by the AI ​​model occupies the sum of the parking area and the vehicle area, and determine whether the vehicle is parked in the parking area or has left the parking area. Specifically, as shown in the following formula, whether the vehicle is parked can be determined based on the ratio (IOU) of the overlapping area of ​​the parking area and the vehicle area to the sum of the areas.

[0047] It can be seen that the 6435 vehicle parked in Fig. 13 has left the parking area in Fig. 14. It can be seen in Fig. 14 that the IoU for the 6435 vehicle is 0, and the computing device can therefore see that the 6435 vehicle has left the parking area. Meanwhile, the IoU reference value for whether or not a vehicle is parked in a parking area can be set depending on the setting, but if it is 0.7 or more, it can be determined that the vehicle is parked. JPEG2025510615000002.jpg63128

[0048] On the other hand, if the camera is blocked or the AI ​​engine generates an error that the vehicle has not been detected, it may be erroneously determined that the vehicle has left. To prevent this phenomenon, the vehicle may be determined to have left only when the vehicle is continuously tracked through the AI ​​engine, and even if the parking area and the vehicle area do not overlap, it may not be determined that the vehicle has left for a preset time (e.g., about one minute), and it may be determined that the vehicle has left if the parking area and the vehicle area do not overlap even after the preset time.

[0049] According to another embodiment, the computing device may also determine whether the vehicle is parked in or has exited a parking area based on whether the center of the vehicle area is in the parking area.

[0050] The computing device may then remove any tracking results for which IoU matching was not achieved.

[0051] Meanwhile, the computing device may generate new tracking results for detection results for which IoU matching has not been performed, estimate the tracking results using a Kalman filter, and then perform IoU matching again.

[0052] In addition, the computing device may correct the tracking result obtained through IoU matching with a Kalman filter, and then perform IoU matching again.

[0053] According to yet another embodiment, if a vehicle leaving a first parking area enters a second parking area within a preset time, the computing device may determine that the vehicle entering the second parking area is the same as the vehicle leaving the first parking area and assign the same or similar number, in which case no external comparison of the vehicles may be performed.

[0054] In summary, the parking management system of the present invention can use re-identification technology, tracking technology, IoU matching, etc. to identify vehicles. As a result, the parking management system can accurately identify the entering vehicle and charge a parking fee even when the camera cannot accurately recognize the vehicle number.

[0055] FIG. 15 is a diagram illustrating a QR code recognition process according to an embodiment of the present invention, and FIG. 16 is a diagram illustrating a QR code processing process according to an embodiment of the present invention.

[0056] Referring to FIG. 15, in addition to the above-mentioned camera-based vehicle number recognition and re-identification technology, the parking management system of this embodiment can also additionally use an identifier (e.g., a QR code (registered trademark)) that is visible on the floor of the parking area or on a separate device.

[0057] For ease of explanation, the identifier is assumed to be a QR code (registered trademark) below. The identifier is used to allow payment to be made later in case a user who has parked leaves the parking lot without paying the parking fee at the payment machine. In this case, the exit camera and exit barrier do not need to exist. Of course, the parking fee can be paid without the payment machine, so the payment machine does not need to exist either.

[0058] That is, the parking management system can identify vehicles using a camera and re-identification technology and settle parking fees based on information input through a QR code. In this case, parking fee settlement can be additionally made based on information input through a QR code only for vehicles that have not been settled through a payment machine, or parking fees can be settled based only on information input through a QR code without the need for a payment machine.

[0059] Specifically, when a user uses a camera to take a picture of the QR code sign installed on the floor of a parking area as shown in FIG. 15, it is connected to a specific application as shown in FIG. 16, and the application can display information about the parking area where the car is currently parked.

[0060] The user can input a phone number, vehicle number and card number into the application, so that parking fees for the user's vehicle can be automatically charged.

[0061] Of course, there may be cases where the card number is not entered, in which case the parking fee may be charged to the entered phone number.

[0062] According to another embodiment, if the parking fee is not paid by the above-mentioned process, the computing device of the parking management system may transmit billing information to a server such as a parking lot management entity so that the parking lot management entity may collect the parking fee, or may prohibit parking the next time the user visits the parking lot. If parking is prohibited, the computing device may store information about the vehicle for which payment has not been made.

[0063] In order to calculate parking fees, it is necessary to know when a vehicle is parked, when it leaves, and when it moves. For this purpose, the tracking technology described above can be used.

[0064] In summary, the parking management system of the present embodiment can use re-identification technology, tracking technology, IoU matching, information input via QR code (registered trademark), etc. to identify a vehicle. As a result, even if the camera cannot accurately recognize the vehicle number, the parking management system can accurately identify the vehicle that entered and charge the parking fee. In particular, even if the vehicle leaves without paying the parking fee, the parking fee can be automatically paid later.

[0065] Below, we take a closer look at re-identification technology that can operate robustly under a variety of external environmental conditions (light intensity, weather, etc.).

[0066] 17 to 24 are diagrams illustrating a re-identification technique according to another embodiment of the present invention. The parking management system can identify vehicles by taking into account the physical location of the camera or the geographical features of the shooting location in order to make the re-identification technique robust under various external environmental conditions.

[0067] Although such a re-identification technique uses the external similarity of the vehicle to identify whether it is the same vehicle or not, it may be difficult to accurately identify the vehicle only by comparing the external information of the vehicle due to various external environments. Therefore, the parking management system of the present invention can identify the vehicle by considering the physical location of the camera or the geographical features of the shooting location.

[0068] According to one embodiment, when multiple cameras are installed as shown in Fig. 17, a moving vehicle may be seen differently on the cameras. Therefore, the computing device may configure an adjacency matrix according to the physical positions of the cameras belonging to one network, and adjust weights of a matching function used in a re-identification process through the configured adjacency matrix.

[0069] When a camera is installed as shown in Fig. 15, the adjacency matrix of the cameras is as follows. Examining these adjacency matrices, it can be confirmed that Camera A is adjacent to Camera B but not to Camera C, Camera B is adjacent to both Camera A and Camera C, and Camera C is adjacent to Camera B but not to Camera A. Cameras that are adjacent can have a value of "1", and those that are not adjacent can have a value of "0". However, such value assignment can be variously deformed.

[0070] JPEG2025510615000003.jpg32128

[0071] Under such camera installation conditions, assume that as shown in Fig. 18, Vehicle No. 1 is detected by Camera C at time t0 and Vehicle No. 2 is detected by Camera B at time t1 (where t0 < t1). Since Camera B and Camera C are physically adjacent to each other, they will have a value of 1 in the adjacency matrix. To avoid a penalty when calculating the distance (distance, D(f1, f2)) between the feature vectors extracted from Vehicle No. 1 and the feature vectors extracted from Vehicle No. 2 by the re-identification engine, a distance function or a matching function can be configured as follows [Function]. [Function] JPEG2025510615000004.jpg14128

[0072] Here, N represents the dimension of the extracted feature vector, and V represents the similarity matrix. VBC can have a value of 1 if Camera B and Camera C are physically adjacent, and a value of 0 otherwise. λ is a parameter representing the degree of penalty and can have a value greater than 1.

[0073] As another example, assume that as shown in FIG. 19, vehicle No. 1 is detected by camera C at time t0 and vehicle No. 3 is detected by camera A at time t1 (at this time, t0 < t1). Since camera A and camera C are not physically adjacent to each other, they have a value of 0 on the adjacency matrix. As a result, when calculating the distance between the feature vectors of the two vehicles, a weighting value is applied only by the numerical value of, that is, it will be disadvantaged.

[0074] To summarize, when calculating the distance between the feature vectors of vehicles, the computing device can set different weighting values according to the degree of adjacency of the cameras. Specifically, the computing device can increase the probability that a vehicle detected by one camera is matched with a vehicle detected by an adjacent camera, thereby improving the accuracy of re-identification. That is, the computing device can improve the accuracy by imposing spatial constraints on the re-identification algorithm.

[0075] According to another embodiment, the computing device can compare the distances between vehicles based on the modeling result including the geographical information of the space photographed by the camera, thereby improving the accuracy of re-identification. This is to solve the case where it is difficult to perform accurate re-identification by only using the external shape information of the vehicle when performing re-identification in one camera.

[0076] For example, when photographing a two-lane road as shown in FIGS. 20 and 21 with one camera, assume that one lane is blocked by a tree (obstacle) and an object traveling in that lane disappears from the camera and then reappears.

[0077] In such an environment, as shown in FIG. 22, there are vehicle No. 1 detected at time t0 before being blocked by the tree and vehicle No. 2 and vehicle No. 3 detected at time t1 after a certain period of time has passed. If the external shapes of vehicle No. 2 and vehicle No. 3 are similar, it is difficult to obtain an accurate re-identification result only by comparing the external shapes of the vehicles.

[0078] To solve this, additional spatial constraints need to be utilized, but utilizing the commonly used straight-line distance (Euclidean distance) for spatial constraints can lead to the erroneous result that car 2 is more similar to car 1.

[0079] To prevent such errors, the computing device sets virtual lines A and B to model a two-lane road as shown in FIG. 23, and when calculating the spatial distance between two vehicles, the computing device can indirectly calculate the distance between the vehicles using lines A and B rather than directly calculating the distance between the vehicles. For example, as shown in FIG. 24, cars 1 and 3 can be assumed to be located on line A and have a distance of 0 from line A, and car 2 can be assumed to be located on line B and have a distance of 0 from line B.

[0080] When comparing carriages 1 and 3 using lines A and B, carriages 1 and 3 are both located on line A, so their distances to line A should be the same at 0, and there should not be a significant difference in their distances to line B. On the other hand, when comparing carriages 1 and 2, carriage 1 has a distance of 0 to line A, but carriage 2 has a large value, and carriage 1 has a large distance to line B, while carriage 2 has a distance of 0. As a result, the computing device can determine that carriages 1 and 3 are more similar.

[0081] In short, the computing device can indirectly compare the distance between vehicles based on the modeling results that include geographic information of the space captured by the camera, thereby obtaining more accurate re-identification results.

[0082] Of course, although the re-identification process has been described above using lanes, the re-identification process can be performed in consideration of other geographical circumstances, not limited to lanes. For example, the re-identification process can be performed in consideration of two routes on which vehicles travel, instead of lanes.

[0083] FIG. 25 is a flowchart illustrating a parking space guidance process according to an embodiment of the present invention. Referring to FIG. 25, the computing device first performs an object detection algorithm on the image acquired from the camera to obtain the position of the vehicle.

[0084] The computing device can then transform the predefined parking area and the detected vehicle position into a coordinate system based on the camera-real world relationship information, where the distance between two points on the same plane and the actual measured distance in the real world can be the same within an error range in the transformed coordinate system.

[0085] The computing device may then calculate an overlap degree between the detected vehicle and the parking area, thereby detecting a parking space where parking is possible. Specifically, if the overlap degree between the detected vehicle and the parking area is high, it is determined that a vehicle is parked in the parking area, and if the overlap degree between the detected vehicle and the parking area is zero or small, it is determined that a vehicle is not parked in the parking area.

[0086] Next, when calculating parking space, the computing device may provide a function of displaying how many vehicles can be parked in the currently remaining parking space by vehicle type based on the average / standard deviation of the overall width / length by vehicle type (compact SUV, mid-size sedan, etc.).

[0087] FIG. 26 is a block diagram illustrating a computing device in accordance with one embodiment of the present invention. 26, the computing device of this embodiment may include a control unit 2600, a communication unit 2602, a device management unit 2604, a re-identification unit 2606, a tracking unit 2608, a settlement unit 2610, and a storage unit 2612. Furthermore, the computing device may additionally include an identifier unit.

[0088] The communication unit 2602 is a communication connection path to a camera, a barrier, or a payment machine. The device management unit 2604 can manage the operation of cameras, barriers, or payment machines. The re-identification unit 2606 can re-identify a vehicle through RE-ID technology and identify the vehicle by combining image information acquired by a camera with information detected through the re-identification. The tracking unit 2608 may track the vehicle using an internal camera in the parking lot and detect the vehicle number during the tracking process. In particular, the tracking unit 2608 may use the IoU technology for tracking the vehicle.

[0089] The identifier unit receives input of user information through identifier recognition. The payment unit 2610 can perform payment for the specified vehicle only for the parking time. In particular, the payment unit 2610 can automatically settle the parking fee without using a payment machine based on the user information input through the identifier recognition.

[0090] The storage unit 2612 can store various information such as images captured by the camera, re-identification information, tracking results, etc. The control unit 2600 may be capable of overall control of the operation of the components of the computing device.

[0091] Meanwhile, the components of the above-described embodiments can be easily understood from a process perspective, i.e., each component can be understood as a respective process, and the processes of the above-described embodiments can be easily understood from the perspective of the components of the apparatus.

[0092] In addition, the above-described technical contents may be embodied in the form of program instructions that can be executed by various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiments, or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions, such as ROMs, RAMs, flash memories, and the like. Examples of program instructions include not only machine language codes, such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, and the like. A hardware device may be configured to operate as one or more software modules to perform operations of the embodiments, and vice versa.

[0093] The above-described embodiments of the present invention have been disclosed for illustrative purposes, and a person skilled in the art having ordinary knowledge of the present invention may make various modifications, changes, and additions within the spirit and scope of the present invention, and such modifications, changes, and additions should be considered to fall within the scope of the following claims. [Explanation of symbols]

[0094] 2600 Control Unit 2602 Communications Department 2604 Equipment Management Department 2606 Re-identification Department 2608 Tracking Department 2610 Settlement Department 2612 Storage Unit

Claims

1. a first image capture device for recognizing the license plate number of a vehicle at the entrance or inside of the parking lot; A second image capture device installed inside the parking lot; a computing device communicatively coupled to the first image capture device or the second image capture device; The computing device identifies the vehicle by considering vehicle information detected using a re-identification (RE-ID) technique and vehicle information obtained by analyzing an image acquired by at least one of the first and second image acquisition devices; A parking management system characterized in that the re-identification technology is a technology that determines whether a vehicle is the same based on the vehicle's external shape information contained in an image captured by the first image capture device and the vehicle's external shape information contained in an image captured by the second image capture device.

2. the first image capture device is an entrance camera; The re-identification technique detects a vehicle type by comparing an image of the vehicle included in the image acquired by the first image acquisition device with an image of the vehicle stored in a DB, and based on the detected vehicle type, compares an image of the vehicle included in the image acquired by the second image acquisition device with an image of the vehicle included in the image acquired by the first image acquisition device to determine whether they are the same vehicle; The parking management system of claim 1 , wherein the computing device links the vehicle number when it is determined through the re-identification technique that the vehicle is the same.

3. The re-identification technique is trained via a deep learning model that uses a triplet loss loss function, 2. The parking management system of claim 1, wherein, when a plurality of images including vehicles are input, the re-identification technique positions images of the same vehicle relatively close to each other in an N-dimensional feature space (an integer of 2 or more) through the triplet loss, and positions images of different vehicles relatively far from each other in the feature space.

4. The computing device tracks the vehicle via a tracking function; 2. The parking management system according to claim 1, wherein the tracking function determines whether a vehicle has parked in or exited from a parking area by comparing vehicle position information acquired in a previous frame acquired by the second image acquisition device with vehicle position information acquired in a current frame.

5. The tracking function determines whether the vehicle has parked in or exited from a parking area through IoU (Intersection over Union) matching; The IoU matching is a technique for detecting to what extent a vehicle area detected by an AI model occupies the sum of the parking area and the vehicle area, The parking management system according to claim 4, characterized in that if the result value of the IoU matching is equal to or greater than a preset value, it is determined that the vehicle has been parked in the parking area, and if the result value is less than the preset value, it is determined that the vehicle has left the parking area.

6. The parking management system according to claim 4, wherein the tracking function determines whether the vehicle has parked in or exited from the parking area depending on whether the center of the vehicle area detected by the AI ​​model is within the parking area.

7. The computing device configures an adjacency matrix according to physical locations of image capture devices belonging to one network, and adjusts weights of functions used in a re-identification process using the configured adjacency matrix; 2. The parking management system of claim 1, wherein the adjusting of the weight value comprises applying a value included in the adjacency matrix to the function when calculating a distance between the feature vectors of the vehicles acquired in the image acquisition device to assign a weight value. [function] Here, N denotes the dimension of the extracted feature vector, V denotes the similarity matrix, and VBC has a value of 1 if the image capture devices are physically adjacent and 0 otherwise.

8. The parking management system of claim 1, wherein the computing device indirectly compares the distance between vehicles based on a modeling result including geographical information of the space captured by at least one of the first and second image capture devices to identify the vehicles.

9. When a first lane and a second lane exist, the computing device sets virtual A-lines and B-lines on the first lane and the second lane, and indirectly calculates the distance between the two vehicles by utilizing the A-lines and the B-lines when calculating the spatial distance between the two vehicles; A distance between a vehicle located in the first lane and the A line is set to 0, and a distance between a vehicle located in the second lane and the A line is set to a value greater than 0; The parking management system according to claim 8, characterized in that a distance between a vehicle located in the second lane and the B line is set to 0, and a distance between a vehicle located in the first lane and the B line is set to a value greater than 0.

10. 2. The parking management system of claim 1, wherein the computing device detects the position of the vehicle by applying an object detection algorithm to the image captured by the second image capture device, transforms a preset parking area and the position of the detected vehicle into a coordinate system based on camera-real world relationship information, and calculates an overlap degree between the detected vehicle and the parking area based on the transformation to detect an available parking space.

11. a first image capture device for recognizing a license plate number of a vehicle; a second image capture device; and An identifier; a computing device communicatively coupled to the first image capture device or the second image capture device; The computing device identifies the vehicle using at least one of vehicle information obtained by analyzing an image captured by at least one of the first and second image capture devices and vehicle information detected using a re-identification (RE-ID) technique; When a user of the vehicle inputs user information through recognition of the identifier, the computing device charges the user or automatically charges a parking fee based on the input user information; A parking management system characterized in that the re-identification technology is a technology that determines whether a vehicle is the same based on the vehicle's external shape information contained in an image captured by the first image capture device and the vehicle's external shape information contained in an image captured by the second image capture device.

12. the first and second image capture devices are cameras, and the identifier is a QR code (registered trademark); the QR code is visible on a floor or equipment of a parking area, and the user information includes at least one of a vehicle number, a telephone number, and account information; The parking management system of claim 11, wherein the re-identification technology detects a vehicle type by comparing an image of the vehicle included in the image acquired by the first image acquisition device with an image of the vehicle stored in a DB, and based on the detected vehicle type, compares an image of the vehicle included in the image acquired by the second image acquisition device with an image of the vehicle included in the image acquired by the first image acquisition device to determine whether they are the same vehicle.

13. The parking fee is automatically settled based on the user information via the identifier, The parking management system of claim 11, wherein if the parking fee is not paid, the computing device transfers information about the parking fee to a server of a parking lot management entity, or stores information about the vehicle so that parking is prohibited the next time the parking lot is used.

14. The re-identification technique is trained via a deep learning model that uses a triplet loss loss function, 12. The parking management system of claim 11, wherein the re-identification technique, when a plurality of images including vehicles are input, positions images of the same vehicle relatively close to each other in an N-dimensional feature space (an integer of 2 or more) through the triplet loss, and positions images of different vehicles relatively far from each other in the feature space.

15. The computing device configures an adjacency matrix according to physical locations of image capture devices belonging to one network, and adjusts weights of functions used in a re-identification process using the configured adjacency matrix; 12. The parking management system of claim 11, wherein the adjusting of the weight value comprises applying a value included in the adjacency matrix to the function when calculating a distance between the feature vectors of the vehicles acquired in the image acquisition device to assign a weight value. [function] Here, N denotes the dimension of the extracted feature vector, V denotes the similarity matrix, and VBC has a value of 1 if the image capture devices are physically adjacent and 0 otherwise.

16. When a first lane and a second lane exist, the computing device sets virtual A-lines and B-lines on the first lane and the second lane, and indirectly calculates the distance between the two vehicles by utilizing the A-lines and the B-lines when calculating the spatial distance between the two vehicles; A distance between a vehicle located in the first lane and the A line is set to 0, and a distance between a vehicle located in the second lane and the A line is set to a value greater than 0; The parking management system according to claim 11, characterized in that a distance between a vehicle located in the second lane and the B line is set to 0, and a distance between a vehicle located in the first lane and the B line is set to a value greater than 0.

17. 1. A computing device for use in a parking management system, comprising: a communication unit which is a communication connection path with the first image capture device or the second image capture device; a re-identification unit for identifying a vehicle that has entered the parking lot by considering vehicle information detected using a re-identification (RE-ID) technology and vehicle information obtained by analyzing an image acquired by at least one of the first and second image acquisition devices, The re-identification technology is a technology for determining whether the vehicle is the same vehicle based on vehicle exterior information included in an image captured by the first image capture device and vehicle exterior information included in an image captured by the second image capture device.

18. The re-identification technique is trained via a deep learning model that uses a triplet loss loss function, 18. The computing device of claim 17, wherein, when a plurality of images including vehicles are input, the re-identification technique positions images of the same vehicle relatively close to each other in an N-dimensional feature space (N (an integer equal to or greater than 2)) through the triplet loss, and positions images of different vehicles relatively far apart in the feature space.

19. the re-identification unit configures an adjacency matrix according to physical locations of image capture devices belonging to one network, and adjusts weights of functions used in a re-identification process by using the configured adjacency matrix; 18. The computing device of claim 17, wherein the adjusting of the weights comprises applying values ​​included in the adjacency matrix to the function to assign weights when calculating distances between feature vectors of vehicles acquired in the image acquisition device. [function] Here, N denotes the dimension of the extracted feature vector, V denotes the similarity matrix, and VBC has a value of 1 if the image capture devices are physically adjacent and 0 otherwise.

20. The computing device of claim 17, wherein the computing device identifies the vehicles by indirectly comparing distances between vehicles based on a modeling result including geographic information of a space captured by at least one of the image capture devices.

21. 1. A computing device for use in a parking management system, comprising: a communication unit which is a communication connection path with the first image capture device or the second image capture device; an identifier unit that receives user information through identifier recognition; A settlement department; a re-identification unit for identifying a vehicle that has entered the parking lot by using at least one of vehicle information detected using a RE-ID (re-identification) technology and vehicle information obtained by analyzing an image acquired by at least one of the image acquisition devices; When the user of the vehicle inputs user information through the identifier recognition, the settlement unit bills the user for a parking fee based on the input user information or automatically settles the parking fee, The re-identification technology is a technology for determining whether the vehicle is the same vehicle based on vehicle exterior information included in an image captured by the first image capture device and vehicle exterior information included in an image captured by the second image capture device.

22. 22. The computing device of claim 21, wherein if the parking fee is not paid, the computing device transfers information about the parking fee to a public institution's server or stores information about the vehicle so that parking will be prohibited the next time the parking lot is used.

Images