Parking lot management system and parking lot management method using lidar device

Abstract

A parking lot management system according to the present invention may comprise: an image acquisition device that is installed at an entrance to the parking lot and acquires an image of a vehicle passing through the entrance; a plurality of LiDAR devices installed on the perimeter and inside of the parking lot; a plurality of LiDAR processors connected to at least one of the plurality of LiDAR devices; and a parking lot management processor that communicates with the image acquisition device and the plurality of LiDAR processors.

Description

Parking lot management system and parking lot management method using lidar device

[0001] The present invention relates to a parking lot management system and a parking lot management method, and more particularly, to a parking lot management system and a parking lot management method using a lidar device.

[0002]

[0003] A parking lot management system may be a system that manages and supervises parking of vehicles in a parking lot, including identifying the occupancy status of parking slots located in the parking lot, and further provides guidance.

[0004] These parking lot management systems have been developed with a lot of technology in recent years, and in the case of indoor parking lots, they use cameras or embedded sensors to determine the occupancy status of parking slots, and use LED lights to guide people to empty parking slots.

[0005] However, in order to know the occupancy status of parking slots through a camera, a separate structure for installing the camera is required, and in order to know the occupancy status of parking slots through a built-in sensor, etc., a separate built-in sensor must be installed under the parking surface.

[0006] In particular, for cameras, since it is difficult to know the exact 3D position of a parking slot and the exact 3D position of the current vehicle, they must be positioned adjacent to the parking slots to monitor the occupancy status of a limited number of parking slots (e.g., 4 to 6 cars), and in order to implement a parking lot management system using cameras in a parking lot that can accommodate more than 1,000 cars, more than 250 cameras are required, and in the case of an outdoor parking lot, a separate structure is required to install more than 250 cameras.

[0007] Due to these circumstances, the development of parking management systems in outdoor parking lots is delayed compared to indoor parking lots. Implementing parking management systems in outdoor parking lots using cameras and / or embedded sensors is expensive and does not contribute significantly to generating profits as it narrows the parking space itself due to additional, separate structures. Therefore, it is rare for parking management systems to be installed and operated in outdoor parking lots.

[0008] However, lidar devices are devices that use lasers to obtain distance information about the surroundings, and because of their superior precision and resolution and the ability to perceive objects in three dimensions, they are being applied to various fields such as not only automobiles but also drones and aircraft.

[0009] Therefore, when operating a parking lot management system using a lidar device, it is expected that an accurate system can be built at a lower cost than a camera, as accurate 3D location information on parking slots and vehicles can be obtained through the lidar device.

[0010] However, in order to operate a parking lot management system using a lidar device, it is not enough to simply purchase and install a lidar device. In order to implement a practically operable parking lot management system, such as accurately determining whether parking slots are occupied within a parking lot, more specific development of a parking lot management system using a lidar device is necessary.

[0011]

[0012] An object of the present invention is to provide a parking lot management system using a lidar device.

[0013] The problems to be solved by the present invention are not limited to the problems described above, and problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention pertains from this specification and the attached drawings.

[0014]

[0015] According to one embodiment of the present application, a parking lot management system for monitoring the current status of a parking lot comprises: an image acquisition device installed at the entrance of the parking lot and acquiring an image of a vehicle passing through the entrance; a plurality of lidar devices installed on the periphery and inside the parking lot, wherein each of the plurality of lidar devices has an assigned surveillance area; a plurality of lidar processors connected to at least one of the plurality of lidar devices, wherein each of the plurality of lidar processors generates an individual point cloud for the surveillance area of ​​at least one corresponding lidar device; and a parking lot management processor communicating with the image acquisition device and the plurality of lidar processors, wherein the parking lot management processor acquires a vehicle number of a vehicle passing through the entrance of the parking lot, wherein the vehicle number of the vehicle is acquired based on the license plate of the vehicle appearing in the acquired image; and generates a vehicle number list using the acquired vehicle number, wherein the vehicle number list is sorted in the order in which the vehicle numbers are acquired; the parking lot management processor sets an entry surveillance area, wherein: The above-mentioned vehicle entry surveillance area is set to be adjacent to the entrance of the parking lot, but is set to be located within the parking lot, and a vehicle located in the above-mentioned vehicle entry surveillance area is identified based on information obtained from the plurality of lidar processors, and a temporary vehicle ID is assigned to the identified vehicle to generate a temporary vehicle ID list, wherein the temporary vehicle ID list is sorted in the order in which the temporary vehicle IDs are assigned, and the identified vehicle is tracked to obtain a parking slot ID for the parking location where the identified vehicle is parked.A parking lot management system can be provided that matches and stores the parking slot ID and the vehicle number by matching the temporary vehicle ID and the vehicle number corresponding to the temporary vehicle ID based on the vehicle number list and the temporary vehicle ID list.

[0016] According to another embodiment of the present application, a parking lot management system for monitoring the current status of a parking lot includes an image acquisition device installed at the entrance of the parking lot and acquiring an image of a vehicle passing through the entrance, a plurality of lidar devices installed on the periphery and inside the parking lot, wherein each of the plurality of lidar devices has an assigned surveillance area, a plurality of lidar processors connected to at least one of the plurality of lidar devices, wherein each of the plurality of lidar processors generates an individual point cloud for the surveillance area of ​​at least one corresponding lidar device, and a parking lot management processor communicating with the image acquisition device and the plurality of lidar processors, wherein the parking lot management processor acquires a vehicle number of a vehicle passing through the entrance of the parking lot, wherein the vehicle number of the vehicle is acquired based on the license plate of the vehicle appearing in the acquired image, and generates a vehicle number list using the acquired vehicle number, wherein the vehicle number list includes information on the time at which the vehicle number was acquired, and the parking lot management processor sets an entry surveillance area, wherein The above-mentioned vehicle entry surveillance area is set to be adjacent to the entrance of the parking lot, a vehicle located in the above-mentioned vehicle entry surveillance area is identified based on information obtained from the plurality of lidar processors, a temporary vehicle ID is assigned to the identified vehicle to generate a temporary vehicle ID list, wherein the temporary vehicle ID list includes information on the time at which a vehicle corresponding to the temporary vehicle ID was identified, and the identified vehicle is tracked to obtain a parking slot ID for the parking location where the identified vehicle is parked.A parking lot management system may be provided that matches and stores the parking slot ID and the vehicle number by matching the temporary vehicle ID and the vehicle number corresponding to the temporary vehicle ID based on the vehicle number acquisition time information included in the vehicle number list and the vehicle identification time information included in the temporary vehicle ID list.

[0017] The solutions to the problems of the present invention are not limited to the solutions described above, and solutions that are not mentioned can be clearly understood by a person having ordinary skill in the art to which the present invention pertains from this specification and the attached drawings.

[0018]

[0019] According to one embodiment of the present invention, a parking lot management system using a lidar device can be provided.

[0020] The effects of the present invention are not limited to the effects described above, and effects not mentioned can be clearly understood by a person skilled in the art to which the present invention pertains from this specification and the attached drawings.

[0021]

[0022] FIG. 1 is a drawing for explaining a lidar device according to one embodiment.

[0023] Figure 2 is a drawing showing various embodiments of a lidar device.

[0024] FIG. 3 is a diagram for explaining the operation of a lidar device and lidar data according to one embodiment.

[0025] FIG. 4 is a diagram for explaining lidar data according to one embodiment.

[0026] FIG. 5 is a diagram for explaining lidar data according to one embodiment.

[0027] FIG. 6 is a diagram for explaining information included in attribute data according to one embodiment.

[0028] FIG. 7 and FIG. 8 are drawings for explaining the configuration of a parking lot management system according to one embodiment.

[0029] FIG. 9 is a drawing for explaining an operation method of a parking lot management system using a lidar device according to one embodiment.

[0030] FIG. 10 is a drawing for explaining generation of parking line detection information according to one embodiment.

[0031] FIG. 11 is a diagram illustrating a method for evaluating the validity of a monitoring point cloud according to one embodiment.

[0032] Fig. 12 is a drawing for explaining a method for processing a point cloud according to one embodiment.

[0033] FIG. 13 is a diagram illustrating an operation method of a lidar processor for generating occupancy status of a parking slot according to one embodiment.

[0034] FIG. 14 is a diagram illustrating an operation method of a parking lot management processor for generating occupancy status of a parking slot according to one embodiment.

[0035] FIG. 15 and FIG. 16 are drawings for explaining the configuration of a parking lot management system according to one embodiment.

[0036] FIG. 17 is a drawing for explaining an operation method of a parking lot management processor included in a parking lot management system according to one embodiment.

[0037] Fig. 18 is a drawing for explaining an entrance surveillance area according to one embodiment.

[0038]

[0039] Since the embodiments described in this specification are intended to clearly explain the idea of ​​the present invention to a person having ordinary skill in the art to which the present invention pertains, the present invention is not limited to the embodiments described in this specification, and the scope of the present invention should be interpreted to include modified or altered examples that do not depart from the idea of ​​the present invention.

[0040] The terms used in this specification have been selected from widely used terms, taking into account the functions of the present invention. However, these terms may vary depending on the intentions of those skilled in the art, precedents, or the emergence of new technologies. However, if a specific term is defined and used with an arbitrary meaning, the meaning of that term will be described separately. Therefore, the terms used in this specification should be interpreted based on the actual meaning of the term and the overall content of this specification, rather than simply the name of the term.

[0041] The drawings attached to this specification are intended to facilitate explanation of the present invention, and the shapes depicted in the drawings may be exaggerated as necessary to help understand the present invention, and therefore the present invention is not limited by the drawings.

[0042] When an element or layer described herein is referred to as being “on” or “on” another element or layer, it may include not only directly on top of the other element or layer, but also cases where there are other layers or other components interposed therebetween.

[0043] Throughout this specification, identical reference numbers may, in principle, represent identical components.

[0044] The numbers (e.g., first, second, etc.) used in the description of this specification can be understood as identification symbols to distinguish one component from another.

[0045] The suffixes "module" and "part" used for components in the description of this specification are used or used interchangeably depending on the ease of writing the specification, and may not have distinct meanings or roles in themselves.

[0046] In this specification, if it is determined that a detailed description of the composition or function of a known document related to the present invention may obscure the gist of the present invention, a detailed description thereof will be omitted as necessary.

[0047] According to one embodiment of the present invention, a parking lot management system for monitoring the current status of a parking lot comprises: an image acquisition device installed at the entrance of the parking lot and acquiring an image of a vehicle passing through the entrance; a plurality of lidar devices installed on the periphery and inside the parking lot, wherein each of the plurality of lidar devices has an assigned surveillance area; a plurality of lidar processors connected to at least one of the plurality of lidar devices, wherein each of the plurality of lidar processors generates an individual point cloud for the surveillance area of ​​at least one corresponding lidar device; and a parking lot management processor communicating with the image acquisition device and the plurality of lidar processors, wherein the parking lot management processor acquires a vehicle number of a vehicle passing through the entrance of the parking lot, wherein the vehicle number of the vehicle is acquired based on the license plate of the vehicle appearing in the acquired image; and generates a vehicle number list using the acquired vehicle number, wherein the vehicle number list is sorted in the order in which the vehicle numbers are acquired; the parking lot management processor sets an entry surveillance area, wherein: An entrance monitoring area is set to be adjacent to the entrance of the parking lot, but is set to be located within the parking lot, a vehicle located in the entrance monitoring area is identified based on information obtained from the plurality of lidar processors, a temporary vehicle ID is assigned to the identified vehicle to generate a temporary vehicle ID list, and the temporary vehicle ID list is sorted in the order in which the temporary vehicle IDs are assigned, and the identified vehicle is tracked to obtain a parking slot ID for the parking location where the identified vehicle is parked.A parking lot management system can be provided that matches and stores the parking slot ID and the vehicle number by matching the temporary vehicle ID and the vehicle number corresponding to the temporary vehicle ID based on the vehicle number list and the temporary vehicle ID list.

[0048] Here, identifying a vehicle located in the entrance surveillance area based on information obtained from the plurality of lidar processors may include obtaining individual point clouds from the plurality of lidar processors, generating an integrated point cloud for the entire parking lot based on the obtained individual point clouds, and identifying a vehicle located in the entrance surveillance area based on the integrated point cloud.

[0049] Here, identifying a vehicle located in the entrance surveillance area based on information acquired from the plurality of lidar processors may include acquiring some point data included in individual point clouds from the plurality of lidar processors, generating an integrated point cloud for the entire parking lot based on the acquired point data, and identifying a vehicle located in the entrance surveillance area based on the integrated point cloud.

[0050] Here, identifying a vehicle located in the entrance surveillance area based on information acquired from the plurality of lidar processors may include acquiring center position information and bounding box information corresponding to some point data included in individual point clouds from the plurality of lidar processors, positioning the acquired center position information and bounding box information on an integrated coordinate system, and identifying a vehicle located in the entrance surveillance area based on the center position information and bounding box information located on the integrated coordinate system.

[0051] Here, the parking lot entrance includes a first entrance and a second entrance, and the parking lot management processor obtains the vehicle number of a vehicle passing through the first entrance of the parking lot, and creates a first vehicle number list for the first entrance using the obtained vehicle number, obtains the vehicle number of a vehicle passing through the second entrance of the parking lot, and creates a second vehicle number list for the second entrance using the obtained vehicle number, and the parking lot management processor sets a first entry surveillance area corresponding to the first entrance, and identifies a vehicle located in the first entry surveillance area based on information obtained from the plurality of lidar processors and assigns a temporary vehicle ID to generate a first temporary vehicle ID list corresponding to the first entrance, and sets a second entry surveillance area corresponding to the second entrance, and identifies a vehicle located in the second entry surveillance area based on information obtained from the plurality of lidar processors and assigns a temporary vehicle ID to generate a second temporary vehicle ID list corresponding to the second entrance, wherein the parking lot management processor sets the first vehicle number list and the first temporary vehicle ID list. Based on the ID list, the temporary vehicle ID of the first temporary vehicle ID list and the vehicle number of the first vehicle number list can be matched, and based on the second vehicle number list and the second temporary vehicle ID list, the temporary vehicle ID of the second temporary vehicle ID list and the vehicle number of the second vehicle number list can be matched.

[0052] Here, matching of the temporary vehicle ID and the vehicle number corresponding to the temporary vehicle ID can be performed based on the order of the temporary vehicle ID in the temporary vehicle ID list and the order of the vehicle number in the vehicle number list.

[0053] Here, the vehicle number list includes vehicle numbers and vehicle number acquisition time information, the temporary vehicle ID list includes a temporary vehicle ID and information on the time at which a vehicle corresponding to the temporary vehicle ID was identified, and matching of the temporary vehicle ID and the vehicle number corresponding to the temporary vehicle ID can be performed by further considering information on the time at which a vehicle included in the temporary vehicle ID list was identified and information on the time at which a vehicle number included in the vehicle number list was acquired.

[0054] Here, the surveillance area of ​​at least one of the plurality of lidar devices may be positioned to include the entrance surveillance area.

[0055]

[0056] According to another embodiment of the present application, a parking lot management system for monitoring the current status of a parking lot includes an image acquisition device installed at the entrance of the parking lot and acquiring an image of a vehicle passing through the entrance, a plurality of lidar devices installed on the periphery and inside the parking lot, wherein each of the plurality of lidar devices has an assigned surveillance area, a plurality of lidar processors connected to at least one of the plurality of lidar devices, wherein each of the plurality of lidar processors generates an individual point cloud for the surveillance area of ​​at least one corresponding lidar device, and a parking lot management processor communicating with the image acquisition device and the plurality of lidar processors, wherein the parking lot management processor acquires a vehicle number of a vehicle passing through the entrance of the parking lot, wherein the vehicle number of the vehicle is acquired based on the license plate of the vehicle appearing in the acquired image, and generates a vehicle number list using the acquired vehicle number, wherein the vehicle number list includes information on the time at which the vehicle number was acquired, and the parking lot management processor sets an entry surveillance area, wherein The above-mentioned vehicle entry surveillance area is set to be adjacent to the entrance of the parking lot, a vehicle located in the above-mentioned vehicle entry surveillance area is identified based on information obtained from the plurality of lidar processors, a temporary vehicle ID is assigned to the identified vehicle to generate a temporary vehicle ID list, wherein the temporary vehicle ID list includes information on the time at which a vehicle corresponding to the temporary vehicle ID was identified, and the identified vehicle is tracked to obtain a parking slot ID for the parking location where the identified vehicle is parked.A parking lot management system may be provided that matches and stores the parking slot ID and the vehicle number by matching the temporary vehicle ID and the vehicle number corresponding to the temporary vehicle ID based on the vehicle number acquisition time information included in the vehicle number list and the vehicle identification time information included in the temporary vehicle ID list.

[0057] Here, identifying a vehicle located in the entrance surveillance area based on information obtained from the plurality of lidar processors may include obtaining individual point clouds from the plurality of lidar processors, generating an integrated point cloud for the entire parking lot based on the obtained individual point clouds, and identifying a vehicle located in the entrance surveillance area based on the integrated point cloud.

[0058] Here, identifying a vehicle located in the entrance surveillance area based on information acquired from the plurality of lidar processors may include acquiring some point data included in individual point clouds from the plurality of lidar processors, generating an integrated point cloud for the entire parking lot based on the acquired point data, and identifying a vehicle located in the entrance surveillance area based on the integrated point cloud.

[0059] Here, identifying a vehicle located in the entrance surveillance area based on information acquired from the plurality of lidar processors may include acquiring center position information and bounding box information corresponding to some point data included in individual point clouds from the plurality of lidar processors, positioning the acquired center position information and bounding box information on an integrated coordinate system, and identifying a vehicle located in the entrance surveillance area based on the center position information and bounding box information located on the integrated coordinate system.

[0060] Here, the parking lot entrance includes a first entrance and a second entrance, and the parking lot management processor obtains the vehicle number of a vehicle passing through the first entrance of the parking lot, and creates a first vehicle number list for the first entrance using the obtained vehicle number, obtains the vehicle number of a vehicle passing through the second entrance of the parking lot, and creates a second vehicle number list for the second entrance using the obtained vehicle number, and the parking lot management processor sets a first entry surveillance area corresponding to the first entrance, and identifies a vehicle located in the first entry surveillance area based on information obtained from the plurality of lidar processors and assigns a temporary vehicle ID to generate a first temporary vehicle ID list corresponding to the first entrance, and sets a second entry surveillance area corresponding to the second entrance, and identifies a vehicle located in the second entry surveillance area based on information obtained from the plurality of lidar processors and assigns a temporary vehicle ID to generate a second temporary vehicle ID list corresponding to the second entrance, wherein the parking lot management processor sets the first vehicle number list and the first temporary vehicle ID list. Based on the ID list, the temporary vehicle ID of the first temporary vehicle ID list and the vehicle number of the first vehicle number list can be matched, and based on the second vehicle number list and the second temporary vehicle ID list, the temporary vehicle ID of the second temporary vehicle ID list and the vehicle number of the second vehicle number list can be matched.

[0061] Here, the surveillance area of ​​at least one of the plurality of lidar devices may be positioned to include the entrance surveillance area.

[0062]

[0063] I. [Parking Lot Management System Using Lidar Device]

[0064] 1. [Overview of LiDAR devices]

[0065] Below, a lidar device according to the present invention is described.

[0066] However, the LiDAR device described in this specification can be understood as a concept that includes various devices that measure distance using lasers, and can be understood as a concept that includes, for example, LiDAR (Light Detection And Ranging), TOF sensor (Time-of-Flight sensor), etc., but is not limited thereto.

[0067] LiDAR (Light Detection And Ranging Device) is a device that uses lasers to measure the distance between the LiDAR device and an object.

[0068] More specifically, a lidar device is a device that outputs a laser and detects the laser reflected from an object to measure the distance between the lidar device and the object.

[0069] In general, to measure the distance from a lidar device to a target object, the time-of-flight (TOF) for the flight path of the laser output from the lidar device that travels back and forth between the lidar device and the target object is used.

[0070] A LiDAR device is a device that uses a laser to detect the distance to an object and the position of the object. For example, a LiDAR device can output a laser, and when the output laser is reflected from an object, the device can receive the reflected laser to measure the distance between the object and the LiDAR device and the position of the object. At this time, the distance and position of the object can be expressed through a coordinate system. For example, the distance and position of the object can be expressed in a spherical coordinate system (r, , φ) can be expressed as. However, it is not limited to this, and the rectangular coordinate system (X, Y, Z) or the cylindrical coordinate system (r, , z) can be expressed as such.

[0071] Additionally, at this time, the target may mean at least one object, but is not limited thereto, and may also mean a part of an object for reflecting at least a part of the laser output from the lidar device.

[0072] Additionally, a lidar device according to one embodiment may utilize a laser output from the lidar device and reflected from the object to measure the distance to the object.

[0073] For example, a lidar device according to one embodiment may utilize the time of flight (TOF) of a laser from the time the laser is output until it is detected to measure the distance to an object.

[0074] For a more specific example, a lidar device according to one embodiment can measure the distance to an object by using the difference between a time value based on the output time of an output laser and a time value based on the detected time of a laser reflected from an object and detected.

[0075] At this time, a time value based on the output time of the laser can be obtained based on a control unit included in a lidar device according to one embodiment.

[0076] For example, the time value based on the laser output time may be obtained based on the generation time of a trigger signal generated by a control unit included in a lidar device according to one embodiment, but is not limited thereto.

[0077] Additionally, a time value based on the output time of the laser can be obtained based on a laser output unit included in a lidar device according to one embodiment.

[0078] For example, a time value based on the output time of the laser may be obtained by detecting the operation of a laser output unit included in a lidar device according to one embodiment, but is not limited thereto.

[0079] At this time, detection of the operation of the laser output unit may mean detection of the flow of current of the laser output unit, change in electric field, etc., but is not limited thereto.

[0080] Additionally, a time value based on the output time of the laser can be obtained based on a detector unit included in a lidar device according to one embodiment.

[0081] For example, the time value based on the laser output time may be obtained based on the time value at which the detector unit included in the lidar device according to one embodiment detects a laser that is not reflected from the target object, but is not limited thereto.

[0082] At this time, a reference optical path may be provided for the laser output from the laser output unit to be received by the detector unit, but is not limited thereto.

[0083] Additionally, a time value based on the detected time of the laser reflected from the target object can be obtained based on a detector unit included in a lidar device according to one embodiment.

[0084] For example, a time value based on the detected time of a laser reflected from the target object may be obtained based on a time value of a laser reflected from the target object detected by a detector unit included in a lidar device according to one embodiment, but is not limited thereto.

[0085]

[0086] In addition, the lidar device according to one embodiment may use, but is not limited to, a triangulation method, an interferometry method, a phase shift measurement method, etc. in addition to the time of flight to measure the distance to the target object.

[0087]

[0088] FIG. 1 is a drawing for explaining a lidar device according to one embodiment.

[0089] Referring to FIG. 1, a lidar device (1000) according to one embodiment may include a laser output unit (100).

[0090] At this time, the laser output unit (100) according to one embodiment can generate or output a laser.

[0091] Additionally, the laser output unit (100) according to one embodiment may include one or more laser output elements.

[0092] For example, the laser output unit (100) according to one embodiment may include a single laser output element, or may include a plurality of laser output elements.

[0093] In addition, the laser output unit (100) according to one embodiment may be configured as an array in which a plurality of laser output elements are arranged in an array form, but is not limited thereto.

[0094] For example, the laser output unit (100) according to one embodiment may be implemented as a VCSEL array in which a plurality of VCSELs (Vertical Cavity Surface Emitting Lasers) are arranged in an array form, but is not limited thereto.

[0095] In addition, the laser output unit (100) according to one embodiment may include a laser output element such as a laser diode (LD), a solid-state laser, a high power laser, a light entitling diode (LED), a vertical cavity surface emitting laser (VCSEL), an external cavity diode laser (ECDL), etc., but is not limited thereto.

[0096] Additionally, the wavelength of the laser output from the laser output unit (100) according to one embodiment may be located within a specific wavelength range.

[0097] For example, the wavelength of the laser output from the laser output unit (100) according to one embodiment may be located in the 905 nm band, may be located in the 940 nm band, or may be located in the 1550 nm band, but is not limited thereto.

[0098] At this time, the wavelength band may mean a band within a certain range based on the center wavelength.

[0099] For example, a 905 nm band may mean a band within a range of 10 nm difference based on 905 nm, a 940 nm band may mean a band within a range of 10 nm difference based on 940 nm, and a 1550 nm band may mean a band within a range of 10 nm difference based on 1550 nm, but is not limited thereto.

[0100] Additionally, the wavelength of the laser output from the laser output unit (100) according to one embodiment may be located in various wavelength ranges.

[0101] For example, the wavelength of the first laser output from the first laser output element included in the laser output unit (100) according to one embodiment may be located in the 905 nm band, and the wavelength of the second laser output from the second laser output element included in the laser output unit (100) according to one embodiment may be located in the 1550 nm band, but is not limited thereto.

[0102] Additionally, the wavelength of the laser output from the laser output unit (100) according to one embodiment may be within a specific wavelength range but may be different wavelengths.

[0103] For example, the wavelength of the first laser output from the first laser output element included in the laser output unit (100) according to one embodiment may be located in the 940 nm band and may have a wavelength of 939 nm, and the wavelength of the second laser output from the second laser output element included in the laser output unit (100) according to one embodiment may be located in the 940 nm band and may have a wavelength of 943 nm, but is not limited thereto.

[0104]

[0105] Referring again to FIG. 1, a lidar device (1000) according to one embodiment may include an optical unit (200).

[0106] At this time, the optical part may be expressed in various ways, such as a steering part, a scanning part, etc., for the purpose of explaining the present invention, but is not limited thereto.

[0107] According to one embodiment, the optical unit (200) may function to change the flight path of the laser.

[0108] For example, the optical unit (200) according to one embodiment may function to change the flight path of the laser output from the laser output unit (100), and when the laser output from the laser output unit (100) is reflected from the object, may function to change the flight path of the laser reflected from the object, but is not limited thereto.

[0109] Additionally, the optical unit (200) according to one embodiment can function to change the flight path of the laser by reflecting the laser.

[0110] For example, the optical unit (200) according to one embodiment may function to change the flight path by reflecting the laser output from the laser output unit (100), and when the laser output from the laser output unit (100) is reflected from an object, the optical unit (200) may function to change the flight path by reflecting the laser reflected from the object, but is not limited thereto.

[0111] At this time, the optical unit (200) according to one embodiment may include at least one optical means among various optical means for reflecting a laser.

[0112] For example, the optical unit (200) according to one embodiment may include at least one optical means among optical means such as a mirror, a resonance scanner, a MEMS mirror, a VCM (Voice Coil Motor), a polygonal mirror, a rotating mirror, or a Galvano mirror, but is not limited thereto.

[0113] Additionally, the optical unit (200) according to one embodiment can change the flight path of the laser by refracting the laser.

[0114] For example, the optical unit (200) according to one embodiment may function to change the flight path by refracting the laser output from the laser output unit (100), and when the laser output from the laser output unit (100) is reflected from an object, may function to change the flight path by refracting the laser reflected from the object, but is not limited thereto.

[0115] At this time, the optical unit (200) according to one embodiment may include at least one optical means among various optical means for refracting a laser.

[0116] For example, the optical unit (200) according to one embodiment may include at least one optical means such as a lens, a prism, a micro lens, a microfluidic lens, or a metasurface, but is not limited thereto.

[0117] Additionally, the optical unit (200) according to one embodiment can change the flight path of the laser by changing the phase of the laser.

[0118] For example, the optical unit (200) according to one embodiment may function to change the phase of the laser output from the laser output unit (100) to change the flight path, and when the laser output from the laser output unit (100) is reflected from an object, may function to change the phase of the laser reflected from the object to change the flight path, but is not limited thereto.

[0119] At this time, the optical unit (200) according to one embodiment may include at least one optical means among various optical means for changing the phase of the laser.

[0120] For example, the optical unit (200) according to one embodiment may include at least one optical means such as an optical phased array (OPA), a meta lens, or a meta surface, but is not limited thereto.

[0121] Additionally, the optical unit (200) according to one embodiment may include two or more optical units.

[0122] For example, the optical unit (200) according to one embodiment may include, but is not limited to, a transmitting optic unit for irradiating a laser output from a laser output unit (100) according to one embodiment to a scan area of ​​a lidar device and a receiving optic unit for transmitting a laser reflected from a target to a detector unit (300).

[0123] In addition, for example, the optical unit (200) according to one embodiment may include a first optical unit for changing the flight path of the laser output from the laser output unit (100) according to one embodiment in the direction of the first group and a second optical unit for changing the flight path of the laser output from the laser output unit (100) according to one embodiment in the direction of the second group, but is not limited thereto.

[0124] In addition, in addition to the examples described above, the optical unit (200) according to one embodiment may be provided in a combination of various configurations to expand the scan area of ​​the lidar device using the laser output from the laser output unit (100) according to one embodiment and transmit the laser reflected from the target object to the detector unit (300) according to one embodiment.

[0125]

[0126] Referring again to FIG. 1, a lidar device (100) according to one embodiment may include a detector unit (300).

[0127] At this time, the detector unit may be expressed in various ways as a light receiving unit, a receiving unit, a sensor unit, etc., for the purpose of explaining the present invention, but is not limited thereto.

[0128] According to one embodiment, the detector unit (300) can function to detect a laser.

[0129] For example, the detector unit (300) according to one embodiment can detect a laser reflected from an object located within a scan area of ​​the lidar device (100) according to one embodiment.

[0130] Additionally, the detector unit (300) according to one embodiment may be arranged to receive a laser and may function to generate an electrical signal based on the received laser.

[0131] For example, the detector unit (300) according to one embodiment may be arranged to receive a laser reflected from an object located within a scan area of ​​the lidar device (100) according to one embodiment, and may generate an electrical signal based on the laser.

[0132] At this time, the detector unit (300) according to one embodiment may be arranged to receive a laser reflected from an object located within a scan area of ​​the lidar device (100) according to one embodiment through at least one optical means, and the at least one optical means may be included in the above-described optical unit and may include an optical filter or the like, but is not limited thereto.

[0133] Additionally, the detector unit (300) according to one embodiment can generate laser detection information based on the generated electrical signal.

[0134] For example, the detector unit (300) according to one embodiment may generate laser detection information by comparing a predetermined threshold value with the rising edge, falling edge, or median of the rising edge and falling edge of the generated electrical signal, but is not limited thereto.

[0135] In addition, for example, the detector unit (300) according to one embodiment may generate histogram data corresponding to the detection information of the laser based on the generated electrical signal, but is not limited thereto.

[0136] Additionally, the detector unit (300) according to one embodiment can determine the laser detection time based on the detection information of the generated laser.

[0137] For example, the detector unit (300) according to one embodiment may determine the detection point of the laser based on the detection information of the generated laser based on the rising edge of the generated electrical signal, may determine the detection point of the laser based on the detection information of the generated laser based on the falling edge of the generated electrical signal, and may determine the detection point of the laser based on the detection information of the generated laser based on the rising edge of the generated electrical signal and the detection information of the generated laser based on the falling edge, but is not limited thereto.

[0138] In addition, for example, the detector unit (300) according to one embodiment may determine the detection point of the laser based on histogram data generated based on the generated electrical signal, but is not limited thereto.

[0139] For a more specific example, the detector unit (300) according to one embodiment may determine the detection point of the laser based on, but is not limited to, the peak of the generated histogram data, the judgment of the rising edge and the falling edge based on a predetermined value, etc.

[0140] At this time, the histogram data may be generated based on an electrical signal generated from a detector unit (300) according to one embodiment for at least one scan cycle.

[0141] Additionally, the detector unit (300) according to one embodiment may include at least one detector element among various detector elements.

[0142] For example, the detector unit (300) according to one embodiment may include at least one detector element among detector elements such as a PN photodiode, a phototransistor, a PIN photodiode, an APD (Avalanche Photodiode), a SPAD (Single-photon avalanche diode), SiPM (Silicon PhotoMultipliers), a comparator, a CMOS (Complementary metal-oxide-semiconductor), or a CCD (charge coupled device), but is not limited thereto.

[0143] Additionally, the detector unit (300) according to one embodiment may include one or more detector elements.

[0144] For example, the detector unit (300) according to one embodiment may include a single detector element or may include a plurality of detector elements.

[0145] In addition, the detector unit (300) according to one embodiment may be configured as an array in which a plurality of detector elements are arranged in an array form, but is not limited thereto.

[0146] For example, the detector unit (300) according to one embodiment may be implemented as a SPAD array in which a plurality of SPADs (Single Photon Avalanche Diodes) are arranged in an array form, but is not limited thereto.

[0147]

[0148] Referring again to FIG. 1, a lidar device (1000) according to one embodiment may include a control unit (400).

[0149] At this time, the control unit may be expressed in various ways, such as a controller, etc., to explain the present invention, but is not limited thereto.

[0150] According to one embodiment, a control unit (400) can control the operation of a laser output unit (100), an optical unit (200), or a detector unit (300).

[0151] Additionally, the control unit (400) according to one embodiment can control the operation of the laser output unit (100).

[0152] For example, the control unit (400) can control the output timing of the laser output from the laser output unit (100). In addition, the control unit (400) can control the power of the laser output from the laser output unit (100). In addition, the control unit (400) can control the pulse width of the laser output from the laser output unit (100). In addition, the control unit (400) can control the cycle of the laser output from the laser output unit (100). In addition, when the laser output unit (100) includes a plurality of laser output elements, the control unit (400) can control the laser output unit (100) so that some of the plurality of laser output elements are operated.

[0153] Additionally, the control unit (400) according to one embodiment can control the operation of the optical unit (200).

[0154] For example, the control unit (400) can control the operating speed of the optical unit (200). Specifically, if the optical unit (200) includes a rotational mirror, the rotational speed of the rotational mirror can be controlled, and if the optical unit (200) includes a MEMS mirror, the repetition cycle of the MEMS mirror can be controlled, but is not limited thereto.

[0155] Additionally, for example, the control unit (400) can control the degree of operation of the optical unit (200). Specifically, when the optical unit (200) includes a MEMS mirror, the operating angle of the MEMS mirror can be controlled, but is not limited thereto.

[0156] Additionally, the control unit (400) according to one embodiment can control the operation of the detector unit (300).

[0157] For example, the control unit (400) can control the sensitivity of the detector unit (300). Specifically, the control unit (400) can control the sensitivity of the detector unit (300) by adjusting a predetermined threshold value, but is not limited thereto.

[0158] In addition, for example, the control unit (400) can control the operation of the detector unit (300). Specifically, the control unit (400) can control the On / Off of the detector unit (300), and when the control unit (300) includes a plurality of sensor elements, the control unit (400) can control the operation of the detector unit (300) so that some of the plurality of sensor elements are operated.

[0159] Additionally, the control unit (400) according to one embodiment can generate laser detection information based on an electrical signal generated from the detector unit (300).

[0160] For example, the control unit (400) according to one embodiment may generate laser detection information by comparing a predetermined threshold value with the rising edge, falling edge, or median of the rising edge and falling edge of the electrical signal generated from the detector unit (300), but is not limited thereto.

[0161] In addition, for example, the control unit (400) according to one embodiment may generate histogram data corresponding to the detection information of the laser based on the electrical signal generated from the detector unit (300), but is not limited thereto.

[0162] Additionally, the control unit (400) according to one embodiment can determine the laser detection time based on the laser detection information generated from the detector unit (300).

[0163] For example, the control unit (400) according to one embodiment may determine the detection point of the laser based on the detection information of the laser generated based on the rising edge of the electrical signal generated from the detector unit (300), may determine the detection point of the laser based on the detection information of the laser generated based on the falling edge of the electrical signal generated, and may determine the detection point of the laser based on the detection information of the laser generated based on the rising edge of the electrical signal generated and the detection information of the laser generated based on the falling edge, but is not limited thereto.

[0164] In addition, for example, the control unit (400) according to one embodiment may determine the detection point of the laser based on histogram data generated based on the electrical signal generated from the detector unit (300), but is not limited thereto.

[0165] For a more specific example, the control unit (400) according to one embodiment may determine the detection point of the laser based on the peak of the histogram data generated from the detector unit (300), the judgment of the rising edge and falling edge based on a predetermined value, etc., but is not limited thereto.

[0166] At this time, the histogram data may be generated based on an electrical signal generated from a detector unit (300) according to one embodiment for at least one scan cycle.

[0167] Additionally, the control unit (400) according to one embodiment can obtain distance information to the target based on the detection point of the determined laser.

[0168] For example, the control unit (400) according to one embodiment can obtain distance information to the target based on the determined output time of the laser and the determined detection time of the laser, but is not limited thereto.

[0169]

[0170] Figure 2 is a drawing showing various embodiments of a lidar device.

[0171] Referring to (a) of FIG. 2, a lidar device according to one embodiment may include a laser output unit (110), an optical unit (210), and a detector unit (310). The optical unit (210) may include a nodding mirror (211) that nods within a preset range and a multi-faceted mirror (212) that rotates around at least one axis, but is not limited thereto.

[0172] At this time, since the above-described contents can be applied to the laser output unit (110), the optical unit (210), and the detector unit (310), redundant descriptions will be omitted, and (a) of FIG. 2 is a diagrammatic drawing that is simply used to explain one embodiment among various embodiments of the lidar device, and various embodiments of the lidar device are not limited to (a) of FIG. 2.

[0173] In addition, referring to (b) of FIG. 2, a lidar device according to one embodiment may include a laser output unit (120), an optical unit (220), and a detector unit (320), and the optical unit (220) may include at least one lens (221) capable of collimating and steering a laser output from the laser output unit (120) and a multi-faceted mirror (222) that rotates around at least one axis, but is not limited thereto.

[0174] At this time, since the above-described contents can be applied to the laser output unit (120), the optical unit (220), and the detector unit (320), redundant descriptions will be omitted, and (b) of FIG. 2 is a diagrammatic drawing that is simply used to explain one embodiment among various embodiments of the lidar device, and various embodiments of the lidar device are not limited to (b) of FIG. 2.

[0175] In addition, referring to (c) of FIG. 2, a lidar device according to one embodiment may include a laser output unit (130), an optical unit (230), and a detector unit (330), and the optical unit (230) may include at least one lens (231) capable of collimating and steering a laser output from the laser output unit (130) and at least one lens (232) capable of transmitting a laser reflected from a target object to the detector unit (330), but is not limited thereto.

[0176] At this time, since the above-described contents can be applied to the laser output unit (130), the optical unit (230), and the detector unit (330), redundant descriptions will be omitted, and (c) of FIG. 2 is a diagrammatic drawing that is simply used to explain one embodiment among various embodiments of the lidar device, and various embodiments of the lidar device are not limited to (c) of FIG. 2.

[0177] In addition, referring to (d) of FIG. 2, a lidar device according to one embodiment may include a laser output unit (140), an optical unit (240), and a detector unit (340), and the optical unit (240) may include at least one lens (241) capable of collimating and steering a laser output from the laser output unit (130) and at least one lens (242) capable of transmitting a laser reflected from a target object to the detector unit (340), but is not limited thereto.

[0178] At this time, since the above-described contents can be applied to the laser output unit (140), the optical unit (240), and the detector unit (340), redundant descriptions will be omitted, and (d) of FIG. 2 is a diagrammatic drawing that is simply used to explain one embodiment among various embodiments of the lidar device, and various embodiments of the lidar device are not limited to (d) of FIG. 2.

[0179]

[0180] FIG. 3 is a diagram for explaining the operation of a lidar device and lidar data according to one embodiment.

[0181] Referring to FIG. 3, a lidar device (1000) according to one embodiment includes a laser output unit for outputting a laser and a detector unit for detecting a laser. Descriptions of the laser output unit and the detector unit have been described above, and any redundant descriptions will be omitted.

[0182] In addition, referring to FIG. 3, a data processing unit according to one embodiment can obtain lidar data (1200) based on a laser detected by the lidar device (1000).

[0183] At this time, the data processing unit may be included in the lidar device (1000), and may be included in the control unit of the lidar device (1000) described above, but is not limited thereto, and may be positioned to be connected to the lidar device (1000) through at least one communication method and obtain a signal generated from the detector unit included in the lidar device (1000).

[0184] In addition, referring to FIG. 3, a lidar device (1000) according to one embodiment can form a field of view (1100) by irradiating a laser, and can detect a laser reflected within the field of view (1100) to obtain lidar data (1200).

[0185] At this time, the field of view (1100) of the lidar device (1000) may mean an area where the laser is irradiated or an area where the laser can be detected, but is not limited thereto.

[0186] In addition, the above lidar data (1200) may refer to various types of data obtained from the lidar device (1000), and may refer to, for example, point data, point cloud, frame data, etc. obtained from the lidar device (1000), but is not limited thereto.

[0187] At this time, the point data may be data including distance information, location information, etc., and the point cloud may mean cluster data of the point data, but is not limited thereto.

[0188] Additionally, the frame data may refer to a group of the point data, but is not limited thereto.

[0189] Additionally, the field of view (1100) of the lidar device (1000) may include a horizontal field of view (1110) for a horizontal scan range and a vertical field of view (1120) for a vertical scan range.

[0190] Additionally, the horizontal viewing angle (1110) and the vertical viewing angle (1120) can be defined by the investigated laser.

[0191] For example, the horizontal viewing angle (1110) of the lidar device (1000) may be defined by a first laser (1111) irradiated at a first angle and a second laser (1112) irradiated at a second angle, and more specifically, may be defined by the difference between the first angle irradiated by the first laser (1111) and the second angle irradiated by the second laser (1112), but is not limited thereto.

[0192] In addition, for example, the vertical viewing angle (1120) of the lidar device (1000) may be defined by a third laser (1121) irradiated at a third angle and a fourth laser (1122) irradiated at a fourth angle, and more specifically, may be defined by the difference between the third angle irradiated by the third laser (1121) and the second angle irradiated by the fourth laser (1122), but is not limited thereto.

[0193] However, the definition of the horizontal viewing angle (1110) and the vertical viewing angle (1120) of the lidar device (1000) is not limited to the above-described example, and may be defined by various methods for expressing an area to which a laser is irradiated from the lidar device (1000).

[0194] Additionally, the horizontal viewing angle (1110) and the vertical viewing angle (1120) may be defined by the detected laser. More specifically, the horizontal viewing angle (1110) and the vertical viewing angle (1120) may be defined by point data generated by the detected laser.

[0195] For example, the horizontal field of view (1110) of the lidar device (1000) may be defined by the first point data (1210) and the second point data (1220), and more specifically, may be defined by the irradiation angle of the laser corresponding to the first point data (1210) and the irradiation angle of the laser corresponding to the second point data (1220), but is not limited thereto.

[0196] In addition, for example, the vertical field of view (1120) of the lidar device (1000) may be defined by the third point data (1230) and the fourth point data (1240), and more specifically, may be defined by the irradiation angle of the laser corresponding to the third point data (1230) and the irradiation angle of the laser corresponding to the fourth point data (1240), but is not limited thereto.

[0197] However, the definition of the horizontal field of view (1110) and the vertical field of view (1120) of the lidar device (1000) is not limited to the above-described example, and may be defined by various methods to express an area in which the lidar device (1000) can detect a laser.

[0198] In addition, referring to FIG. 3, the laser forming the field of view (1100) of the lidar device (1000) according to one embodiment can be irradiated to have angular resolution.

[0199] At this time, the angular resolution may include horizontal angular resolution for resolution in the horizontal direction and vertical angular resolution for resolution in the vertical direction.

[0200] Additionally, the horizontal angular resolution and the vertical angular resolution can be defined by the investigated laser.

[0201] For example, the horizontal angular resolution of the lidar device (1000) may be defined by a fifth laser (1131) irradiated at a fifth angle and a sixth laser (1132) irradiated at a sixth angle, and more specifically, may be defined by the difference between the fifth angle irradiated by the fifth laser (1131) and the sixth angle irradiated by the sixth laser (1132), but is not limited thereto.

[0202] In addition, for example, the vertical angular resolution of the lidar device (1000) may be defined by the seventh laser (1141) irradiated at a seventh angle and the eighth laser (1142) irradiated at an eighth angle, and more specifically, may be defined by the difference between the seventh angle irradiated by the seventh laser (1141) and the eighth angle irradiated by the eighth laser (1142), but is not limited thereto.

[0203] However, the definition of the horizontal angular resolution and vertical angular resolution of the above lidar device (1000) is not limited to the above-described examples, and may be defined by various methods to express the angular resolution capable of distinguishing the detection target object.

[0204] In addition, referring to FIG. 3, lidar data (1200) obtained from a lidar device (1000) according to one embodiment may include point data having angular resolution.

[0205] At this time, the angular resolution may include horizontal angular resolution for resolution in the horizontal direction and vertical angular resolution for resolution in the vertical direction.

[0206] Additionally, the horizontal angular resolution and the vertical angular resolution may be defined by the detected laser. More specifically, the horizontal angular resolution and the vertical angular resolution may be defined by point data generated by the detected laser.

[0207] For example, the horizontal angular resolution of the lidar device (1000) may be defined by the fifth point data (1250) and the sixth point data (1260), and more specifically, may be defined by the irradiation angle of the laser corresponding to the fifth point data (1250) and the irradiation angle of the laser corresponding to the sixth point data (1260), but is not limited thereto.

[0208] In addition, for example, the vertical angular resolution of the lidar device (1000) may be defined by the seventh point data (1270) and the eighth point data (1280), and more specifically, may be defined by the irradiation angle of the laser corresponding to the seventh point data (1270) and the irradiation angle of the laser corresponding to the eighth point data (1280), but is not limited thereto.

[0209] However, the definition of the horizontal angular resolution and vertical angular resolution of the above lidar device (1000) is not limited to the above-described examples, and may be defined by various methods to express the angular resolution capable of distinguishing the detection target object.

[0210] Additionally, the lasers irradiated by the above lidar device (1000) may each have a size and divergence angle.

[0211] For example, each laser irradiated by the above lidar device (1000) may have a major axis length and a minor axis length, and may have a divergence angle, but is not limited thereto.

[0212] Additionally, each point data included in the above lidar data (1200) may include distance information.

[0213] Additionally, an optical origin (1300) can be defined for the above lidar device (1000).

[0214] At this time, the optical origin (1300) may mean the origin of the coordinate system for expressing the above-described lidar data.

[0215] In addition, the optical origin (1300) may mean an origin defined when it is assumed that the laser irradiated from the lidar device (1000) is output from one point.

[0216] Additionally, the optical origin (1300) may mean an origin of distance measurement for measuring distance using a laser in the lidar device (1000).

[0217] Additionally, the optical origin (1300) may mean an origin for describing point data acquired from the lidar device (1000).

[0218] In addition, the optical origin (1300) may mean a physically derived optical origin, but is not limited thereto, and may mean an optical origin artificially assigned to the lidar device (1000), but is not limited thereto.

[0219]

[0220] FIG. 4 is a diagram for explaining lidar data according to one embodiment.

[0221] According to one embodiment, lidar data can be expressed in various formats such as a point cloud, a depth map, and an intensity map.

[0222] At this time, the point cloud may be a format in which information about each measurement point is converted into location information and displayed, and the point cloud according to one embodiment may include location coordinate values ​​(x, y, z) and intensity values ​​(I) acquired based on angle information and distance information irradiated or acquired by a laser, but is not limited thereto.

[0223] In addition, at this time, the depth map may be in a format that includes two-dimensional pixel position information and distance information for each measurement point, and the depth map according to one embodiment may include pixel values ​​(x, y) and distance values ​​(D) acquired based on angle information at which the laser is irradiated or acquired, but is not limited thereto.

[0224] In addition, at this time, the intensity map may be in a format that includes two-dimensional pixel location information and intensity information for each measurement point, and the intensity map according to one embodiment may include pixel values ​​(x, y) and intensity values ​​(I) acquired based on angle information at which the laser is irradiated or acquired, but is not limited thereto.

[0225] In addition to the examples described above, lidar data can be acquired in various formats, but for convenience of explanation, the following explanation will be based on lidar data acquired in the form of a point cloud.

[0226]

[0227] Referring to FIG. 4, lidar data according to one embodiment may include point cloud data (2000).

[0228] Additionally, the point cloud data (2000) according to one embodiment may include a plurality of point data.

[0229] Additionally, each of the plurality of point data according to one embodiment may include, but is not limited to, location coordinate values ​​(x, y, z) and intensity values ​​(i).

[0230] At this time, the location coordinate values ​​included in each of the plurality of point data can be obtained based on the distance value.

[0231] For example, the position coordinate values ​​included in each of the plurality of point data may be obtained based on the angle (or coordinate) value at which the laser is output and the distance value obtained based on the output laser, but are not limited thereto.

[0232] In addition, for example, the position coordinate values ​​included in each of the plurality of point data may be acquired based on the coordinate values ​​of the detector that acquired the laser and the distance values ​​acquired based on the acquired laser, but are not limited thereto.

[0233] Additionally, the intensity value included in each of the plurality of point data can be obtained based on an electrical signal obtained from a detector unit.

[0234] For example, the intensity value included in each of the plurality of point data may be obtained based on the characteristics of the size, width, etc. of the electrical signal obtained from the detector unit, but is not limited thereto, and may be obtained by various algorithms for the electrical signal obtained from the detector unit.

[0235] Additionally, for example, the intensity value included in each of the plurality of point data may be obtained based on histogram data generated based on an electrical signal obtained from a detector unit, but is not limited thereto.

[0236]

[0237] FIG. 5 is a diagram for explaining lidar data according to one embodiment.

[0238] Referring to FIG. 5, lidar data according to one embodiment may include point cloud data (2100).

[0239] At this time, since the above-described contents can be applied to the above point cloud data (2100), redundant descriptions will be omitted.

[0240] Point cloud data (2100) according to one embodiment may include at least one sub-point data set (2110).

[0241] At this time, the at least one sub-point data set (2110) may mean a set of point data grouped by a specific rule or algorithm.

[0242] For example, the at least one sub-point data set (2110) may mean a set of point data grouped by human input, but is not limited thereto.

[0243] Additionally, for example, the at least one sub-point data set (2110) may mean a set of point data grouped by a segmentation algorithm for the same object, but is not limited thereto.

[0244] Additionally, for example, the at least one sub-point data set (2110) may mean a set of point data grouped by a clustering algorithm, but is not limited thereto.

[0245] Additionally, for example, the at least one sub-point data set (2110) may mean a set of point data grouped by a learned machine learning model, but is not limited thereto.

[0246] Additionally, for example, the at least one sub-point data set (2110) may mean a set of point data grouped by a learned deep learning model, but is not limited thereto.

[0247] Additionally, the lidar data processing unit according to one embodiment can obtain attribute data for at least one sub-point data set (2110) described above.

[0248] For example, a lidar data processing unit according to one embodiment may obtain at least one attribute data for at least one sub-point data set (2110) based on a human input, but is not limited thereto.

[0249] Additionally, for example, a lidar data processing unit according to one embodiment may obtain at least one attribute data for at least one sub-point data set (2110) using a specific algorithm, but is not limited thereto.

[0250] Additionally, for example, the lidar data processing unit according to one embodiment may obtain at least one attribute data for at least one sub-point data set (2110) using a learned machine learning model, but is not limited thereto.

[0251] Additionally, for example, the lidar data processing unit according to one embodiment may obtain at least one attribute data for at least one sub-point data set (2110) using a learned deep learning model, but is not limited thereto.

[0252] Additionally, the machine learning model or deep learning model described above may include at least one artificial neural network layer (ANN).

[0253] For example, the machine learning model or deep learning model described above may include, but is not limited to, at least one artificial neural network layer from among various artificial neural network layers such as a feedforward neural network, a radial basis function network, a Cohen self-organizing network, a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a long short-term memory network (LSTM), or gated recurrent units (GRUs).

[0254] Additionally, at least one artificial neural network layer included in the above-described machine learning model or deep learning model may be designed to use the same or different activation functions.

[0255] At this time, the activation function may include, but is not limited to, a sigmoid function, a hyperbolic tangent function, a Relu function (rectified linear unit function), a leaky Relu function, an ELU function (exponential linear unit function), a softmax function, etc., and may include various activation functions (including custom activation functions) for outputting a result value or transmitting it to another artificial neural network layer.

[0256] Additionally, the above-described machine learning model or deep learning model can be trained using at least one loss function.

[0257] At this time, the at least one loss function may include, but is not limited to, MSE (Mean Squared Error), RMSE (Root Mean Squared Error), Binary Crossentropy, Categorical Crossentropy, Sparse Categorical Crossentropy, etc., and may include various functions (including custom loss functions) for calculating the difference between the predicted result value and the actual result value.

[0258] Additionally, the above-described machine learning model or deep learning model can be trained using at least one optimizer.

[0259] At this time, the optimizer can be used to update the relationship parameters between the input values ​​and the result values.

[0260] At this time, the at least one optimizer may include, but is not limited to, Gradient descent, Batch Gradient Descent, Stochastic Gradient Descent, Mini-batch Gradient Descent, Momentum, AdaGrad, RMSProp, AdaDelta, Adam, NAG, NAdam, RAdam, AdamW, etc.

[0261]

[0262] Below, we will describe the acquired attribute data in more detail.

[0263]

[0264] FIG. 6 is a diagram for explaining information included in attribute data according to one embodiment.

[0265] Referring to FIG. 6, a lidar data processing unit according to one embodiment can obtain at least one attribute data (2200) for a sub-point data set (2110) according to one embodiment.

[0266] At this time, the at least one attribute data (2200) may include, but is not limited to, class information (2210), center location information (2220), size information (2230), shape information (2240), movement information (2250), identification information (2260), etc. of the object indicated by the sub-point data set (2110).

[0267] Additionally, the same algorithm or model may be used to obtain each attribute data included in the at least one attribute data (2200), or different algorithms or models may be used.

[0268] Additionally, at least one attribute data (2200) can be acquired based on point cloud data included in one frame data.

[0269] For example, attribute data such as class information (2210), center location information (2220), size information (2230), and shape information (2240) of an object included in at least one attribute data (2200) may be acquired based on point cloud data included in one frame data, but is not limited thereto.

[0270] Additionally, the at least one attribute data (2200) can be acquired based on point cloud data included in a plurality of frame data.

[0271] For example, attribute data such as movement information (2250) and identification information (2260) included in at least one attribute data (2200) may be acquired based on point cloud data included in a plurality of frame data, but is not limited thereto.

[0272]

[0273] In addition, although the description was made based on lidar data acquired in the form of a point cloud through FIGS. 4 to 6, the described contents can also be applied to lidar data acquired in the form of a depth map, intensity map, etc. in addition to the point cloud format as described above.

[0274]

[0275] 2. [Configuration of a parking lot management system using a lidar device]

[0276] FIG. 7 and FIG. 8 are drawings for explaining the configuration of a parking lot management system according to one embodiment.

[0277] Referring to FIGS. 7 and 8, a system for managing parking lots (3000) according to one embodiment includes a plurality of lidar devices (3010), a plurality of lidar processors (3020), and a parking lot management processor (3030).

[0278] (1) [Multiple lidar devices]

[0279] At this time, the plurality of lidar devices (3010) are configured to obtain distance information for at least one point within the field-of-view range.

[0280] Additionally, at this time, the plurality of lidar devices (3010) are arranged to cover at least a portion of the parking lot.

[0281] For example, a first lidar device included in the plurality of lidar devices (3010) may be positioned to cover a first portion of a parking lot, and a second lidar device may be positioned to cover a second portion of the parking lot.

[0282] Additionally, at this time, the plurality of lidar devices (3010) may be positioned on the outskirts of the parking lot and positioned to face at least one parking space.

[0283] For example, a first lidar device included in the plurality of lidar devices (3010) may be placed on the outskirts of a parking lot and may be positioned to face the first to tenth parking lots of the parking lot, and a second lidar device may be placed on the outskirts of the parking lot and may be positioned to face the eleventh to twentieth parking lots of the parking lot.

[0284] Additionally, at this time, the plurality of lidar devices (3010) can be arranged to have a preset relationship with each other.

[0285] For example, the plurality of lidar devices (3010) may be arranged to cover different portions of the parking lot, such that the portion of the parking lot covered by each of the plurality of lidar devices (3010) at least partially overlaps with the portion of the parking lot covered by at least one other lidar device.

[0286] Additionally, at this time, each of the plurality of lidar devices (3010) may be configured to generate an individual point cloud.

[0287]

[0288] (2) [Multiple LiDAR Processors]

[0289] A plurality of lidar processors (3020) may be configured to obtain distance information for at least one point from the plurality of lidar devices (3010) and generate an individual point cloud based on the distance information.

[0290] At this time, the individual point cloud may be generated from the plurality of lidar devices (3010), and may be generated from a lidar processor (3020) connected to at least some of the lidar devices among the plurality of lidar devices (3010).

[0291] In addition, at this time, the individual point cloud is expressed as an individual point cloud to indicate that it is generated from an individual lidar device or an individual lidar processor, but this is only an expression for convenience of explanation, and the point cloud generated from the lidar device or lidar processor is not limited by the term “individual.”

[0292] In addition, at this time, the individual point cloud is expressed as an individual point cloud in that it is a point cloud expressed through an individual coordinate system defined in an individual lidar device or an individual lidar processor, but this is only an expression for convenience of explanation, and the point cloud generated in the lidar device or lidar processor is not limited by the term “individual.”

[0293] Additionally, at this time, the plurality of lidar processors (3020) may be configured to cover at least one lidar device among the plurality of lidar devices (3010).

[0294] At this time, the meaning that the plurality of lidar processors (3020) cover at least one lidar device may mean that they are connected to process information obtained from the at least one lidar device.

[0295] For example, among the plurality of lidar processors (3020), a first lidar processor may be configured to cover a first lidar device and a second lidar device, a second lidar processor among the plurality of lidar processors (3020) may be configured to cover a third lidar device and a fourth lidar device, and a third lidar processor among the plurality of lidar processors (3020) may be configured to cover a fifth lidar device.

[0296] At this time, the number of lidar devices covered by each of the plurality of lidar processors (3020) may be different or the same depending on the configuration of the parking lot management system.

[0297] Additionally, at this time, the plurality of lidar processors (3020) are configured to generate identification information for at least one object based on the generated point cloud.

[0298] At this time, the identification information for at least one object may correspond to the above-described attribute information.

[0299] In addition, at this time, the identification information for the at least one object includes at least one of class information for the at least one object, center position information, bounding box information, ID information, movement direction information of the object, tracking information of the identified object, and point cloud information for the at least one object (i.e., sub-point data set information including at least some of the point data sets among the point data sets).

[0300] Additionally, at this time, the plurality of lidar processors (3020) may be configured to generate reference information for implementing a parking lot management system.

[0301] At this time, the reference information includes at least one of parking line detection information, parking slot space information for determining whether a parking slot is occupied, parking slot ID information, and parking slot coordinate information for the parking slot.

[0302] Additionally, the plurality of lidar processors (3020) can generate information related to parking lot management.

[0303] For example, the plurality of lidar processors (3020) can generate information on whether a parking slot is occupied.

[0304] Additionally, the plurality of lidar processors (3020) are configured to select information to be transmitted to the parking lot management processor (3030).

[0305] For example, the plurality of lidar processors (3020) may be configured to select information to be transmitted to the parking lot management processor (3030) from among the acquired or generated information, and to transmit only the selected information.

[0306] At this time, the selected information may be all of the individual point clouds acquired from each of the plurality of lidar processors (3020), may be a sub-point data set including only some of the point data among the plurality of point data constituting the individual point cloud, and may be identification information for at least one object, but is not limited thereto.

[0307] Additionally, the plurality of lidar processors (3020) are configured to select information to be transmitted to other lidar processors.

[0308] For example, the plurality of lidar processors (3020) may be configured to select information to be transmitted to other lidar processors among the acquired or generated information, and to transmit only the selected information.

[0309] At this time, the selected information may be all of the individual point clouds acquired from each of the plurality of lidar processors (3020), may be a sub-point data set including only some of the point data among the plurality of point data constituting the individual point cloud, and may be identification information for at least one object, but is not limited thereto.

[0310]

[0311] (3) [Parking Lot Management Processor]

[0312] The parking lot management processor (3030) is configured to acquire individual point clouds from each of a plurality of lidar processors (3020) and to generate an integrated point cloud based on the acquired individual point clouds.

[0313] At this time, the integrated point cloud may mean a point cloud expressed through an integrated coordinate system for the parking lot.

[0314] Creating an integrated point cloud is explained in more detail below using other drawings.

[0315] In addition, at this time, the integrated point cloud may mean expressing all point data through an integrated coordinate system based on the case where all point data constituting individual point clouds are acquired from each of the plurality of lidar processors (3020), but is not limited thereto, and may be understood to include expressing the point data acquired based on this in an integrated coordinate system even when only some of the point data constituting individual point clouds are acquired from each of the plurality of lidar processors (3020).

[0316] Additionally, at this time, the parking lot management processor (3030) is configured to generate identification information for at least one object based on the generated integrated point cloud.

[0317] At this time, the identification information for at least one object may correspond to the above-described attribute information.

[0318] In addition, at this time, the identification information for the at least one object includes at least one of class information for the at least one object, center position information, bounding box information, ID information, movement direction information of the object, tracking information of the identified object, and point cloud information for the at least one object (i.e., sub-point data set information including at least some of the point data sets among the point data sets).

[0319] Additionally, at this time, the parking lot management processor (3030) is configured to generate identification information for at least one object based on information received from a plurality of lidar processors (3020).

[0320] At this time, the information transmitted from the plurality of lidar processors (3020) may be all of the individual point clouds acquired from each of the plurality of lidar processors (3020), may be a sub-point data set including only some of the point data among the plurality of point data constituting the individual point cloud, and may be identification information for at least one object, but is not limited thereto.

[0321] Additionally, at this time, the parking lot management processor (3030) may be configured to generate reference information for implementing a parking lot management system.

[0322] At this time, the reference information includes at least one of parking line detection information, parking slot space information for determining whether a parking slot is occupied, a parking slot ID, and parking slot coordinate information for the parking slot.

[0323] Additionally, the parking lot management processor (3030) can generate information related to parking lot management.

[0324] For example, the parking lot management processor (3030) may generate at least one piece of information from among information on whether a parking slot is occupied, information on whether each parking slot is occupied for a plurality of parking slots, location coordinates of parking slots available for parking, and information on the occupancy rate of the entire parking lot.

[0325]

[0326] 3. [Operating method of a parking lot management system using a lidar device]

[0327] FIG. 9 is a drawing for explaining an operation method of a parking lot management system using a lidar device according to one embodiment.

[0328] Referring to FIG. 9, an operation method of a parking lot management system for a lidar device according to one embodiment comprises: i) performing a reference measurement step (S3100) to generate reference information, and ii) performing a monitoring step (S3200) using the generated reference information and a point cloud acquired at the current point in time to generate parking lot management information.

[0329] At this time, for convenience of explanation, point clouds generated through multiple lidar devices in the reference measurement step (S3100) can be expressed as reference point clouds, and point clouds generated through multiple lidar devices in the monitoring step (S3200) can be expressed as monitoring point clouds.

[0330] Additionally, at this time, the reference measurement step (S3100) may include aligning reference point clouds acquired from multiple lidar processors (S3110), aligning the aligned reference point clouds to a preset plane (S3120), and generating reference information (S3130).

[0331] In addition, at this time, the monitoring step (S3200) may include evaluating the validity of the monitoring point cloud (S3210), generating identification information for at least one object based on the monitoring point cloud (S3220), tracking the identified object based on the monitoring cloud (S3230), and generating parking lot management information (S3250).

[0332] Below, each step is explained in more detail.

[0333] (1) [Reference measurement step (S3100) - Aligning reference point clouds acquired from multiple lidar processors (S3110)]

[0334] As described above, each of the multiple LiDAR devices is positioned to cover a predetermined area of ​​the parking lot, and a point cloud is acquired based on the optical origin of each of the multiple LiDAR devices. In other words, the individual point clouds acquired from each of the multiple LiDAR devices may be information expressed by an individual coordinate system whose origin is the optical origin of each LiDAR device.

[0335] Alternatively, the individual point clouds generated from each of the plurality of lidar processors may be information expressed by an individual coordinate system based on the origin for each of the plurality of lidar processors.

[0336] However, in order to implement a parking lot management system using multiple lidar devices covering different areas of a parking lot, it is necessary to align the individual point clouds for each lidar device and integrate and process them on a unified coordinate system for the parking lot.

[0337] According to one embodiment, the alignment of reference point clouds acquired from multiple lidar processors (S3110) may be performed in a parking lot management processor.

[0338] At this time, the parking lot management processor can obtain individual point clouds from the plurality of lidar processors and process and align the obtained individual point clouds.

[0339] Additionally, at this time, the parking lot management processor can align individual point clouds by setting an integrated coordinate system and deriving a relationship for converting the coordinate systems of individual point clouds acquired from the plurality of lidar processors into the integrated coordinate system.

[0340] At this time, a coarse registration step based on feature matching and a fine registration step based on ICP (Interactive-closest point) can be performed to align the individual point clouds.

[0341] Additionally, at this time, the parking lot management processor can create an integrated point cloud by aligning individual point clouds acquired from the plurality of lidar processors.

[0342] Additionally, at this time, the parking lot management processor can generate and store a relationship function for converting the coordinate system of each individual point cloud acquired from each of the plurality of lidar processors into an integrated coordinate system.

[0343] Additionally, at this time, the relationship function generated and stored in the parking lot management processor can be used to generate an integrated monitoring point cloud based on individual monitoring point clouds acquired in a subsequent monitoring step (S3200).

[0344]

[0345] (2) [Reference measurement step (S3100) - Aligning the aligned reference point cloud to the preset plane (S3120)]

[0346] Even if an integrated point cloud is created by aligning individual point clouds acquired from multiple lidar devices through the above-described steps, the ground surface expressed in the integrated point cloud and the plane for the ground surface preset in the integrated coordinate system may not be aligned with each other.

[0347] That is, the integrated coordinate system may be a coordinate system configured to align the ground surface for the parking lot to the XY plane, but the ground surface represented in the integrated point cloud may not be aligned to the XY plane and may be tilted. Therefore, ground surface alignment is required.

[0348] According to one embodiment, aligning the aligned reference point cloud to a preset plane (S3120) may be performed by the parking lot management processor.

[0349] At this time, the parking lot management processor processes the acquired integrated point cloud and aligns it to a preset plane.

[0350] Additionally, at this time, the parking management processor obtains plane information for the ground surface expressed on the integrated point cloud, and derives a relationship for aligning the obtained plane information for the ground surface to a preset plane (e.g., XY plane), thereby aligning the integrated point cloud to the preset plane.

[0351] At this time, in order to obtain planar information about the ground surface expressed on the integrated point cloud, the integrated point cloud acquired over a preset period of time is accumulated, and clustering is performed on the plane expressing the ground surface based on the accumulated integrated point cloud, thereby obtaining planar information about the ground surface.

[0352] Additionally, at this time, the parking lot management processor can align the acquired integrated point cloud to a preset plane to generate a surface-aligned integrated point cloud.

[0353] Additionally, at this time, the parking lot management processor can generate and store a relationship function for generating a surface-aligned integrated point cloud by aligning the integrated point cloud to a preset plane.

[0354] In addition, at this time, the relationship function generated and stored in the parking lot management processor can be used to generate an integrated monitoring point cloud based on individual monitoring point clouds acquired in a subsequent monitoring step (S3200), and to generate an integrated monitoring point cloud aligned with the ground surface based on the generated integrated monitoring point cloud.

[0355] In addition, at this time, the parking lot management processor can generate and store a conversion function for converting individual point clouds acquired from each of a plurality of lidar processors into an integrated point cloud aligned with the ground surface based on the relationship function derived in the step of matching the above-described point clouds and the step of aligning the matched point clouds to a preset plane, which can be used to generate an integrated monitoring point cloud aligned with the ground surface based on individual monitoring point clouds acquired in a subsequent monitoring step (S3200).

[0356]

[0357] (3) [Reference measurement step (S3100) - Generating reference information (S3130)]

[0358] According to one embodiment, in generating reference information (S3130), the reference information includes information related to a parking slot, and the information related to the parking slot may include at least one of parking line detection information, parking slot space information for determining whether a parking slot is occupied, parking slot ID, and parking slot coordinate information.

[0359] In addition, according to one embodiment, when the plurality of LiDAR devices are solid-state LiDAR devices composed of a plurality of detector pixels, in generating reference information (S3130), the reference information may include reference depth information for each of the plurality of detector pixels included in each of the plurality of LiDAR devices.

[0360] Accordingly, according to one embodiment, generating reference information (S3130) may include any one of generating parking line detection information, generating parking slot space information for determining whether a parking slot is occupied, assigning a parking slot ID, generating parking slot coordinate information, and generating reference depth information for each of a plurality of detector pixels included in each of a plurality of lidar devices.

[0361] 1) [Generate parking line detection information]

[0362] FIG. 10 is a drawing for explaining generation of parking line detection information according to one embodiment.

[0363] Referring to FIG. 10, generating parking line detection information (3300) according to one embodiment may include cropping point data corresponding to a ground surface (S3310), comparing an intensity value of the cropped point data with a preset intensity value to select point data corresponding to a parking line (S3320), projecting the selected point data onto a preset plane (S3330), and generating parking line detection information using the projected points (S3340).

[0364] At this time, cropping of point data corresponding to the ground surface (S3310) can be performed on the integrated point cloud in which the ground surface is aligned as described above, and can be performed by the parking lot management processor.

[0365] Additionally, at this time, cropping point data corresponding to the ground surface according to one embodiment (S3310) may include selecting point data having a height within a preset range based on a preset plane.

[0366] At this time, the preset plane may mean a plane corresponding to the earth's surface in the integrated coordinate system, and may mean the XY plane.

[0367] Also, at this time, selecting point data having a preset range of heights based on a preset plane in order to crop point data corresponding to the ground surface may be a coordinate system set by assuming the XY plane as the ground surface of the parking lot, but the ground surface of an actual parking lot does not all have the same height and may be curved and have different heights, so it may be for the purpose of cropping point data corresponding to the ground surface of an actual parking lot.

[0368] Also, at this time, the preset range of heights may be in the range of +z1 to -z2 based on the XY plane, and at this time, z1 and z2 may be the same as each other, but are not limited thereto and may be different from each other.

[0369] In addition, at this time, point data corresponding to a parking line are selected by comparing the intensity value of cropped point data according to one embodiment with a preset intensity value (S3320), and the preset intensity value may be an intensity value corresponding to the color of the parking line of the parking lot.

[0370] For example, the preset intensity value may be the intensity value corresponding to white paint.

[0371] In addition, at this time, projection (S3330) of selected point data according to one embodiment onto a preset plane may be an operation necessary to detect a two-dimensional parking line by using the selected point data, which may be a set of points located in a three-dimensional space.

[0372] In addition, at this time, projection (S3330) of selected point data according to one embodiment onto a preset plane can greatly reduce the computing power required compared to generating line information by directly using 3D point data.

[0373] Additionally, at this time, generating parking line detection information using projected points according to one embodiment (S3340) can be performed through various algorithms for generating parking line detection information.

[0374] For example, generating parking line detection information using projected points according to one embodiment (S3340) may be performed through a Hough transform algorithm, but is not limited thereto.

[0375]

[0376] 2) [Generate parking slot space information to determine whether the parking slot is occupied]

[0377] Parking slot space information according to one embodiment is information on a three-dimensional space corresponding to a parking slot, and refers to a space for determining whether a parking slot is occupied by a vehicle.

[0378] At this time, the parking slot space information can be set as a three-dimensional cube area of ​​a preset size.

[0379] For example, the above parking slot space information may be set as a three-dimensional cube area having a size of 2.5 m wide, 4.5 m long, and 2 m high.

[0380] Additionally, at this time, the parking slot space information can be generated based on the detected parking line detection information.

[0381] For example, the above parking slot space information may be set as a three-dimensional cube area having a width and length corresponding to the detected parking line detection information and a preset height.

[0382] Additionally, at this time, the parking slot space information can be set to be located above a preset height from a preset plane.

[0383] For example, the above parking slot space information may be set as a three-dimensional cube area having a size of 2.5 m wide, 4.5 m long, and 2 m high at a height of 50 cm from a preset plane.

[0384] This can have meaning as a preset to remove the number of point data on the ground surface that may cause errors when the number of point data located within the parking slot space information is used to determine whether a vehicle is occupied in a parking slot.

[0385] In addition, at this time, the standard for how high above the parking slot space information will be set from the preset plane, i.e., the preset height, can be set based on the distance between the point data selected in generating parking line detection information (3300) and the preset plane.

[0386] For example, if the largest value among the distances between the selected point data and the preset plane in generating parking line detection information (3300) is 50 cm, the preset height may be at least 50 cm or more.

[0387]

[0388] 3) [Generate coordinate information for parking slots]

[0389] At this time, the coordinate information of the parking slot may be a coordinate representing the detected parking line information.

[0390] For example, the coordinate information of the above parking slot may be the center coordinates of the detected parking line, but is not limited thereto.

[0391] Additionally, at this time, the coordinate information of the parking slot may be preset coordinates.

[0392] For example, the coordinate information of the above parking slot may be coordinates in a preset order for the convenience of management in the parking lot management system rather than the actual location coordinates of the parking slot on the integrated coordinate system, but is not limited thereto.

[0393] Additionally, at this time, the coordinate information of the parking slot may be coordinates pre-saved on the integrated coordinate system described above.

[0394]

[0395] 4) [Generate reference depth information for each of the multiple detector pixels included in each of the multiple lidar devices]

[0396] When implementing a parking lot management system using multiple lidar devices, the number of point data included in individual point clouds acquired from the multiple lidar devices may be too large, placing a significant load on computer processing power.

[0397] Therefore, reducing the overall computing power of a parking lot management system can be a means of increasing the feasibility of the parking lot management system.

[0398] At this time, the parking lot management system performs monitoring of a fixed parking lot environment, and monitoring objects that change in contrast to the fixed environment may reduce the overall computing power of the parking lot management system.

[0399] Therefore, it may be important to obtain a reference value for a fixed environment so that a changing object can be monitored in contrast to a fixed environment, and in the case of solid-state LiDAR, since each of the multiple detector pixels is fixed to look at the same target in the same direction, if a reference depth value for each of the multiple detector pixels can be obtained in the reference measurement step, it may be easy to detect a changed object in contrast to a fixed environment based on this.

[0400] According to one embodiment, generating reference depth information for each of the plurality of detector pixels included in each of the plurality of lidar devices may be performed in the lidar processor.

[0401] At this time, the lidar processor can obtain depth values ​​corresponding to each of a plurality of detector pixels from the lidar device for a certain period of time and process them to generate reference depth information.

[0402] Additionally, at this time, the lidar processor can store the depth value for each detector pixel acquired from the lidar device for a certain period of time, acquire a reference depth value for each detector pixel based on the stored depth value for each pixel, and generate the acquired reference depth value as reference depth information.

[0403]

[0404] (4) [Monitoring Step (S3200) - Evaluating the Validity of the Monitoring Point Cloud (S3210)]

[0405] According to a parking lot management system according to one embodiment, a plurality of lidar devices are positioned at a height above a certain level from the ground so as to secure a certain number of parking slots covered by each of the plurality of lidar devices.

[0406] At this time, since multiple lidar devices are positioned at a certain height from the ground, the pillar (e.g., pole) on which the lidar devices are installed may shake due to wind, etc., which may mean that the location where the lidar devices are placed changes or the direction in which the lidar devices are facing changes.

[0407] In this way, if the locations where multiple lidar devices are placed change or the direction in which multiple lidar devices are facing change, errors may occur in determining the location of identified objects, whether parking slots are occupied, etc.

[0408] Therefore, in the monitoring step (S3200), it may be necessary to determine whether the point cloud acquired through the multiple lidar devices is a point cloud acquired when the multiple lidar devices are at preset locations, which can be described as evaluating the validity of the monitoring point cloud in this specification.

[0409] In one embodiment, evaluating the validity of the monitoring point cloud can be performed in the lidar processor.

[0410] At this time, the lidar processor generates a point cloud based on data acquired from multiple lidar devices (or acquires point clouds generated from multiple lidar devices), evaluates the validity of the generated point cloud, and if the generated point cloud is determined to be invalid, the generated point cloud can be ignored or deleted.

[0411] FIG. 11 is a diagram illustrating a method for evaluating the validity of a monitoring point cloud according to one embodiment.

[0412] Referring to FIG. 11, a method (3400) for evaluating the validity of a monitoring point cloud according to one embodiment may include setting an area for evaluating validity (S3410), estimating a reference plane for the area for evaluating validity (S3420), calculating a reference incidence angle of a lidar device with respect to the estimated reference plane (S3430), estimating a monitoring plane for the area for evaluating validity (S3440), calculating a monitoring incidence angle of a lidar device with respect to the estimated monitoring plane (S3450), and ignoring the point cloud when the difference between the monitoring incidence angle and the reference incidence angle is greater than a preset value (S3460).

[0413] In setting an area for evaluating validity according to one embodiment (S3410), the area for evaluating validity may be set as an area with little movement of people, cars, etc. within a parking lot.

[0414] Additionally, at this time, the area for evaluating validity can be set to a preset angular range of the lidar device.

[0415] That is, the area for evaluating the above validity can be specified as a first azimuth range and a first elevation range.

[0416] Additionally, at this time, the area for evaluating validity may be set to a group of detector pixels among a plurality of detector pixels of the lidar device.

[0417] That is, the region for evaluating the validity can be specified as at least a first pixel region, and the first pixel region can be a region including pixels corresponding to at least (N,M), (N-1,M), (N+1, M), (N,M-1), (N-1,M-1), (N+1,M-1), (N,M+1), (N-1,M+1), (N+1, M+1) pixel coordinates.

[0418] Additionally, in estimating a reference plane for an area for evaluating validity according to one embodiment (S3420), estimating the reference plane may include calculating a normal vector for the reference plane.

[0419] Additionally, at this time, the reference plane can be estimated for the accumulated point cloud after accumulating the point cloud acquired over a preset period of time.

[0420] Additionally, at this time, calculating the normal vector for the reference plane can be performed by various algorithms for estimating the plane and calculating the normal vector for the estimated plane.

[0421] For example, calculating a normal vector for the reference plane can be performed using a PCA (Principal component analysis) algorithm.

[0422] In addition, in calculating the reference incidence angle of the lidar device with respect to the estimated reference plane according to one embodiment (S3430), the reference incidence angle may be calculated as the angle between the normal vector for the estimated reference plane and a representative angle of a preset angular range (or an angular range covered by some preset groups of detector pixels) of the lidar device corresponding to the area for evaluating validity.

[0423] Additionally, at this time, setting an area for evaluating the validity (S3410), estimating a reference plane for the area for evaluating the validity (S3420), and calculating a reference incidence angle of the lidar device for the estimated reference plane (S3430) can be performed in the reference measurement step (S3100).

[0424] Additionally, in estimating a monitoring plane for an area for evaluating validity according to one embodiment (S3440), estimating the monitoring plane may include calculating a normal vector for the monitoring plane.

[0425] Additionally, at this time, the monitoring plane can be estimated for the point cloud for one frame.

[0426] Additionally, at this time, calculating the normal vector for the monitoring plane can be performed by various algorithms for estimating the plane and calculating the normal vector for the estimated plane.

[0427] For example, calculating the normal vector for the monitoring plane can be performed using the PCA (Principal component analysis) algorithm.

[0428]

[0429] In addition, in calculating the monitoring incidence angle of the lidar device for the estimated monitoring plane according to one embodiment (S3450), the monitoring incidence angle may be calculated as an angle between a normal vector for the estimated monitoring plane and a representative angle of a preset angular range (or an angular range covered by some preset groups of detector pixels) of the lidar device corresponding to an area for evaluating validity.

[0430] In addition, in case the difference between the monitoring incidence angle and the reference incidence angle according to one embodiment is greater than or equal to a preset value, the point cloud is ignored (S3460). If the difference between the monitoring incidence angle and the reference incidence angle is greater than or equal to a preset value, it means that the placement position or direction of the lidar device is different from the preset placement position or direction, and therefore the point cloud for the corresponding frame may be ignored or deleted.

[0431]

[0432] (5) [Monitoring step (S3200) - Generating identification information for at least one object based on the monitoring point cloud (S3220)]

[0433] In generating identification information for at least one object based on a monitoring point cloud according to one embodiment (S3220), the identification information for at least one object may be applied with the contents of the above-described attribute information.

[0434] In addition, at this time, the identification information for the at least one object includes at least one of class information for the at least one object, center position information, bounding box information, ID information, movement direction information of the object, tracking information of the identified object, and point cloud information for the at least one object (i.e., sub-point data set information including at least some of the point data sets among the point data sets).

[0435] Additionally, at this time, obtaining identification information for at least one object can be performed in the lidar processor.

[0436] At this time, the lidar processor generates a point cloud based on data acquired from a plurality of lidar devices, and processes the generated point cloud to obtain at least one of class information for at least one object, center position information, bounding box information, ID information, movement direction information of the object, tracking information of the identified object, and point cloud information for at least one object (i.e., sub-point data set information including at least some of the point data sets among the point data sets).

[0437] Additionally, at this time, obtaining identification information for at least one object can be performed by the parking lot management processor.

[0438] At this time, the parking lot management processor can obtain point clouds from a plurality of lidar processors, generate an integrated point cloud based on the obtained point clouds, and process the generated integrated point cloud to obtain identification information for at least one object.

[0439] Additionally, at this time, the parking lot management processor can obtain point data sets selected from a plurality of lidar processors, generate an integrated point cloud based on the obtained point data sets, and process the generated integrated point cloud to obtain identification information for at least one object.

[0440] Additionally, at this time, the parking lot management processor can obtain bounding box information and center location information for at least one object from a plurality of lidar processors, and obtain identification information for at least one object based on the obtained bounding box information and center location information.

[0441]

[0442] (6) [Monitoring Step (S3200) - Tracking the identified object based on the monitoring point cloud (S3230)]

[0443] Tracking an identified object may mean tracing the movement path of the identified object, determining the identity between the object represented in the point cloud at a specific point in time (a specific frame) and the object represented in the point cloud at the next point in time (the next frame), and updating the location information of the identified object.

[0444] That is, tracking an identified object may mean, but is not limited to, obtaining location information of a specific object from the point cloud of the first frame, determining an object identical to the specific object in the next frame, the second frame, and updating the location information of the specific object. The term may include concepts understood as typical tracking.

[0445] According to one embodiment, tracking an object identified based on a monitoring point cloud (S3230) may be performed in each lidar processor.

[0446] At this time, the lidar processor generates a point cloud of the first frame, generates identification information for at least one object based on the generated point cloud of the first frame, generates a point cloud of the second frame, generates identification information for at least one object based on the point cloud of the second frame, determines identity between the identification information for at least one object generated in the first frame and the identification information for at least one object generated in the second frame, and if it is determined as a result of the determination that they are the same object, tracking information for tracing the movement path of the object can be generated.

[0447] At this time, the determination of identity can be performed based on the position of at least one object generated in the first frame and the position of at least one object generated in the second frame.

[0448] Additionally, at this time, the tracking information may include information that the first object is at the first position at the first time corresponding to the first frame and then moves to the second position at the second time corresponding to the second frame.

[0449] Additionally, according to one embodiment, tracking an object identified based on a monitoring point cloud (S3230) may be performed in each lidar processor, but may be assisted by communication between two or more lidar processors.

[0450] This may be because, since there are areas individually covered by multiple lidar devices within a parking lot, if a vehicle moves from an area covered by each of the multiple lidar devices to an area not covered by each of the multiple lidar devices, each of the multiple lidar devices may determine that a vehicle previously tracked has disappeared or a vehicle previously untracked has been newly identified, and thus a means to assist this may be necessary.

[0451] In this case, when at least one object is newly identified, each lidar processor selects another lidar device that covers at least a part of the location of the identified object, requests identification information about the object from another lidar processor that covers the selected lidar device, and when obtaining identification information about the object from another lidar processor, compares the obtained identification information about the object with the newly identified identification information about the object to determine identity.

[0452] That is, when a first object is newly identified, the first lidar processor requests identification information on at least one object from a second lidar processor covering a second lidar device covering at least a part of the location of the first object, and when 'identification information on at least one object is acquired from the second lidar processor, the acquired identification information on the object is compared with the newly identified identification information on the object to determine identity, and when the objects are determined to be identical as a result of the judgment, an ID included in the acquired identification information on the object is assigned to the newly identified first object to generate tracking information, but when the objects are determined to be non-identical as a result of the judgment, a new ID is assigned to the newly identified first object to generate tracking information.

[0453] Additionally, according to one embodiment, tracking an object identified based on a monitoring point cloud (S3230) may be performed in a parking lot management processor.

[0454] At this time, the parking lot management processor can obtain individual point clouds from a plurality of lidar processors, generate an integrated point cloud based on the obtained individual point clouds, generate identification information for at least one object based on the generated integrated point cloud, and track the identified object based on the generated identification information for at least one object.

[0455] In addition, at this time, the parking lot management processor can obtain point data sets of some of the individual point clouds from a plurality of lidar processors, generate an integrated point cloud based on the obtained point data sets, generate identification information for at least one object based on the generated integrated point cloud, and track the identified object based on the generated identification information for at least one object.

[0456] In addition, at this time, the parking lot management processor obtains center position information and bounding box information for the identified object from a plurality of lidar processors, positions the obtained center position information and bounding box information on an integrated coordinate system, and tracks the identified object based on the center position information and bounding box information positioned on the integrated coordinate system.

[0457] Additionally, according to one embodiment, tracking of an object identified based on a monitoring point cloud (S3230) may be performed in each lidar processor, but may be assisted by a parking lot management processor.

[0458] At this time, the lidar processor generates an individual point cloud, assigns an individual vehicle ID to the identified object based on the generated individual point cloud, and specifies a sub-point data set for the identified object.

[0459] Additionally, at this time, the parking lot management processor obtains individual vehicle IDs and sub-point data sets from a plurality of lidar processors, and uses the obtained individual vehicle IDs and sub-point data sets to determine the identity of each individual vehicle ID to generate tracking information.

[0460] That is, when the parking lot management processor obtains a first ID and a first sub-point data set from a first lidar processor and obtains a second ID and a second sub-point data set from a second lidar processor, it generates an integrated point cloud based on the first sub-point data set and the second sub-point data set, determines whether the first sub-point data set and the second sub-point data set refer to the same object based on the generated integrated point cloud, and generates tracking information by matching the first ID and the second ID as IDs for the same vehicle or assigning a vehicle ID when it is determined that the first sub-point data set and the second sub-point data set refer to the same object.

[0461]

[0462] (7) [Monitoring Step (S3200) - Method for transmitting only a portion of the point cloud from the lidar processor to the parking lot management processor]

[0463] To implement a parking lot management system for a single parking lot, the number of lidar devices required may vary depending on the size of the parking lot.

[0464] At this time, depending on the size of the parking lot, several dozen LiDAR devices may be required, or as many as a hundred or more LiDAR devices may be required.

[0465] However, in this case, if all point clouds generated by multiple LiDAR devices are transmitted to the parking management processor and processed by the parking management processor, operating the parking management system in real time may become practically impossible. (Or, it may require extremely high computer processing power, significantly increasing the cost of installing the parking management system itself.)

[0466] Therefore, it is possible to consider transmitting only some of the point data from among the point clouds generated by multiple lidar devices so that the parking lot management system can be operated in real time, which can enable processing of the point cloud in real time while minimizing the computing power of the parking lot management processor.

[0467] To this end, one can consider detecting a sub-point data set corresponding to an object (i.e., a set of point data corresponding to an object) in the lidar processor and transmitting only the detected sub-point data set.

[0468] However, due to the limited computing power of the Liada processor, errors may occur in the detection of sub-point data sets corresponding to objects, which may lead to incorrect judgments by the parking management processor (e.g., determining that a vehicle does not exist when it actually does, or determining that a vehicle exists when it actually does not).

[0469] Therefore, it is necessary to develop a method to transmit only a portion of the point clouds generated by multiple lidar devices, and to transmit only a portion of the point clouds that require analysis by the parking lot management processor later without error even with the low computing performance of the lidar processor.

[0470] Fig. 12 is a drawing for explaining a method for processing a point cloud according to one embodiment.

[0471] Referring to FIG. 12, a method for processing a point cloud (3500) according to one embodiment may include storing a reference depth value corresponding to each detector pixel of a lidar device (S3510), obtaining a monitoring depth value corresponding to each detector pixel of the lidar device (S3520), comparing the reference depth value and the monitoring depth value corresponding to each detector pixel of the lidar device to select a pixel to be shared (S3530), generating a point cloud based on the pixel coordinates and the monitoring depth value of the selected pixel to be shared (S3540), and transmitting the generated point cloud to a parking lot management processor (S3550).

[0472] As for storing the reference depth value corresponding to each detector pixel of the lidar device according to one embodiment (S3510), the contents described in the reference measurement step (S3100) may be applied, so redundant descriptions will be omitted.

[0473] Additionally, obtaining a monitoring depth value corresponding to each detector pixel of the lidar device according to one embodiment (S3520) can be understood as obtaining a current distance value corresponding to each detector pixel of the lidar device.

[0474] In addition, in selecting a target pixel to be shared by comparing the reference depth value and the monitoring depth value corresponding to each detector pixel of the lidar device according to one embodiment (S3530), if the reference depth value and the monitoring depth value differ by a preset value or more, it can be estimated that a change has occurred in a fixed environment, that is, it can mean that a new object has appeared in the corresponding pixel.

[0475] Accordingly, in selecting a target pixel for sharing by comparing the reference depth value and the monitoring depth value corresponding to each detector pixel of the lidar device according to one embodiment (S3530), a pixel in which the reference depth value and the monitoring depth value differ by a preset value or more can be selected as the target pixel for sharing.

[0476] In addition, in the case of the lidar device, since the direction covered by each detector pixel (i.e., (theta, phi)) is determined based on the pixel coordinates and monitoring depth values ​​of the selected shared target pixel according to one embodiment of the present invention, a three-dimensional coordinate for a point can be derived by using the direction covered by each detector pixel and the monitoring depth value for the detector pixel.

[0477] Additionally, since a point cloud is generated based on the selected shared target pixels at this time, it becomes possible to share only point data for pixels where changes have occurred in a fixed environment (i.e., pixels where new objects appear).

[0478] Additionally, at this time, the generated point cloud can dramatically increase the processing speed of the computer because it only contains point data for pixels where new objects appear among the entire point cloud.

[0479]

[0480] (8) [Monitoring Step (S3200) - Generating Parking Lot Management Information (S3250)]

[0481] According to one embodiment, the parking lot management information may include at least one of whether a parking slot is occupied, whether each parking slot is occupied for a plurality of parking slots, location coordinates of available parking slots, and the occupancy rate of the entire parking lot.

[0482] At this time, the most basic thing for generating parking lot management information is whether a parking slot is occupied, and when generating the occupancy status of each of multiple parking slots, the occupancy status of each parking slot, the location coordinates of parking slots available for parking, and the occupancy rate of the entire parking lot can be generated.

[0483] In one embodiment, generating whether a parking slot is occupied can be performed by the lidar processor.

[0484] FIG. 13 is a diagram illustrating an operation method of a lidar processor for generating occupancy status of a parking slot according to one embodiment.

[0485] Referring to FIG. 13, an operation method (3600) of a lidar processor for generating occupancy information of a parking slot according to one embodiment includes storing parking slot space information (S3610), generating a monitoring point cloud (S3620), calculating the number of points located within the parking slot space based on the generated monitoring point cloud (S3630), and determining that the corresponding parking slot is occupied if the number of calculated points is greater than or equal to a preset value (S3640).

[0486] At this time, since the above-described contents can be applied to storing parking slot space information according to one embodiment (S3610) and generating a monitoring point cloud (S3620), redundant descriptions will be omitted.

[0487] In addition, calculating the number of points located within a parking slot space based on the generated monitoring point cloud according to one embodiment (S3630) can be calculated based on point clouds for multiple frames.

[0488] For example, calculating the number of points located within a parking slot space based on the generated monitoring point cloud according to one embodiment (S3630) may include calculating the number of points located within a parking slot space in a point cloud for a frame at a current time, calculating the number of points located within a parking slot space in a point cloud for the previous N frames, and calculating the number of points located within a parking slot space as an average of the calculated numbers of points.

[0489] In addition, if the number of points calculated according to one embodiment is greater than or equal to a preset value, it is determined that the corresponding parking slot is occupied (S3640). The preset value may be the same depending on the parking slot, but is not limited thereto and may be different depending on the parking slot.

[0490] For example, the preset value for a first parking slot, which is close to the lidar device, may be greater than, but is not limited to, a preset value for a second parking slot, which is relatively far from the lidar device.

[0491] This is because the closer you are to the lidar device, the higher the density of point data per unit area, and the farther you are from the lidar device, the lower the density of point data per unit area.

[0492]

[0493] Additionally, according to one embodiment, generating whether a parking slot is occupied can be performed by the parking management processor.

[0494] FIG. 14 is a diagram illustrating an operation method of a parking lot management processor for generating occupancy status of a parking slot according to one embodiment.

[0495] Referring to FIG. 14, an operation method (3700) of a parking lot management processor for generating occupancy status of a parking slot according to one embodiment includes storing parking slot space information (S3710), obtaining individual point clouds from a plurality of lidar processors (S3720), generating an integrated point cloud based on the obtained individual point clouds (S3730), calculating the number of points located within the parking slot space based on the generated integrated point cloud (S3740), and determining that the corresponding parking slot is occupied if the number of calculated points is greater than or equal to a preset value (S3750).

[0496] At this time, the above-described contents can be applied to storing parking slot space information according to one embodiment (S3710), acquiring individual point clouds from multiple lidar processors (S3720), and generating an integrated point cloud based on the acquired individual point clouds (S3730), so redundant descriptions will be omitted.

[0497] In addition, calculating the number of points located within a parking slot space based on the generated integrated point cloud according to one embodiment (S3740) can be calculated based on the integrated point cloud for a plurality of frames.

[0498] For example, calculating the number of points located within a parking slot space based on the generated integrated point cloud according to one embodiment (S3740) may include calculating the number of points located within a parking slot space in the integrated point cloud for the frame at the current time, calculating the number of points located within a parking slot space in the integrated point cloud for the previous N frames, and calculating the number of points located within a parking slot space as an average of the calculated numbers of points.

[0499] In addition, if the number of points produced is greater than or equal to a preset value, the parking slot is determined to be occupied (S3750). The preset value may be the same for each parking slot, but is not limited thereto and may differ for each parking slot.

[0500] For example, a preset value for a first parking slot covered by one lidar device may be less than a preset value for a second parking slot covered by multiple lidar devices, but is not limited thereto.

[0501] This is because the number of points that need to be obtained may increase depending on the number of lidar devices covering one parking slot.

[0502]

[0503] Additionally, according to one embodiment, generating whether a parking slot is occupied may be performed by each of the plurality of lidar processors, but may be assisted by the parking management processor.

[0504] In this case, each of the multiple lidar processors may make different judgments about the same parking slot, or the parking management processor may generate auxiliary information if the parking slot is covered by multiple lidar devices.

[0505] This is to enable more accurate generation of parking slot occupancy by receiving assistance from a parking lot management processor capable of generating an integrated point cloud when multiple lidar processors each make different judgments about the same parking slot. In addition, in the case of a parking slot covered by multiple lidar devices, each lidar device may determine that the parking slot is not occupied, but when point clouds acquired from multiple lidar devices are combined, the parking slot may be determined to be occupied.

[0506] In addition, in addition to the above-described methods, generating whether a parking slot is occupied can also be performed by recognizing a sub-point data set corresponding to a vehicle and determining whether the recognized sub-point data set is located within a parking slot space, or can be determined in various ways, such as by determining based on the degree to which a bounding box corresponding to a vehicle overlaps a parking slot space.

[0507]

[0508] II. [Parking Lot Management System Using LiDAR Devices to Implement Find My Car Function]

[0509] 1. [Problem]

[0510] According to the conventional 'camera'-based parking lot management system, in response to the input of a vehicle number, it informs the user where the vehicle corresponding to that vehicle number is parked in the parking lot.

[0511] This is called the 'Find My Car' feature.

[0512] However, in order to enable this 'Find My Car' function, it is necessary to check the license plates of the vehicles in the parking lot, and for this purpose, a large number of cameras are installed in the parking lot where the parking lot management system that implements the 'Find My Car' function is installed, and each camera is installed to check the license plates of 4 to 8 vehicles.

[0513] That means that to check vehicle license plates for 1,000 parking slots, between 125 and 250 cameras would need to be installed.

[0514] Also, this is the situation for indoor parking lots, and for outdoor parking lots, there is not even a structure to install more than 100 cameras.

[0515] On the other hand, parking lot management systems using lidar devices have the problem that they cannot recognize vehicle license plates because the lidar devices have lower resolution than cameras, so they essentially have the problem that they cannot implement functions such as 'find my car'.

[0516] If both cameras and lidar devices are used, the 'Find My Car' function can be implemented in parking lot management systems that use lidar devices.

[0517] However, unlike conventional parking lot management systems, a method is required that can drastically reduce the number of cameras required.

[0518]

[0519] 2. [Configuration of a parking lot management system using a lidar device that implements the Find My Car function]

[0520] FIG. 15 and FIG. 16 are drawings for explaining the configuration of a parking lot management system according to one embodiment.

[0521] Referring to FIGS. 15 and 16, a parking lot management system according to one embodiment includes a plurality of lidar devices (4010), a plurality of lidar processors (4020), a parking control processor (4030), and at least one image acquisition device (4040).

[0522] At this time, the above-described contents can be applied to multiple lidar devices (4010), multiple lidar processors (4020), and parking control processors (4030), so redundant descriptions will be omitted.

[0523] According to one embodiment, at least one image acquisition device (4040) is installed at the entrance of a parking lot and is configured to acquire an image of a vehicle passing through the entrance of the parking lot and acquire vehicle number information based on the acquired image.

[0524] When configuring a parking lot management system as described above, the parking lot management processor can obtain number information of a vehicle passing through the entrance of the parking lot, and can obtain information on which parking slot a vehicle entering the parking lot is parked in.

[0525] However, in order to implement the 'Find My Car' function, the parking slot where the vehicle is parked and the vehicle number information must be mapped. However, it is not easy to map the number information of a vehicle passing through the parking lot entrance and the information about which parking slot the vehicle entering the parking lot is parked in, as these are separate pieces of information.

[0526] To this end, one could consider judging the similarity between the shape of the vehicle image acquired by the camera and the shape of the vehicle expressed in the point cloud of the lidar device, but this is a practically impossible solution because the properties of the image acquired from the camera and the point cloud are very different.

[0527] Hereinafter, a parking lot management system and an operation method of the parking lot management system for implementing a 'find my car' function according to the present invention will be described in more detail.

[0528]

[0529] FIG. 17 is a drawing for explaining an operation method of a parking lot management processor included in a parking lot management system according to one embodiment.

[0530] Referring to FIG. 17, an operation method (4100) of a parking lot management processor according to another embodiment includes obtaining a vehicle number of a vehicle passing through a parking lot entrance (S4110), generating a vehicle number list using the obtained vehicle number (S4120), setting an entry monitoring area (S4130), identifying a vehicle located in the entry monitoring area based on information obtained from a plurality of lidar processors (S4140), assigning a temporary vehicle ID to the identified vehicle to generate a temporary vehicle ID list (S4150), tracking the identified vehicle to obtain a parking slot ID for a parking location where the identified vehicle is parked (S4160), and matching and storing the parking slot ID and the vehicle number (S4170).

[0531] In obtaining the vehicle number of a vehicle passing through a parking lot entrance according to one embodiment (S4110), the vehicle number of the vehicle is obtained based on the license plate of the vehicle appearing in an image obtained from at least one image obtaining device installed at the parking lot entrance.

[0532] At this time, obtaining the vehicle number of a vehicle passing through the parking lot entrance according to one embodiment (S4110) may include obtaining an image from at least one image obtaining device installed at the parking lot entrance and analyzing the obtained image to obtain the vehicle number, but is not limited thereto, and may include receiving an image and vehicle number from at least one image obtaining device installed at the parking lot entrance.

[0533] Additionally, in generating a vehicle number list using the acquired vehicle numbers according to one embodiment (S4120), the vehicle number list can be sorted according to the order in which the vehicle numbers were acquired.

[0534] In addition, in generating a vehicle number list using the acquired vehicle number according to one embodiment (S4120), the vehicle number list can be sorted according to the order in which images corresponding to the vehicle numbers were acquired.

[0535] In addition, in generating a vehicle number list using the acquired vehicle number according to one embodiment (S4120), the vehicle number list may include information on the time at which the vehicle number was acquired.

[0536] In addition, in generating a vehicle number list using the acquired vehicle number according to one embodiment (S4120), the vehicle number list may include information on the time at which an image corresponding to the vehicle number was acquired.

[0537] In addition, in setting an entry monitoring area according to one embodiment (S4130), the entry monitoring area may be set to be adjacent to the entrance of the parking lot.

[0538] In addition, in setting an entry monitoring area according to one embodiment (S4130), the entry monitoring area may be set to be adjacent to the entrance of the parking lot, but may be set to be located within the parking lot.

[0539] Below, the entry surveillance area is described in more detail.

[0540] Fig. 18 is a drawing for explaining an entrance surveillance area according to one embodiment.

[0541] Referring to FIG. 18, an entry monitoring area (4251, 4252) can be set in a parking lot management system according to one embodiment.

[0542] At this time, the entry monitoring area (4251, 4252) may mean an area for determining whether a new vehicle is identified based on point clouds acquired from multiple lidar devices within the parking lot.

[0543] At this time, the entrance surveillance area (4251, 4252) can be set adjacent to the entrance of the parking lot (4241, 4242).

[0544] At this time, the reason why the entry surveillance area (4251, 4252) is set adjacent to the entrance (4241, 4242) of the parking lot may be to limit the area for identifying a new vehicle entering the parking lot, so as to prevent an error in the matching relationship between the order and / or time at which the vehicle number is recognized and the order and / or time at which the newly identified vehicle is identified.

[0545] That is, if the entry monitoring area (4251, 4252) is not set separately, newly identified vehicles may occur due to certain errors throughout the parking lot, and in this case, a mismatch occurs between the order and / or time at which the vehicle number is recognized and the order and / or time at which the newly identified vehicles are identified, which greatly increases the possibility of an error in the 'Find My Car' function.

[0546] In addition, if the entry surveillance area (4251, 4252) is not set adjacent to the parking lot entrance (4241, 4242), the times at which vehicles entering the parking lot pass through the entry surveillance area may all be different, and depending on the movement path of the vehicles entering the parking lot, they may not pass through the entry surveillance area.

[0547] Additionally, at this time, the entrance surveillance area (4251, 4252) can be set to be located within the parking lot.

[0548] This is to clearly define the chronological relationship between the order and / or time at which vehicle numbers are recognized and the order and / or time at which newly identified vehicles are identified, thereby enabling more clear matching relationships to be determined.

[0549] That is, if the entry surveillance area (4251, 4252) is located from the outside of the parking lot to the inside of the parking lot, the chronological relationship between the order and / or time at which the vehicle number is recognized and the order and / or time at which the vehicle is newly identified is not clear, so there is a possibility that an error in the matching relationship may occur when vehicles enter consecutively.

[0550] In addition, if the entry surveillance area (4251, 4252) is set outside the parking lot, a vehicle that has not entered the parking lot may become a newly identified vehicle, thus increasing the possibility of a mismatch occurring between the vehicle number and the identified vehicle.

[0551] Additionally, the entrance surveillance area (4251, 4252) can be set to correspond to each entrance (4241, 4242) of the parking lot.

[0552] For example, the first entrance surveillance area (4251) may be set to be adjacent to the first entrance (4241) of the parking lot but located within the parking lot, and the second entrance surveillance area (4252) may be set to be adjacent to the second entrance (4242) of the parking lot but located within the parking lot.

[0553] Additionally, multiple lidar devices of the parking lot management system are installed on the periphery and inside of the parking lot, and each of the multiple lidar devices is arranged to have an assigned surveillance area.

[0554] Additionally, in this case, the surveillance area of ​​at least one of the multiple lidar devices of the parking lot management system may be positioned to include the entry surveillance area (4251, 4252).

[0555] For example, the surveillance area of ​​the first lidar device (4211) of the parking lot management system may be positioned to include the first entry surveillance area (4251), and the surveillance area of ​​the second lidar device (4212) may be positioned to include the second entry surveillance area (4252).

[0556]

[0557] Referring again to FIG. 17, identifying a vehicle located in an entrance surveillance area based on information acquired from a plurality of lidar processors according to one embodiment (S4140) may include identifying a vehicle located in an entrance surveillance area based on individual point clouds acquired from the plurality of lidar processors.

[0558] More specifically, identifying a vehicle located in an entrance surveillance area based on information acquired from a plurality of lidar processors according to one embodiment (S4140) may include acquiring individual point clouds from the plurality of lidar processors, generating an integrated point cloud for the entire parking lot based on the acquired individual point clouds, and identifying a vehicle located in the entrance surveillance area based on the integrated point cloud.

[0559] At this time, since the above-described contents can be applied to obtaining individual point clouds from the plurality of lidar processors and generating an integrated point cloud for the entire parking lot based on the obtained individual point clouds, redundant descriptions will be omitted, and since the above-described contents for obtaining identification information for at least one object can be applied to identifying a vehicle located in the entrance surveillance area based on the integrated point cloud, redundant descriptions will be omitted.

[0560] Additionally, identifying a vehicle located in an entrance surveillance area based on information acquired from a plurality of lidar processors according to one embodiment (S4140) may include identifying a vehicle located in an entrance surveillance area based on some point data included in individual point clouds acquired from the plurality of lidar processors.

[0561] More specifically, identifying a vehicle located in an entrance surveillance area based on information acquired from a plurality of lidar processors according to one embodiment (S4140) may include acquiring some point data included in individual point clouds from the plurality of lidar processors, generating an integrated point cloud for the entire parking lot based on the acquired point data, and identifying a vehicle located in the entrance surveillance area based on the integrated point cloud.

[0562] At this time, the above-described contents can be applied to obtaining some point data included in individual point clouds from the plurality of lidar processors and generating an integrated point cloud for the entire parking lot based on the obtained point data, so redundant descriptions will be omitted, and the above-described contents for obtaining identification information for at least one object can be applied to identifying a vehicle located in the entrance surveillance area based on the integrated point cloud, so redundant descriptions will be omitted.

[0563] In addition, identifying a vehicle located in an entrance surveillance area based on information acquired from a plurality of lidar processors according to one embodiment (S4140) may include identifying a vehicle located in an entrance surveillance area based on center position information and bounding box information corresponding to some of the point data included in individual point clouds acquired from the plurality of lidar processors.

[0564] More specifically, identifying a vehicle located in an entrance monitoring area based on information acquired from a plurality of lidar processors according to one embodiment (S4140) may include: acquiring center position information and bounding box information corresponding to some point data included in individual point clouds from the plurality of lidar processors; positioning the acquired center position information and bounding box information on an integrated coordinate system; and identifying a vehicle located in the entrance monitoring area based on the center position information and bounding box information positioned on the integrated coordinate system.

[0565] At this time, the above-described contents can be applied to obtaining center position information and bounding box information corresponding to some point data included in individual point clouds from the plurality of lidar processors, and positioning the obtained center position information and bounding box information on an integrated coordinate system, so redundant descriptions will be omitted. In addition, the contents of obtaining identification information for at least one object described above can be applied to identifying a vehicle located in the entrance surveillance area based on the center position information and bounding box information positioned on the integrated coordinate system, so redundant descriptions will be omitted.

[0566] In addition, in generating a temporary vehicle ID list by assigning a temporary vehicle ID to an identified vehicle according to one embodiment (S4150), the temporary vehicle ID list can be sorted according to the order in which the temporary vehicle IDs are assigned.

[0567] In addition, in generating a temporary vehicle ID list by assigning a temporary vehicle ID to an identified vehicle according to one embodiment (S4150), the temporary vehicle ID list can be sorted according to the order in which the vehicles corresponding to the temporary vehicle IDs are identified.

[0568] In addition, in generating a temporary vehicle ID list by assigning a temporary vehicle ID to an identified vehicle according to one embodiment (S4150), the temporary vehicle ID list may include information on the time at which the temporary vehicle ID was assigned.

[0569] In addition, in generating a temporary vehicle ID list by assigning a temporary vehicle ID to an identified vehicle according to one embodiment (S4150), the temporary vehicle ID list may include information on the time at which a vehicle corresponding to the temporary vehicle ID was identified.

[0570] In addition, since the above-described contents can be applied to tracking the identified vehicle according to one embodiment and obtaining a parking slot ID for the parking location where the identified vehicle is parked (S4160), redundant descriptions will be omitted.

[0571] In addition, since the contents of determining whether a parking slot is occupied can be applied to obtaining a parking slot ID for a parking location where the identified vehicle is parked by tracking the identified vehicle according to one embodiment (S4160), redundant descriptions will be omitted.

[0572] Additionally, tracking an identified vehicle according to one embodiment and obtaining a parking slot ID for a parking location where the identified vehicle is parked (S4160) may include tracking the identified vehicle, determining whether the identified vehicle occupies a parking slot, and, if it is determined that the identified vehicle occupies a specific parking slot, obtaining a parking slot ID for the parking slot occupied by the identified vehicle.

[0573] In addition, the storing of the parking slot ID and the vehicle number by matching them according to one embodiment (S4170) may include storing the parking slot ID and the vehicle number by matching the temporary vehicle ID and the vehicle number corresponding to the temporary vehicle ID based on the vehicle number list and the temporary vehicle ID list.

[0574] At this time, matching of the temporary vehicle ID and the vehicle number corresponding to the temporary vehicle ID can be performed based on the order of the temporary vehicle ID in the temporary vehicle ID list and the order of the vehicle number in the vehicle number list.

[0575] In addition, at this time, matching of the temporary vehicle ID and the vehicle number corresponding to the temporary vehicle ID may be performed by further considering information on the time at which the vehicle included in the temporary vehicle ID list was identified and information on the time at which the vehicle number included in the vehicle number list was acquired.

[0576] Additionally, at this time, matching of the temporary vehicle ID and the vehicle number corresponding to the temporary vehicle ID can be performed based on vehicle number acquisition time information included in the vehicle number list and vehicle identification time information included in the temporary vehicle ID list.

[0577]

[0578] In addition, when a parking lot includes multiple entrances as in the situation illustrated in FIG. 18, matching of a temporary vehicle ID and a vehicle number corresponding to the temporary vehicle ID can be performed based on an image acquisition device located at each of the multiple entrances and an entry surveillance area corresponding to each of the multiple entrances.

[0579] More specifically, if the parking lot entrance includes a first entrance and a second entrance,

[0580] The parking lot management processor may obtain the vehicle numbers of vehicles passing through the first entrance of the parking lot through a first image acquisition device located at the first entrance of the parking lot, and create a first vehicle number list for the first entrance using the obtained vehicle numbers, and may obtain the vehicle numbers of vehicles passing through the second entrance through a second image acquisition device located at the second entrance of the parking lot, and create a second vehicle number list for the second entrance using the obtained vehicle numbers.

[0581] In addition, in this case, the parking lot management processor may set a first entry surveillance area corresponding to the first entrance, identify a vehicle located in the first entry surveillance area based on information obtained from the plurality of lidar processors, assign a temporary vehicle ID to the vehicle, and generate a first temporary vehicle ID list corresponding to the first entrance, and set a second entry surveillance area corresponding to the second entrance, identify a vehicle located in the second entry surveillance area based on information obtained from the plurality of lidar processors, assign a temporary vehicle ID to the vehicle, and generate a second temporary vehicle ID list corresponding to the second entrance.

[0582] Additionally, in this case, the parking lot management processor may match the temporary vehicle ID of the first temporary vehicle ID list with the vehicle number of the first vehicle number list based on the first vehicle number list and the first temporary vehicle ID list, and may match the temporary vehicle ID of the second temporary vehicle ID list with the vehicle number of the second vehicle number list based on the second vehicle number list and the second temporary vehicle ID list.

[0583]

[0584] The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, etc., alone or in combination. The program commands recorded on the medium may be those specially designed and configured for the embodiment or may be those known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program commands, such as ROMs, RAMs, and flash memories. Examples of the program commands include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiment, and vice versa.

[0585] Although the embodiments described above have been described by way of limited examples and drawings, those skilled in the art will appreciate that various modifications and variations can be made based on the above teachings. For example, appropriate results can still be achieved even if the described techniques are performed in a different order than described, and / or components of the described systems, structures, devices, circuits, etc. are combined or combined in a different manner than described, or are replaced or substituted with other components or equivalents.

[0586] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims described below.

[0587]

[0588] As described above, the relevant matters have been described in the best mode for carrying out the invention.

Claims

1. As a parking lot management system for monitoring the current status of the parking lot, An image acquisition device installed at the entrance of the above parking lot and acquiring images of vehicles passing through the entrance; A plurality of lidar devices installed on the periphery and inside of the above parking lot, wherein each of the plurality of lidar devices has an assigned surveillance area; A plurality of lidar processors connected to at least one lidar device among the plurality of lidar devices, wherein each of the plurality of lidar processors generates an individual point cloud for a surveillance area of ​​at least one corresponding lidar device; and A parking lot management processor communicating with the image acquisition device and the plurality of lidar processors; The above parking lot management processor, Obtaining the license plate number of a vehicle passing through the above parking lot entrance - wherein the license plate number of the vehicle is obtained based on the license plate number of the vehicle appearing in the above acquired image - A list of vehicle numbers is generated using the acquired vehicle numbers, wherein the list of vehicle numbers is sorted in the order in which the vehicle numbers are acquired. The above parking lot management processor, Setting an entrance surveillance area - wherein the entrance surveillance area is set to be adjacent to the entrance of the parking lot, but is set to be located within the parking lot -, Identifying a vehicle located in the entry surveillance area based on information obtained from the plurality of lidar processors; Create a temporary vehicle ID list by assigning a temporary vehicle ID to the identified vehicle, wherein the temporary vehicle ID list is sorted in the order in which the temporary vehicle IDs are assigned. Tracking the identified vehicle to obtain a parking slot ID for the parking location where the identified vehicle is parked; The parking slot ID and the vehicle number are matched and stored by matching the temporary vehicle ID and the vehicle number corresponding to the temporary vehicle ID based on the vehicle number list and the temporary vehicle ID list. Parking lot management system.

2. In paragraph 1, Identifying a vehicle located in the entrance surveillance area based on information obtained from the plurality of lidar processors, Acquiring individual point clouds from the plurality of lidar processors; Generating an integrated point cloud for the entire parking lot based on the acquired individual point clouds; Identifying a vehicle located in the entrance surveillance area based on the integrated point cloud; Parking lot management system.

3. In paragraph 1, Identifying a vehicle located in the entrance surveillance area based on information obtained from the plurality of lidar processors, Obtaining some point data included in individual point clouds from the plurality of lidar processors; Generate an integrated point cloud for the entire parking lot based on the acquired point data; Identifying a vehicle located in the entrance surveillance area based on the integrated point cloud; Parking lot management system.

4. In paragraph 1, Identifying a vehicle located in the entrance surveillance area based on information obtained from the plurality of lidar processors, Obtaining center position information and bounding box information corresponding to some point data included in individual point clouds from the plurality of lidar processors; Positioning the acquired center position information and bounding box information on the integrated coordinate system; Identifying a vehicle located in the entrance surveillance area based on center position information and bounding box information located on the integrated coordinate system; Parking lot management system.

5. In paragraph 1, The above parking lot entrances include the first entrance and the second entrance, The above parking lot management processor, Obtaining the vehicle number of a vehicle passing through the first entrance of the above parking lot, and using the obtained vehicle number, generating a first vehicle number list for the first entrance, Obtaining the vehicle number of a vehicle passing through the second entrance of the above parking lot, and using the obtained vehicle number, generating a second vehicle number list for the second entrance, The above parking lot management processor A first entry surveillance area corresponding to the first entrance is set, and a vehicle located in the first entry surveillance area is identified based on information obtained from the plurality of lidar processors and a temporary vehicle ID is assigned to generate a first temporary vehicle ID list corresponding to the first entrance. A second entry surveillance area corresponding to the second entrance is set, and a vehicle located in the second entry surveillance area is identified based on information obtained from the plurality of lidar processors and a temporary vehicle ID is assigned to generate a second temporary vehicle ID list corresponding to the second entrance. The above parking lot management processor Matching the temporary vehicle ID of the first temporary vehicle ID list with the vehicle number of the first vehicle number list based on the first vehicle number list and the first temporary vehicle ID list, Matching the temporary vehicle ID of the second temporary vehicle ID list with the vehicle number of the second vehicle number list based on the second vehicle number list and the second temporary vehicle ID list Parking lot management system.

6. In paragraph 1, The matching of the temporary vehicle ID and the vehicle number corresponding to the temporary vehicle ID is performed based on the order of the temporary vehicle ID in the temporary vehicle ID list and the order of the vehicle number in the vehicle number list. Parking lot management system.

7. In paragraph 6, The above vehicle number list includes vehicle number and vehicle number acquisition time information, The above temporary vehicle ID list includes information on the temporary vehicle ID and the time at which the vehicle corresponding to the temporary vehicle ID was identified, The matching of the temporary vehicle ID and the vehicle number corresponding to the temporary vehicle ID is performed by further considering the information on the time at which the vehicle included in the temporary vehicle ID list was identified and the information on the time at which the vehicle number included in the vehicle number list was acquired. Parking lot management system.

8. In paragraph 1, The surveillance area of ​​at least one of the plurality of lidar devices is positioned to include the entrance surveillance area. Parking lot management system.

9. As a parking lot management system for monitoring the current status of the parking lot, An image acquisition device installed at the entrance of the above parking lot and acquiring images of vehicles passing through the entrance; A plurality of lidar devices installed on the periphery and inside of the above parking lot, wherein each of the plurality of lidar devices has an assigned surveillance area; A plurality of lidar processors connected to at least one lidar device among the plurality of lidar devices, wherein each of the plurality of lidar processors generates an individual point cloud for a surveillance area of ​​at least one corresponding lidar device; and A parking lot management processor communicating with the image acquisition device and the plurality of lidar processors; The above parking lot management processor, Obtaining the license plate number of a vehicle passing through the above parking lot entrance - wherein the license plate number of the vehicle is obtained based on the license plate number of the vehicle appearing in the above acquired image - Generating a list of vehicle numbers using the acquired vehicle numbers, wherein the list of vehicle numbers includes information on when the vehicle numbers were acquired. The above parking lot management processor, Setting up an entrance surveillance area - wherein the entrance surveillance area is set adjacent to the entrance of the parking lot -, Identifying a vehicle located in the entry surveillance area based on information obtained from the plurality of lidar processors; Create a temporary vehicle ID list by assigning a temporary vehicle ID to the identified vehicle, wherein the temporary vehicle ID list includes information on the time at which the vehicle corresponding to the temporary vehicle ID was identified. Tracking the identified vehicle to obtain a parking slot ID for the parking location where the identified vehicle is parked; The parking slot ID and the vehicle number are matched and stored by matching the temporary vehicle ID and the vehicle number corresponding to the temporary vehicle ID based on the vehicle number acquisition time information included in the vehicle number list and the vehicle identification time information included in the temporary vehicle ID list. Parking lot management system.

10. In paragraph 9, Identifying a vehicle located in the entrance surveillance area based on information obtained from the plurality of lidar processors, Acquiring individual point clouds from the plurality of lidar processors; Generating an integrated point cloud for the entire parking lot based on the acquired individual point clouds; Identifying a vehicle located in the entrance surveillance area based on the integrated point cloud; Parking lot management system.

11. In paragraph 9, Identifying a vehicle located in the entrance surveillance area based on information obtained from the plurality of lidar processors, Obtaining some point data included in individual point clouds from the plurality of lidar processors; Generate an integrated point cloud for the entire parking lot based on the acquired point data; Identifying a vehicle located in the entrance surveillance area based on the integrated point cloud; Parking lot management system.

12. In paragraph 9, Identifying a vehicle located in the entrance surveillance area based on information obtained from the plurality of lidar processors, Obtaining center position information and bounding box information corresponding to some point data included in individual point clouds from the plurality of lidar processors; Positioning the acquired center position information and bounding box information on the integrated coordinate system; Identifying a vehicle located in the entrance surveillance area based on center position information and bounding box information located on the integrated coordinate system; Parking lot management system.

13. In paragraph 9, The above parking lot entrances include the first entrance and the second entrance, The above parking lot management processor, Obtaining the vehicle number of a vehicle passing through the first entrance of the above parking lot, and using the obtained vehicle number, generating a first vehicle number list for the first entrance, Obtaining the vehicle number of a vehicle passing through the second entrance of the above parking lot, and using the obtained vehicle number, generating a second vehicle number list for the second entrance, The above parking lot management processor A first entry surveillance area corresponding to the first entrance is set, and a vehicle located in the first entry surveillance area is identified based on information obtained from the plurality of lidar processors and a temporary vehicle ID is assigned to generate a first temporary vehicle ID list corresponding to the first entrance. A second entry surveillance area corresponding to the second entrance is set, and a vehicle located in the second entry surveillance area is identified based on information obtained from the plurality of lidar processors and a temporary vehicle ID is assigned to generate a second temporary vehicle ID list corresponding to the second entrance. The above parking lot management processor Matching the temporary vehicle ID of the first temporary vehicle ID list with the vehicle number of the first vehicle number list based on the first vehicle number list and the first temporary vehicle ID list, Matching the temporary vehicle ID of the second temporary vehicle ID list with the vehicle number of the second vehicle number list based on the second vehicle number list and the second temporary vehicle ID list Parking lot management system.

14. In paragraph 9, The surveillance area of ​​at least one of the plurality of lidar devices is positioned to include the entrance surveillance area. Parking lot management system.

Images