System for supporting automatic valet parking and method thereof

Abstract

To disclose a method for exactly grasping a position of a vehicle in a map of a parking lot, and a generation method of a digital map by which the exact position of the vehicle can be grasped.SOLUTION: An automatic valet parking method of a vehicle can include operations of: acquiring image data in the surroundings by using at least one sensor; recognizing a unique marker installed in a parking lot from the image data to set position information of the vehicle; and recognizing anonymous marker installed in the parking lot from the image data to correct the position information of the vehicle.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a system and method for assisting automatic valet parking, and more particularly, to a method for grasping the exact position of a vehicle in a parking lot map and a method for generating a digital map capable of grasping the exact position of the vehicle, and to a system and method for assisting automatic valet parking related thereto.

Background Art

[0002] An automatic valet parking system can be said to be a system that, when a driver stops the vehicle in a drop-off area and gets out, drives the vehicle without the driver on board to an empty parking space in the parking lot to complete parking.

[0003] Also, when a driver calls, the automatic valet parking system automatically drives the parked vehicle to a pick-up area so that the driver can pick up the vehicle in the pick-up area.

[0004] In addition, the automatic valet parking system can cooperate with a vehicle capable of automatic driving in the absence of a driver and park the vehicle in an empty parking space.

[0005] In order for an automatic valet parking system to control an autonomous vehicle and drive it to an empty parking space, it is necessary to clearly grasp the current position of the vehicle in the parking lot. However, there may be places in the parking lot where GPS signals and the like do not reach. Also, in a multi-story parking lot, it is not easy to clearly grasp the position of the vehicle.

Summary of the Invention

Problems to be Solved by the Invention

[0006] In order for an automatic valet parking system to accurately guide an autonomous vehicle to a parking space, it is necessary to accurately grasp the position of the vehicle in the parking lot map.

[0007] An object of the present invention is to disclose a method for grasping the exact position of a vehicle in a parking lot map and a method for generating a digital map in which the exact position of the vehicle can be grasped.

[0008] The technical problems of the present invention are not limited to the above-mentioned technical problems. Incidentally, other technical problems not mentioned in the present disclosure will also be clearly understood by those having ordinary knowledge in the technical field to which the present disclosure pertains from the following description.

Means for Solving the Problems

[0009] The vehicle of the present invention includes a sensor that acquires image data around the vehicle using at least one sensor, a controller that controls the operation of at least one function of the vehicle, a communication device that communicates with an automatic valet parking server, and a processor that controls the overall operation of the vehicle. The processor can recognize a unique marker installed in the parking lot from the image data to set the position information of the vehicle, and can recognize an anonymous marker installed in the parking lot from the image data to correct the position information of the vehicle.

[0010] The automatic valet parking server of the present invention also includes a storage unit that stores digital map information, a communication device that communicates with the vehicle, and a processor. The processor can recognize a unique marker installed in the parking lot from the image data acquired from the vehicle during the execution of automatic valet parking for the vehicle to set the position information of the vehicle, and can recognize an anonymous marker installed in the parking lot from the image data to correct the position information of the vehicle.

[0011] The automatic valet parking method for a vehicle according to the present invention can include an operation of generating peripheral image data using at least one sensor, an operation of recognizing a unique marker installed in a parking lot from the image data to set the position information of the vehicle, and an operation of recognizing an anonymous marker installed in the parking lot from the image data to correct the position information of the vehicle.

Advantages of the Invention

[0012] According to the present invention, the position of an automatic driving vehicle in a parking lot can be accurately grasped, and parking by an automatic valet parking system can be easily performed without errors.

Brief Description of the Drawings

[0013]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Embodiments for Carrying Out the Invention

[0014] Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

[0015] The configuration of the present invention and the effects it exhibits will be clearly understood from the following detailed description. On the other hand, the same components in the present invention will be assigned the same reference numerals as much as possible even if they are shown in other drawings. Also, when it is determined that there is a concern about obscuring the gist of the present disclosure, a specific description of known configurations may be omitted.

[0016] Prior to the specific description of the present invention, the terms used in the present invention can be defined as follows.

[0017] The vehicle is a vehicle equipped with an ADS (Automated Driving System) and capable of autonomous driving. For example, the vehicle can perform at least one of steering, accelerating, decelerating, changing lanes, and stopping (or halting) by the ADS without the driver's operation. The ADS can include, for example, at least one of PDCMS (Pedestrian Detection and Collision Mitigation System), LCDAS (Lane Change Decision Aid System), LDWS (Land Departure Warning System), ACC (Adaptive Cruise Control), LKAS (Lane Keeping Assistance System), RBDPS (Road Boundary Departure Prevention System), CSWS (Curve Speed Warning System), FVCWS (Forward Vehicle Collision Warning System), and LSF (Low Speed Following).

[0018] Here, a driver is a human who uses the vehicle and receives the service of the autonomous driving system.

[0019] The control authority of a vehicle is the authority to control at least one component of the vehicle and / or at least one function of the vehicle. At least one function of the vehicle can include, for example, a steering function, an acceleration function, a deceleration function (or a braking function), a lane change function, a lane detect function, a lateral control function, an obstacle recognition and distance sensing function, a powertrain control function, a safe area sensing function, an Engine on / off function, at least one of Power on / off and vehicle lock / unlock. However, the functions of the vehicle as described above are only examples for helping understanding, and the embodiments of the present invention are not limited thereto.

[0020] The vehicle is a vehicle with an automatic valet parking function.

[0021] The automatic valet parking server monitors the vehicles in the parking lot, determines a guide route for the vehicle to drive to a parking space (target position), determines an allowed driving area, and transmits a control command to the vehicle to instruct the automatic driving of the vehicle. Also, in case of emergency, an emergency stop command can be transmitted to stop the vehicle.

[0022] The infrastructure can be a parking facility or a sensor arranged in the parking facility. Also, the infrastructure can include a parking gate and, according to one embodiment, can be a concept including an automatic valet parking server.

[0023] The target position can be an area where the vehicle parks, that is, an available parking space. Also, the target position in the situation where the vehicle leaves the parking lot can be an area where the driver boards, that is, a boarding area.

[0024] The guide route can be the route that the vehicle passes through to reach the target position. For example, in the situation where parking is executed, it is the route from the getting-off area to the available parking space. Also, the guide route can be in the form of moving forward 50m or turning left at a corner, etc.

[0025] The driving route can be the route that the vehicle follows.

[0026] The permitted driving area can be the area where driving is permitted, such as the driving route in a parking lot. Also, the permitted driving area can be defined by a partition wall, parked vehicles, or parking lines.

[0027] FIG. 1 is a drawing showing an automatic valet parking system according to the present invention.

[0028] As shown in FIG. 1, the automatic valet parking system 10 can include an infrastructure 11 and an automatic valet parking device 13.

[0029] The infrastructure 11 can mean a device or system for operating, managing, and executing the automatic valet parking system 10. For example, the infrastructure 11 can be a parking facility. According to an embodiment, the infrastructure 11 can include sensors, communication devices, warning devices, display devices, and a server for controlling the aforementioned devices. Also, the infrastructure 11 can include a parking gate and an automatic valet parking server for controlling vehicles.

[0030] The automatic valet parking device 13 can mean a vehicle that performs automatic valet parking. The automatic valet parking device 13 can mean a component or a set of components included in a vehicle capable of performing automatic valet parking.

[0031] FIG. 2 is a block diagram of a vehicle and an automatic valet parking server according to the present invention.

[0032] The vehicle and the automatic valet parking server shown in FIG. 2 may each be composed of one chip, one component, or one electronic circuit, or may be composed of a combination of chips, components, and / or electronic circuits. Some of the components shown in FIG. 2 may be separated into multiple components and may be composed of different chips, different components, or different electronic circuits from each other. Also, some components may be combined with each other and may be composed of one chip, one component, or one electronic circuit. Some of the components shown in FIG. 2 may be omitted, or other components not shown may be added. Regarding at least some of the components in FIG. 2, they will be described below with reference to the drawings.

[0033] As shown in FIG. 2, the automatic valet parking server 50 can include a processor 51, a storage unit 53, and a communication device 55.

[0034] The storage unit 53 can store digital map information of a parking lot where automatic valet parking is possible.

[0035] Also, the storage unit 53 can store statically or temporarily all the information necessary for automatic valet parking, including components such as vehicles parked in the parking lot and infrastructure. The said information can be stored as information included in the digital map.

[0036] The processor 51 can control the automatic valet parking for the vehicle 100. The processor 51 generates all the control signals for the vehicle 100 to perform automatic valet parking, transmits them to the vehicle 100 via the communication device 55, and controls the vehicle 100 so that automatic parking can be performed. In this case, all the information for automatic valet parking is stored in the automatic valet parking server. And instead of the processor 130 of the vehicle 100, the processor 51 of the automatic valet parking server can generate and transmit control commands to be provided to the controller 120.

[0037] The automatic valet parking server 50 can command the vehicle 100 to perform automatic valet parking by itself. In this case, the processor 51 of the automatic valet parking server 50 can provide the vehicle 100 with a digital map related to the parking lot, a target position for the vehicle 100 to park, and information on a guide route for the vehicle 100 to drive for parking. The vehicle 100 can receive the above-mentioned information from the automatic valet parking server and perform automatic driving by itself to achieve automatic valet parking.

[0038] As shown in FIG. 2, the vehicle 100 can include a sensor 110, a controller 120, a processor 130, a display 140, and a communication device 150.

[0039] The sensor 110 can sense the surrounding environment of the vehicle 100 using at least one sensor and generate data related to the surrounding environment based on the sensing result. For example, the sensor 110 can obtain information about objects (e.g., other vehicles, people, objects, curbs, guardrails, lanes, obstacles, etc.) around the vehicle based on the sensing data obtained from at least one sensor. The information about the objects around the vehicle can include at least one of the position of the object, the size of the object, the shape of the object, the distance to the object, and the relative speed with respect to the object. As another example, the sensor 110 can measure the position of the vehicle 100 using at least one sensor. The sensor 110 can include at least one of, for example, a camera, a Light Detection and Ranging (LIDAR), a Radio Detection and Ranging (RADAR), an ultrasonic sensor, an infrared sensor, and a position measurement sensor. However, the above-mentioned sensors are only examples for the sake of understanding, and the sensors of the present disclosure are not limited thereto.

[0040] The camera can capture the surroundings of the vehicle and generate image data including objects located in front of, behind, and / or to the side of the vehicle 100. The lidar can use light (or laser) to generate information about objects located in front of, behind, and / or to the side of the vehicle 100. The radar can use electromagnetic waves (or radio waves) to generate information about objects located in front of, behind, and / or to the side of the vehicle 100. The ultrasonic sensor can use ultrasonic waves to generate information about objects located in front of, behind, and / or to the side of the vehicle 100. The infrared sensor can use infrared rays to generate information about objects located in front of, behind, and / or to the side of the vehicle 100.

[0041] The position measurement sensor can measure the current position of the vehicle 100. The position measurement sensor can include at least one of a GPS (Global Positioning System) sensor, a DGPS (Differential Global Posioning System) sensor, and a GNSS (Global Navigation Satellite System) sensor. The position measurement sensor can generate position data of the vehicle 100 based on a signal generated by at least one of the GPS sensor, the DGPS sensor, and the GNSS sensor.

[0042] The controller 120 can control the operation of at least one component of the vehicle 100 and / or at least one function of the vehicle 100 under the control of the processor 130. The at least one function can include, for example, at least one function among a steering function, an acceleration function (or a longitudinal acceleration function), a deceleration function (or a longitudinal deceleration function, a braking function), a lane change function, a lane sensing function, an obstacle recognition and distance sensing function, a lateral control function, a power train control function, a safe area sensing function, engine on / off, power on / off, and lock / unlock of the vehicle 100.

[0043] The controller 120 can control the operation of at least one component of the vehicle 100 and / or at least one function of the vehicle 100 for the automatic driving of the vehicle 100 under the control of the processor 130. For example, the controller 120 can control the operation of at least one of the steering function, acceleration function, deceleration function, lane change function, lane sensing function, lateral control function, obstacle recognition and distance sensing function, power train control function, and safe area detection function for automatic driving.

[0044] The display 140 can visually display information about the vehicle 100. For example, the display 140 can provide various information about the state of the vehicle 100 to the driver of the vehicle 100 under the control of the processor 130. The various information about the state of the vehicle 100 can include at least one of various components included in the vehicle 100 and / or information indicating whether the normal operation of at least one function of the vehicle 100 is possible, and information indicating the driving state of the vehicle 100. However, when performing automatic valet parking without a driver, the display 140 may be turned off without displaying information.

[0045] The communication device 150 can exchange data with the infrastructure 11. Such communication can be referred to as vehicle-to-infrastructure (V2I) communication. In addition, the communication device 150 can exchange data with other vehicles. Such communication can be referred to as vehicle-to-vehicle (V2V) communication. Also, V2I communication and V2V communication can be integrated and referred to as vehicle-to-everything (V2X) communication. The communication device 150 can receive data transmitted from the infrastructure 11 (such as target position, guidance route, or command), process the received data, and transmit it to the processor 130. In addition, the communication device 150 can transmit data generated in the vehicle 100 to the infrastructure 11. The communication device 150 can exchange data with the terminal of the driver of the vehicle 100.

[0046] The communication device 150 can transmit or receive data using a wireless communication protocol or a wired communication protocol. For example, the wireless communication protocol may include Wireless LAN (WLAN), DLNA (registered trademark) (Digital Living Network Alliance), Wireless Broadband (Wibro), World Interoperability for Microwave Access (Wimax), Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), Code Division Multi Access 2000 (CDMA2000), Enhanced Voice-Data Optimized or Enhanced Voice-Data Only (EV-DO), Wideband CDMA (registered trademark) (WCDMA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), IEEE802.16, Long Term Evolution (LTE), Long Term Evolution-Advanced (LTE-A), Wireless Mobile Broadband Service (WMBS), Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra-Wideband (UWB), ZigBee, Near Field Communication (NFC), Ultra Sound Communication (USC), Visible Light Communication (VLC), Wi-Fi, Wi-Fi Direct, and the like.In addition, the wired communication protocols may include wired LAN (Local Area Network), wired WAN (Wide Area Network), high-speed power line communication (Power Line Communication: PLC), USB communication, Ethernet, serial communication, optical / coaxial cables, etc. However, the wireless communication protocol and the wired communication protocol are not limited to those described above, and all of these can be included as long as they are protocols that can provide a communication environment with other devices.

[0047] The processor 130 can control the overall operation of the vehicle 100. The processor 130 includes an ECU (Electrical Control Unit) that can integrally control the components within the vehicle 100. For example, the processor 130 includes a CPU (Central Processing Unit) or an MCU (Micro Processing Unit) capable of performing arithmetic processing.

[0048] The processor 130 can control the controller 120 based on the data received via the sensor 110 and the communication device 150. According to an embodiment, the processor 130 can generate a control signal for controlling the vehicle 100 based on the data received from the infrastructure 11, and transmit the generated control signal to the controller 120.

[0049] The processor 130 may mean a device capable of performing a series of operations or judgments for controlling the vehicle 100 and automatically parallel parking. For example, the processor 130 may be a part of the automatic parallel parking device 13 in which a program including instructions for performing automatic parallel parking is executed.

[0050] The processor 130 is part of the automatic valet parking device 13 and can control the automatic driving so that the vehicle 100 can park in the target position in the parking lot through cooperation with the infrastructure 11. The processor 130 provides instructions to the controller 120 to enable the controller 120 to perform automatic driving.

[0051] Based on the guidance route information received from the infrastructure 11, the processor 130 can control the vehicle 100 to travel along the guidance route.

[0052] In the above-mentioned FIG. 2, the controller 120 and the processor 130 were described as separate components, but the controller 120 and the processor 130 can be integrated into one component.

[0053] FIG. 3 is a conceptual diagram for explaining the automatic valet parking system and its method according to the present invention.

[0054] As shown in FIG. 3, in (1), the driver can drive the vehicle 100 into the parking lot and drive the vehicle 100 to the getting-off area.

[0055] (2), the driver parks the vehicle in the getting-off area and can transfer the driving authority of the vehicle to the infrastructure 11 when getting out of the vehicle.

[0056] (3), the infrastructure 11 or the automatic valet parking server included in the infrastructure 11 searches for an empty parking space from a plurality of parking spaces existing in the parking lot and determines a parking space (target position) suitable for the vehicle to park. Also, the infrastructure 11 determines a guidance route to the determined target position. When the target position and the guidance route are determined, the vehicle 100 travels along the guidance route under the control of the infrastructure 11 or automatically, and after reaching around the target position, automatic valet parking to the target position can be performed.

[0057] In (4), the driver can decide to drive his / her vehicle 100 and move to the boarding area.

[0058] In (5), the infrastructure 11 or the automatic valet parking server included in the infrastructure 11 can determine a suitable target position. For example, the suitable target position can be an empty parking space among a plurality of parking spaces existing in the boarding area. Also, the infrastructure 11 or the automatic valet parking server can determine a guidance route to the determined target position. When the target position and the guidance route are determined, the vehicle 100 can travel along the guidance route under the control of the infrastructure 11 or automatically, and after reaching around the target position, perform automatic valet parking and park at the target position.

[0059] In (6), the driver can reach the boarding area, obtain the driving authority for the vehicle 100 from the infrastructure 11, and drive the vehicle 100 towards the exit of the parking lot.

[0060] As shown in FIG. 3, for the vehicle 100 to travel along the guidance route automatically or under the control of the infrastructure 11, the vehicle 100 or the infrastructure 11 must have a map of the parking lot, grasp the position of the vehicle 100 within the map, and recognize whether the vehicle is traveling accurately along the guidance route. For this purpose, the vehicle 100 needs to use a sensor such as a camera to photograph the surroundings and recognize its own position based on the photographed video.

[0061] Also, markers can be used. A marker is something that assists in confirming the position of the vehicle 100 and can be a physical display that identifies a unique ID (Identification) installed in the parking lot that the vehicle 100 can sense.

[0062] FIG. 4 is a drawing showing an example of installing markers in a parking lot. Also, FIG. 5 is a drawing showing examples of unique markers and anonymous markers.

[0063] As shown in FIG. 4, markers 410 and 411 including unique IDs are installed at specific positions in the parking lot where vehicle 100 can be sensed. Vehicle 100 recognizes the marker, extracts the unique ID, thereby grasps the position corresponding to the unique ID from the digital map, and can grasp the exact position within the parking lot.

[0064] As shown in FIGS. 4 and 5, the marker is square and can be a black-and-white quadrilateral. For example, the marker can display various forms as white rectangles on the surface of a black square, and can give a unique ID.

[0065] Here, the markers can be of two types: unique markers and anonymous markers.

[0066] A unique marker can be a physical display that can identify one of the unique IDs from 1 to 250. Here, only one unique ID from 1 to 250 can exist within the parking lot. Therefore, vehicle 100 that identifies the unique marker can accurately grasp the position within the digital map corresponding to the unique ID. Also, the unique marker can necessarily be installed at a position defined by laws and standards, for example, the entrance where one enters the parking lot from the drop-off area or the exit where one exits the parking lot to the pick-up area. In this case, the unique ID displayed on the unique marker can also be preset.

[0067] The size of the unique marker can also be determined by regulations, and the display form for displaying the unique ID on the unique marker can also be determined in advance.

[0068] For the unique marker, a marker with a specific shape or pattern drawn at a specific position (e.g., the entrance and exit) according to standards or regulations must be used. Therefore, it is difficult to install freely, and as shown in FIG. 4, installing a large number of unique markers may also damage the aesthetics of the parking lot.

[0069] Anonymous markers may not be subject to the aforementioned constraints of unique markers. An anonymous marker can have an ID from 251 to 255, and the complexity for displaying the ID may also differ compared to unique markers. The display for giving an ID to an anonymous marker may be simpler than the display for giving an ID to a unique marker. Referring to Figure 5, it can be seen that the pattern for expressing the ID for the unique marker 511 is more complex than that for the anonymous marker 521.

[0070] Also, anonymous markers can be installed at any position and as needed. That is, different from unique markers, anonymous markers can be arbitrarily installed and utilized considering features such as the size and structure of each parking lot. For example, in a large-scale parking lot, in order to correct the position error of a vehicle, multiple anonymous markers can be used between unique markers. That is, although the current exact position of the vehicle can be grasped using the first unique marker, it may be difficult to track the position of the vehicle with an absolute accuracy of 50 cm while traveling from the position of the first unique marker to the position of the second unique marker. In such a case, the accuracy with respect to the vehicle position can be improved by using multiple anonymous markers. Anonymous markers can be used as a function to assist unique markers in grasping the precise position of a vehicle. A vehicle can recognize a unique marker and set the position information of the vehicle at the position corresponding to the unique marker. Also, a vehicle can recognize an anonymous marker and correct the position information of the vehicle using the position information corresponding to the anonymous marker.

[0071] Accordingly, when the vehicle recognizes a unique marker, it can reset the current position of the vehicle to the position corresponding to the unique marker and further reset the route to the destination (e.g., parking space). If no route reset is necessary, the vehicle can continue to travel while maintaining the existing route. When the vehicle recognizes an anonymous marker, it can reset (correct) the current position of the vehicle to the position corresponding to the anonymous marker, but there may be cases where such a route reset is not possible. If it is recognized that the current position of the vehicle recognized through the anonymous marker is significantly deviated from the existing route, the vehicle can handle the information recognized through the anonymous marker as an error and not perform a route reset.

[0072] However, as shown in FIG. 5, the unique marker 511 and the anonymous marker 521 must be coded to include ID information, such as a QR code (registered trademark), and must be installed at specific positions. That is, they must be manufactured and installed so that their unique functions can be performed.

[0073] Based on this, in the present invention, a method is proposed in which signs pre-installed in a parking lot, such as display boards, floor characters or symbols, guide boards, advertisement boards, etc., which must be obligatorily installed in the parking lot, can be used as markers.

[0074] FIG. 6 is a drawing showing examples of display boards, floor characters or symbols, guide boards, and advertisement boards commonly seen in a parking lot.

[0075] As shown in FIG. 6, various signs are installed in the parking lot. Also, each sign includes different symbols or characters and can be sufficiently distinguished. Thus, the signs already installed in the parking lot can be acquired and used as anonymous markers for distinguishing and specifying positions. At this time, an ID for the anonymous marker can be assigned to the sign set for specifying the position.

[0076] Thus, when replacing the signs already installed in the parking lot with anonymous markers, it is necessary to determine in advance the features for distinguishing the first sign replacing the anonymous marker from the second sign, and input the position of the first sign in the digital map of the parking lot and enter it in the digital map.

[0077] FIG. 7 is a drawing showing a system for generating a digital map including a unique sign in a parking lot as an anonymous marker. Further, FIG. 8 is a drawing showing a method for generating a digital map including a unique sign in a parking lot as an anonymous marker.

[0078] As shown in FIGS. 7 and 8, the sign acquisition device 710 can include a video acquisition unit 711, a positioning information acquisition unit 713, and a communication unit 715. Further, the digital map generation device 720 can include a communication unit 721 and a processor 723.

[0079] The sign acquisition device 710 can be a vehicle or a device that is temporarily equipped on a vehicle and uses a sensor attached to the vehicle. Further, the sign acquisition device 710 can be an autonomous mobile robot, or a smart device such as a mobile phone that is moved by an administrator.

[0080] The digital map generation device 720 can be an automatic valet parking server, but can be a device different from the automatic valet parking server. Further, the digital map generation device 720 can be a computer program that can be executed on an automatic valet parking server or other general computing device.

[0081] As shown in FIGS. 7 and 8, in operation S801, the sign acquisition device 710 uses the video acquisition unit 711 to acquire video within the parking lot. At the same time, the positioning information acquisition unit 713 can acquire positioning information indicating the position information acquired from each video. The video acquisition unit 711 can be a camera provided in a vehicle or a mobile phone. Further, the positioning information acquisition unit 713 can be a GPS system, or can be a separate positioning device that can accurately measure the position of the sign using a facility provided for precise positioning.

[0082] The identification acquisition device 710 can transmit the acquired video information and positioning information to the digital map generation device 720 via the communication unit 715.

[0083] In operation S803, the digital map generation device 720 can detect a label from the video information received via the communication unit 721 and extract the features of each label. Further, the digital map generation device 720 can recognize a display board, a guide board, characters, or symbols shown in part or all of the video and detect the label.

[0084] The label can be a display board, symbolic information such as a disabled symbol displayed on the floor surface, an identification number of a parking space displayed on a wall or a pillar, or guiding text in various forms and contents.

[0085] After detecting the label, the digital map generation device 720 can extract the features related to each label. Here, the features related to each label can include, but are not necessarily limited to, the type of the label, the components described in the label, the colors included in the label, or the relative position information between the components of the label. And the components can include Hangul characters, English characters, or symbols.

[0086] The digital map generation device 720 can extract at least one of the features related to the information and position of English characters, the information and position of Hangul characters, etc., and the information and position of symbols included in each label.

[0087] Taking the display board 610 in FIG. 6 as an example, the digital map generation device 720 can detect the display board 610 from the video, and extract the information and position of the handle characters, as well as the information and position of the symbols, from the detected display board 610. The digital map generation device 720 can extract the feature that the handle characters "Elderly Priority Parking Area" are described at the upper end of the display board 610. In addition, the digital map generation device 720 can extract the feature that the symbols "P" and the symbol representing the elderly are arranged side by side in a large size in the middle of the display board 610.

[0088] Taking the identification number 620 of the parking surface in FIG. 6 as an example, the digital map generation device 720 can detect the identification number 620 of the parking surface from the video, and obtain the information and position of the English characters from the detected identification number 620 of the parking surface. The digital map generation device 720 can extract the identification numbers of the parking surfaces "C3" and "MF" from the video, and extract the information that "C3" and "MF" are arranged vertically. In addition, the feature that "C3" is displayed in a larger size than "MF" can also be extracted.

[0089] The detection of the above-described signs and the extraction of the features of each sign can be performed based on artificial intelligence.

[0090] Here, as shown in FIGS. 7 and 8 again, in operation S805, based on the positioning information of each video obtained by the digital map generation device 720 from the sign acquisition device 710 via the communication unit 721, the digital map generation device 720 can associate the position of the sign detected from each video in the digital map with the features of the sign and include them in the digital map. The digital map generation device 720 can display the symbol of the sign detected at the position corresponding to the positioning information in the digital map. In addition, the digital map generation device 720 can include in the digital map the information obtained by matching the position corresponding to the positioning information in the digital map with the features of the detected sign.

[0091] In a digital map, positional information based on markers can additionally be used. Existing parking lots already have unique markers or anonymous markers for notifying positional information, and the digital map can store by associating the markers with the positional information. Therefore, a vehicle can acquire the positional information of the vehicle based on the markers. However, due to constraints when installing markers, etc., it may not be possible to install markers at all necessary locations within the parking lot. Or, it may be difficult to grasp the precise positions required by standards and regulations with only the current markers. In such a case, signs at the necessary positions can be used to replace the markers or enhance the accuracy of the positional information.

[0092] As described above, marker information can be included in the digital map. For an automatic valet parking system, the digital map should include structural information of the actual static parking lot for positioning and navigation, etc. Also, additional information for performing various missions such as vehicle entry, exit, and rearrangement must be included.

[0093] Thus, in order for the digital map to include various information, the present invention proposes a digital map composed of five layers as shown in FIG. 9.

[0094] The first layer 910 of the digital map can include physical three-dimensional space information. And it includes information for examining whether it is possible to provide an automatic valet parking service for various vehicles. For example, in the case of a multi-story parking lot, the height information of each floor can be included in the first layer 910. And based on that height, information for determining whether a vehicle can be parked in the parking lot can be provided. Also, it includes information regarding the size of the parking spaces in the parking lot and can provide information for determining whether a vehicle can be parked in a specific parking space.

[0095] The second layer 920 of the digital map can contain static information. And it can define the use of each space in the parking lot. For example, each area such as a drop-off area, a pick-up area, or a parking area can be set, and the use of each area can be defined. After the marker is installed, or when a sign is set to be used as a marker, it can include information about the marker or the sign. For example, in the case of a marker, the represented ID and the position information of each marker can be included in the second layer 920. In the case of a sign, the features that can distinguish the sign from other signs and the position information of the sign can be included in the second layer 920.

[0096] The third layer 930 of the digital map can contain temporarily static information. For example, it can include information indicating whether a vehicle is parked in a specific parking space. This can be used as information for checking available parking spaces.

[0097] The fourth layer 940 of the digital map can contain dynamic information. For example, it can include the maximum speed information of vehicles performing entry, exit, or rearrangement, or information displaying real-time movement. In the case of a parking lot where people can enter and exit, information about people's movement can also be included in the fourth layer 940.

[0098] The fifth layer 950 of the digital map can contain unknown dynamic information. For example, in a parking lot designed to prevent people from entering or exiting, if there is movement of people, electric scooters, etc., the movement can be included in the fifth layer 950 as unknown dynamic information. Also, it can include objects that are not already installed in the parking lot (such as boxes or wood fallen from a vehicle), but are temporarily detected for other reasons.

[0099] Unlike the conventional maps that have been used simply to provide a guiding route for autonomous vehicles, the digital map used in a parking lot can include all the information within the parking lot. Such a digital map can be actively updated and used by an automated valet parking server.

[0100] FIG. 10 is a drawing showing an example of recognizing vehicle position information by using a unique marker, an anonymous marker, and a sign together.

[0101] As shown in FIG. 10, there may be a plurality of unique markers indicated by (1)(2)(3)(4)(5) within the parking lot. Then, the sign acquisition device 710 and the digital map generation device 720 can detect signs A, B, and C within the parking lot and store the sign information in the digital map in association with the position information.

[0102] When adding a sign to the digital map, the digital map generation device 720 may not add the sign to the digital map if there is a unique marker or an anonymous marker nearby or if another sign is already registered. For example, as shown in FIG. 10, when sign A and marker (4) are located close to each other, sign A may not be necessary for obtaining the vehicle position information. Therefore, the digital map generation device 720 may not include the position information regarding sign A in the digital map.

[0103] Similarly, the digital map generation device 720 may have previously obtained the position information regarding sign B and included it in the digital map. In this case, since sign C is located close to sign B, it may not be necessary for obtaining the vehicle position information. Therefore, the digital map generation device 620 may not include the position information regarding sign C in the digital map.

[0104] As shown in FIG. 10, the vehicle can be parked in the drop-off area 1010 by the driver. Then, it can be automatically driven to the target position P2 by the infrastructure 11 or the automated valet parking server.

[0105] When the vehicle is automatically driven under the control of an automatic valet parking server, the automatic valet parking server can provide a guidance route to the vehicle and instruct it to drive automatically by itself, or provide a series of control instructions to the vehicle based on the guidance route.

[0106] In order to determine whether the vehicle is traveling along the guidance route, the automatic valet parking server can detect markers and signs at the positions where the vehicle passes and obtain position information. For example, when the automatic valet parking server controls the automatic driving to park the vehicle, if marker (1) is detected, it can be known that the vehicle has left the getting-off area and is performing automatic driving. Also, when marker (2) and / or marker (3) are detected, it can be known that the vehicle is traveling along the guidance route. Similarly, when sign B, marker (4), and marker (5) are detected, it can be known that the vehicle has accurately traveled to the target position P2 along the guidance route.

[0107] Also, the sign information can be updated in real time.

[0108] FIG. 11 is a drawing showing an example where the sign is changed.

[0109] Comparing FIGS. 10 and 11, it can be seen that signs A, B, and C are deleted and sign D is added. The autonomous vehicle can acquire images while driving along the guidance route for parking and transmit them to the automatic valet parking server. The server or the digital map generating device 720 provided for automatic valet parking can detect that the existing signs A, B, and C disappear and a new sign D appears. The digital map generating device 720 can recognize that signs A, B, or C are not detected in the images captured by the autonomous vehicle and determine that the signs are removed. Also, when the sign is not detected in the images acquired from the vehicle, the digital map generating device 720 can determine that the sign is removed. Thus, when it is determined that the sign used as a marker for detecting the position is removed, the digital map generating device 720 can delete the sign information and the position information included in the existing digital map.

[0110] Conversely, the digital map generating device 720 recognizes that sign D is newly detected in the images captured by the autonomous vehicle and determines whether to use the sign as a marker for position detection. If it is determined to use it as a marker for position detection, the characteristics and position information of the sign can be associated and stored in the digital map.

[0111] Based on the image information acquired from the vehicle driving for parking, the digital map generating device 720 can update the markers and sign information for position detection included in the existing digital map.

[0112] This also applies to the markers in the same way. When the digital map generating device 720 does not detect the unique markers, anonymous markers, or signs for position detection included in the existing digital map based on the image information acquired from the vehicle driving for parking, it can sound an alarm or immediately update the digital map about the disappearance of the markers or signs.

[0113] As described above, in the present disclosure, markers used to grasp the position information of vehicles in a parking lot are distinguished into unique markers and anonymous markers, and the uses of the unique markers and the anonymous markers are defined. Further, a solution is presented to utilize various signs that are already installed or displayed as anonymous markers instead of markers that must be separately manufactured and installed.

[0114] In addition, when markers are already installed in a parking lot, various signs in the parking lot can be utilized, and additionally, anonymous markers can be provided to further improve the accuracy of vehicle position measurement.

[0115] Also, the functions described above can be implemented by hardware, software, firmware, or any combination thereof. If implemented by software, these functions can be stored and transmitted as one or more instructions or codes on a computer-readable medium. Here, a computer-readable medium can include all of a communication medium and a computer storage medium, including any medium that facilitates the transmission of a computer program from one location to another. Also, a storage medium can be any available medium that can be accessed by a computer. By way of non-limiting example, such a computer-readable medium can be RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, which can be used to transmit or store the desired program code in the form of instructions or data structures and can include any other medium that can be accessed by a computer.

[0116] When implementing the present invention with program code or code segments, it must be recognized that the code segments can be any combination of things such as procedures, functions, subroutines, programs, routines, subroutines, modules, software packages, classes, instructions, data structures, or program instruction statements. The code segments can be connected to other code segments or hardware circuits by sending and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. can be transmitted, sent, or transmitted using any suitable means including sharing of memory, message transmission, token transmission, network transmission, etc. Further, in one aspect, the steps and / or operations of a method or algorithm can be integrated into a computer program and can reside as one or any combination or set of code and / or instructions on a machine-readable medium and / or a computer-readable medium.

[0117] In the implementation with software, the above-described technology can be implemented by modules (e.g., procedures, functions, etc.) that perform the functions described herein. The software code can be stored in a memory unit and executed by a processor. Also, the memory unit can be implemented inside or outside the processor. In this case, the memory unit can be communicably connected to the processor by various means as is known.

[0118] In the implementation with hardware, the processing unit can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, or other electronic units designed to perform the functions described herein, or a combination thereof.

[0119] The foregoing includes examples of one or more embodiments. Of course, when describing the foregoing embodiments, it is not possible to describe every possible combination of components or methods. Rather, one of ordinary skill in the art will fully recognize that many additional combinations or substitutions are possible for various embodiments. Accordingly, the foregoing embodiments are to be understood as including all alternatives, modifications, and variations within the spirit or scope of the claims of the present invention.

[0120] As used herein, the term "infer" or "inference" generally refers to a process of making a determination, or inference, about the state of a system, environment, and / or user from a set of observations captured by events and / or data. An inference can be utilized to identify a particular situation or action, for example, by generating a probability distribution over states. An inference can be probabilistic, i.e., a calculation of a probability distribution over states based on consideration of data and events. An inference can also be a technique utilized to construct higher-level events from a set of events and / or data. Such inferences can cause a new event or action to be inferred from a set of observed and / or stored event data, whether the events are temporally closely correlated, and whether the events and data are derived from one or more event and data sources.

[0121] Furthermore, although the terms "component", "module", "system", etc. used in the present invention are not limited to the following, they include computer-related entities such as hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process executed by a processor, a processor, an object, an executable thread, a program, and / or a computer. For example, both an application running on a computing device and the computing device can be components. One or more components can reside within a process and / or an execution thread. Also, a component can be concentrated on one computer or dispersed among two or more computers. These components can also be executed from various computer-readable media storing various data structures. A component can communicate by following a signal having one or more data packets (e.g., data from other components in a local system, a distributed system, and / or components interacting with other systems via a network such as the Internet by signals).

Description of Reference Numerals

[0122] 10 Barrier Parking System 11 Infrastructure 13 Automatic Barrier Parking Device 50 Automatic Barrier Parking Server 51 Processor 53 Storage Unit 55 Communication Device 100 Vehicle 110 Sensor 120 Controller 130 Processor 140 Display 150 Communication Device 410, 411 Marker 511 Unique Marker 521 Anonymous Marker 610 Display Panel 710 Sign Acquisition Device 711 Image Acquisition Unit 713 Positioning Information Acquisition Unit 715 Communication Unit 720 Generation Device 721 Communication Unit 723 Processor 910 First Layer 920 Second Layer 930 Third Layer 940 Fourth Layer 950 Fifth Layer 1010 Alighting Area

Claims

1. A vehicle, comprising: a sensor that acquires image data around the vehicle using at least one sensor; a controller that controls the operation of at least one function of the vehicle; a communication device that communicates with an automatic valet parking server; and a processor that controls the operation of the vehicle, wherein the processor: recognizes a unique marker installed in a parking lot from the image data and sets the position information of the vehicle; recognizes an anonymous marker installed in the parking lot from the image data and corrects the position information of the vehicle.

2. The processor: acquires a digital map including information on unique markers and anonymous markers from the automatic valet parking server; when recognizing the unique marker, sets the position information of the vehicle at the position corresponding to the unique marker included in the digital map; when recognizing the anonymous marker, corrects the position information of the vehicle based on the position corresponding to the anonymous marker included in the digital map.

3. The processor: when recognizing the unique marker, re-sets the route to the destination of the vehicle.

4. The digital map is composed of five layers, namely, a first layer including physical three-dimensional space information, a second layer including static information, a third layer including temporary static information, a fourth layer including dynamic information, and a fifth layer including unconfirmed dynamic information; the unique marker and the anonymous marker are included in the second layer.

5. The processor: recognizes a sign used as one of the anonymous markers provided in the parking lot from the image data and corrects the position information of the vehicle.

6. The sign is a sign pre-installed in the parking lot, including a display board, characters or symbols on the floor, a guide board, and an advertisement board that are required to be installed in the parking lot, and is pre-set to be used as the anonymous marker.

7. The identifier is used as one of the anonymous markers by including in the digital map by associating the features extracted from the identifier with the position information of the identifier. The processor detects a first identifier from the image data and extracts features of the first identifier, compares the features of the first identifier with the features of the identifiers included in the digital map to obtain an identifier that matches the first identifier, and corrects the position information of the vehicle based on the position information corresponding to the obtained identifier. The vehicle according to claim 5, characterized in that.

8. The features of the identifier include at least one of the type of the identifier, the components described in the identifier, the colors included in the identifier, and the relative position information between the components of the identifier. The vehicle according to claim 7, characterized in that.

9. The components include handle characters, English characters, and symbols, The processor extracts features including information on English characters and relative position information of the English characters, information on Hangul characters and relative position information of the Hangul characters, information on symbols and relative position information of the symbols from the first identifier, and compares them with the features of the identifiers included in the digital map. The vehicle according to claim 8, characterized in that.

10. The unique marker and the anonymous marker are composed of black and white, and are in a form in which at least one white square is arranged on a black square ground, The complexity of the form in which at least one white square is arranged in the unique marker is higher than the complexity of the form in which at least one white square is arranged in the anonymous marker. The vehicle according to claim 1, characterized in that.

11. An automatic valet parking server, A storage unit storing digital map information, A communication device that communicates with the vehicle, and Including a processor, The processor During the execution of automatic valet parking for the vehicle, recognizes a unique marker installed in the parking lot from the image data acquired by the vehicle, sets the position information of the vehicle, and recognizes an anonymous marker installed in the parking lot from the image data to correct the position information of the vehicle. An automatic valet parking server characterized by that.

12. The processor When the unique marker is recognized, set the position information of the vehicle at the position corresponding to the unique marker included in the digital map. The automatic valet parking server according to claim 11, wherein when the anonymous marker is recognized, the position information of the vehicle is corrected based on the position corresponding to the anonymous marker included in the digital map. **Claim 13** The processor The automatic valet parking server according to claim 12, wherein when the unique marker is recognized, the route to the destination of the vehicle is reset. **Claim 14** The digital map is composed of five layers: a first layer including physical three-dimensional space information, a second layer including static information, a third layer including temporary static information, a fourth layer including dynamic information, and a fifth layer including unconfirmed dynamic information. The automatic valet parking server according to claim 12, wherein the unique marker and the anonymous marker are included in the second layer. **Claim 15** The processor The automatic valet parking server according to claim 12, wherein the processor recognizes a sign used as one of the anonymous markers provided in the parking lot from the image data and corrects the position information of the vehicle. **Claim 16** The sign is a sign pre-installed in a parking lot including a display board, characters or symbols on the floor, a guide board, and an advertisement board that are required to be installed in the parking lot, and is preset so that it can be used as the anonymous marker. The automatic valet parking server according to claim 15, characterized in that. **Claim 17** The sign is used as one of the anonymous markers by associating the features extracted from the sign with the position information of the sign and including them in the digital map. The processor Detect a first sign from the image data and extract the features of the first sign. The automatic valet parking server according to claim 15, wherein the features of the first sign are compared with the features of the signs included in the digital map to obtain a sign that matches the first sign, and the position information of the vehicle is corrected based on the position information corresponding to the obtained sign. **Claim 18** The processor The automatic valet parking server according to claim 17, wherein the feature of the identifier includes at least one of the type of the identifier, the components described in the identifier, the colors included in the identifier, and the relative position information between the components of the identifier.

19. The components include handle characters, English characters, and symbols. The automatic valet parking server according to claim 18, wherein the processor extracts features including information on English characters and relative position information of the English characters, information on Hangul characters and relative position information of the Hangul characters, information on symbols and relative position information of the symbols from the first identifier, and compares them with the features of the identifiers included in the digital map.

20. The unique marker and the anonymous marker are composed of black and white, and are in a form in which at least one white square is arranged on a black square ground. The automatic valet parking server according to claim 11, wherein the complexity of the form in which at least one white square is arranged in the unique marker is higher than the complexity of the form in which at least one white square is arranged in the anonymous marker.

21. An automatic valet parking method for a vehicle, comprising: an operation of acquiring surrounding image data using at least one sensor; an operation of recognizing a unique marker installed in a parking lot from the image data and setting position information of the vehicle; and an operation of recognizing an anonymous marker installed in the parking lot from the image data and correcting the position information of the vehicle.

22. further including an operation of acquiring a digital map including unique marker and anonymous marker information from an automatic valet parking server, The operation of setting the position information of the vehicle includes: an operation of setting the position information of the vehicle at a position corresponding to the unique marker included in the digital map, The operation of correcting the position information of the vehicle includes: an operation of correcting the position information of the vehicle based on a position corresponding to the anonymous marker included in the digital map. The automatic valet parking method according to claim 21.

23. The operation of setting the position information of the vehicle includes: The automatic valet parking method according to claim 22, further comprising an operation of re-setting a route to a destination of the vehicle.

24. The operation of correcting the position information of the vehicle includes an operation of recognizing a sign used as one of the anonymous markers provided in the parking lot from the image data and correcting the position information of the vehicle. The automatic valet parking method according to claim 23.

25. The sign is a sign pre-installed in a parking lot including a display board, a character or symbol on the floor, a guide board, and an advertisement board that are obligatorily installed in the parking lot, and is pre-set so as to be usable as the anonymous marker. The automatic valet parking method according to claim 24.

Images