Image-based object detection method for electric vehicle charging, charging robot, and controller of charging robot

Abstract

An image-based object detection method for electric vehicle charging is discharged. The method comprises the steps of: detecting a first object corresponding to an electric vehicle from an image obtained by a camera outside of an electric vehicle; estimating the position and angle of the electric vehicle in a parking space on the basis of position information of the first object in the image; detecting, from the image, at least one second object that excludes the electric vehicle; and estimating the position of the at least one second object in the parking space on the basis of the position of the at least one second object in the image.

Description

Image-based object detection method for charging electric vehicles, charging robot, and controller of the charging robot

[0001] The present invention relates to an electric vehicle power supply equipment (EVSE) including a charging manipulator for charging an electric vehicle, and more particularly, to an image-based object detection method for safe charging of an electric vehicle (IMAGE-BASED OBJECT DETECTION METHOD FOR ELECTRIC VEHICLE CHARGING, CHARGING ROBOT, AND CONTROLLER OF THE CHARGING ROBOT).

[0002] Electric vehicles (EVs) currently under development use battery power to drive a motor, and thus have the advantages of producing fewer air pollutants such as exhaust gases and noise, being less prone to breakdowns, having a longer lifespan, and being easier to drive than conventional gasoline engine vehicles.

[0003] Electric vehicles are categorized by their propulsion system into hybrid electric vehicles (HEVs), plug-in hybrid electric vehicles (PHEVs), and electric vehicles (EVs). HEVs have an engine as their primary power source and a motor as an auxiliary power source. PHEVs have a motor as their primary power source and an engine that powers the vehicle when the battery is discharged. EVs have a motor but no engine.

[0004] An electric vehicle charging system can be fundamentally defined as a system that charges the batteries mounted on an electric vehicle using power from the commercial power grid or energy storage devices. These systems can take various forms depending on the type of electric vehicle. For example, an electric vehicle charging system may include a conductive charging system using cables or a contactless wireless power transfer system.

[0005] When charging an electric vehicle, a vehicle assembly (VA) mounted on the electric vehicle forms an inductive resonant coupling with a transmitting pad of a ground assembly (GA) located at a charging station or charging spot, and the battery of the electric vehicle can be charged using power transmitted from the ground assembly through the inductive resonant coupling.

[0006] When charging an electric vehicle, a robotic arm or manipulator may be used to supply power from an electric vehicle power supply (EVSE) to a charging door / port.

[0007] At this time, considering the various types of electric vehicle charging ports, various types of electric vehicle power supply devices, and various charging methods, it is necessary to define procedures for the positioning between the electric vehicle and the manipulator and the preparatory steps for power supply.

[0008] The purpose of the present invention to solve the above problems is to define and propose a procedure for a preparatory step that can accurately recognize the situation around an EVSE or a charging robot within a charging station in order to ensure safety when supplying power from a power supply device or a power supply equipment (EVSE, Electric Vehicle Supply Equipment) to an electric vehicle.

[0009] Another object of the present invention is to define and propose a procedure for a preparatory step capable of detecting a parking condition unsuitable for charging or a dangerous situation that may arise due to a person approaching during a charging operation, based on an image of the surroundings of a parking space in a charging station, and ensuring safety.

[0010] Another object of the present invention is to propose a linkage system between a camera around a parking space in a charging station and a charging robot capable of obtaining an image of the exterior of an electric vehicle.

[0011] In order to achieve the above object, according to one embodiment of the present invention, an image-based object detection method for charging an electric vehicle is a method executed by a computing system or controller on the side of a charging station that is linked to an electric vehicle power supply (EVSE) including a charging manipulator, the method including: detecting a first object corresponding to an electric vehicle from an image obtained by a camera capable of obtaining an image of an electric vehicle from the outside of the electric vehicle; estimating a position and angle of the electric vehicle on a parking space based on position information of the first object in the image; detecting at least one second object other than the electric vehicle from the image; and estimating a position of at least one second object on the parking space based on the position of the at least one second object in the image.

[0012] In the step of estimating the position and angle of the electric vehicle, the position and angle of the electric vehicle can be estimated on a two-dimensional coordinate system using the position on the boundary line of the parking space as a reference point.

[0013] The step of estimating the position and angle of the electric vehicle may include the step of mapping the first object on a two-dimensional map corresponding to the parking space based on position information of the first object in the image; and the step of estimating the position and angle of the first object on the two-dimensional map as the position and angle of the electric vehicle.

[0014] In an image-based object detection method for charging an electric vehicle according to one embodiment of the present invention, the step of detecting a first object corresponding to the electric vehicle may include a segmentation step of dividing an outline of the first object, and the position and angle of the first object within the image may be obtained based on the segmentation result of the first object.

[0015] An image-based object detection method for charging an electric vehicle according to one embodiment of the present invention may further include a step of detecting a third object corresponding to a license plate of the electric vehicle from an image, wherein the step of estimating a position and angle of the electric vehicle may include a step of assisting in extracting position information of a first object within the image based on the position and angle of the third object within the image.

[0016] An image-based object detection method for charging an electric vehicle according to one embodiment of the present invention may further include a step of detecting a third object corresponding to a license plate of the electric vehicle from an image, wherein the step of estimating a position and angle of the electric vehicle may include a step of estimating an angle of the electric vehicle in a parking space based on an angle of the third object in the image.

[0017] An image-based object detection method for charging an electric vehicle according to one embodiment of the present invention may further include a step of identifying a license plate of an electric vehicle from an image; and a step of extracting vehicle information of the electric vehicle from a database using the license plate of the electric vehicle. In this case, in the step of estimating the position and angle of the electric vehicle, the vehicle information of the electric vehicle may be used to estimate the position and angle of the electric vehicle in a parking space.

[0018] An image-based object detection method for charging an electric vehicle according to one embodiment of the present invention may further include a step of estimating the position of a charging port of an electric vehicle based on the position and angle of the electric vehicle in a parking space; and a step of determining whether the position of the charging port of the electric vehicle is within the operating range of a charging robot.

[0019] An image-based object detection method for charging an electric vehicle according to one embodiment of the present invention may further include a step of guiding the electric vehicle to be re-parked if the location of the charging port of the electric vehicle is not within the operating range of the charging robot; and a step of guiding the charging robot to approach the electric vehicle if the location of the charging port of the electric vehicle is within the operating range of the charging robot.

[0020] In an image-based object detection method for charging an electric vehicle according to one embodiment of the present invention, at least one second object may be a person or an animal.

[0021] An image-based object detection method for charging an electric vehicle according to one embodiment of the present invention may further include a step of determining whether any one or more second objects is located within the operating range of a charging robot; and a step of executing a safety-related action if any one or more second objects is located within the operating range of the charging robot.

[0022] A charging robot and / or a controller of the charging robot, which includes a charging manipulator according to one embodiment of the present invention and is associated with an electric vehicle power supply (EVSE) that transmits power to an electric vehicle, may include a memory that stores at least one command; a processor that controls the movement of the charging robot or the charging manipulator by loading and executing at least one command from the memory; and a communication interface that transmits and receives information necessary for the process of transmitting power to the electric vehicle to and from the EVSE.

[0023] In a charging robot and / or a controller of the charging robot according to one embodiment of the present invention, the processor can obtain a position and angle of the electric vehicle on a parking space, estimated from an image obtained by a camera capable of obtaining an image of the electric vehicle from outside the electric vehicle, can obtain a position of at least one object other than the electric vehicle on the parking space, detected from the image, and can determine whether charging of the electric vehicle is possible based on the position and angle of the electric vehicle on the parking space and the position of the at least one object on the parking space.

[0024] In a charging robot and / or a controller of a charging robot according to one embodiment of the present invention, the position and angle of the electric vehicle on the parking space, and the position of at least one object on the parking space, can be expressed as the position and angle on a two-dimensional coordinate system with the position on the boundary line of the parking space as a reference point.

[0025] In a charging robot and / or a controller of a charging robot according to one embodiment of the present invention, the processor may determine that charging of the electric vehicle is not possible if the location of the charging port of the electric vehicle, estimated based on the location and angle of the electric vehicle in the parking space, is not within the operating range of the charging robot.

[0026] In a charging robot and / or a controller of a charging robot according to one embodiment of the present invention, the processor may determine that charging of an electric vehicle is not possible if the position of at least one object in a parking space is within the operating range of the charging robot.

[0027] In a charging robot and / or a controller of a charging robot according to one embodiment of the present invention, the processor can control the movement of the charging robot and the charging manipulator to move the charging robot to the electric vehicle and connect the charging manipulator to the charging port of the electric vehicle when charging of the electric vehicle is possible.

[0028] According to one embodiment of the present invention, in order to ensure safety when supplying power from a power supply device or electric vehicle supply equipment (EVSE) to an electric vehicle, a procedure for a preparatory step capable of accurately recognizing the situation around an EVSE or a charging robot within a charging station can be provided.

[0029] According to one embodiment of the present invention, a procedure for a preparatory step for detecting a parking condition unsuitable for charging or a dangerous situation that may arise due to a person approaching during a charging operation, based on an image of the surroundings of a parking space in a charging station, and ensuring safety can be provided.

[0030] According to one embodiment of the present invention, a linkage system between a camera around a parking space in a charging station and a charging robot capable of obtaining an image of the exterior of an electric vehicle can be implemented.

[0031] According to one embodiment of the present invention, the cost required to ensure safety and efficiency of the charging process can be reduced by recognizing the situation using images from a general camera without the need to mount additional sensors on the charging robot.

[0032] FIG. 1 is a drawing showing an example of an image-based object detection process for charging an electric vehicle to which one embodiment of the present invention is applied, in which the charging port of the electric vehicle is parked so that it is out of the operating range of a charging robot.

[0033] FIG. 2 is a drawing illustrating cameras placed in a charging station and a parking space as an example of an image-based object detection process for charging an electric vehicle to which one embodiment of the present invention is applied.

[0034] FIG. 3 is a diagram conceptually illustrating a process of obtaining an image by a camera placed in a parking space as an example of an image-based object detection process for charging an electric vehicle to which one embodiment of the present invention is applied.

[0035] FIGS. 4 and 5 are conceptual drawings illustrating examples of a two-dimensional reference coordinate system of a parking space to which one embodiment of the present invention is applied.

[0036] FIG. 6 is a flowchart illustrating an image-based object detection process for charging an electric vehicle according to one embodiment of the present invention.

[0037] FIG. 7 is a flowchart illustrating an image-based object detection process for charging an electric vehicle according to another embodiment of the present invention.

[0038] FIG. 8 is a diagram conceptually illustrating a process of estimating the position and angle of an electric vehicle on a two-dimensional reference coordinate system of a parking space, as an example of an image-based object detection process for charging an electric vehicle to which one embodiment of the present invention is applied.

[0039] FIG. 9 is a flowchart conceptually illustrating an embodiment in which an image-based object detection process for charging an electric vehicle and a charging robot control process are linked according to one embodiment of the present invention.

[0040] FIG. 10 is a flowchart conceptually illustrating a charging robot control process linked to an image-based object detection process for charging an electric vehicle according to one embodiment of the present invention.

[0041] FIG. 11 is a flowchart illustrating a process for checking the safety of a charging process and / or a charging preparation process as an image-based object detection process for charging an electric vehicle according to another embodiment of the present invention.

[0042] FIG. 12 is a flowchart illustrating an image-based object detection process for charging an electric vehicle according to another embodiment of the present invention.

[0043] FIG. 13 is a diagram conceptually illustrating a process of estimating the position of an object by considering the position of a camera in an image-based object detection process for charging an electric vehicle according to one embodiment of the present invention.

[0044] FIG. 14 is a diagram conceptually illustrating a process of estimating an angle of an electric vehicle based on an angle of a license plate within an image in an image-based object detection process for charging an electric vehicle according to one embodiment of the present invention.

[0045] FIG. 15 is a flowchart conceptually illustrating an image-based object detection method for charging an electric vehicle according to one embodiment of the present invention.

[0046] FIG. 16 is a flowchart conceptually illustrating a control process of a charging robot linked to an image-based object detection method for charging an electric vehicle according to one embodiment of the present invention.

[0047] FIG. 17 is a block diagram illustrating a generalized configuration of hardware of a device, a controller, a computing system, and / or a charging robot for detecting an object based on an image, in relation to an electric vehicle charging system, an electric vehicle power supply equipment (EVSE), and / or a charging manipulator according to one embodiment of the present invention.

[0048] The present invention is susceptible to various modifications and embodiments. Specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and technical scope of the present invention. Throughout the description of each drawing, similar reference numerals have been used to designate similar components.

[0049] Terms such as "first," "second," "A," and "B" may be used to describe various components, but these components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present invention, the first component could be referred to as the "second component," and similarly, the second component could also be referred to as the "first component." The term "and / or" includes any combination of multiple related items listed or any one of multiple related items listed.

[0050] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0051] The terminology used in this application is only used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprise" or "have" indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0052] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and will not be interpreted in an idealized or overly formal sense unless explicitly defined herein.

[0053] Some terms used in this specification are defined as follows:

[0054] An electric vehicle (EV) may refer to an automobile as defined in 49 CFR (Code of Federal Regulations) 523.3, among other provisions. An EV is capable of highway travel and can be powered by electricity supplied by an onboard energy storage device, such as a rechargeable battery, from an external power source. This power source may include a residential or public power service, or a generator powered by onboard fuel.

[0055] An electric vehicle (EV) can be referred to as an electric car, electric automobile, ERV (electric road vehicle), PV (plug-in vehicle), xEV (plug-in vehicle), etc., and an xEV can be referred to as or distinguished as a BEV (plug-in all-electric vehicle or battery electric vehicle), PEV (plug-in electric vehicle), HEV (hybrid electric vehicle), HPEV (hybrid plug-in electric vehicle), PHEV (plug-in hybrid electric vehicle), etc.

[0056] A plug-in electric vehicle (PEV) can be referred to as an electric vehicle that recharges its onboard primary battery by connecting to the power grid.

[0057] A plug-in vehicle (PV) may be referred to herein as a vehicle that can be recharged wirelessly from an Electric Vehicle Supply Equipment (EVSE) without using a physical plug and socket.

[0058] Heavy duty vehicles (HD Vehicles) may refer to any vehicle with four or more wheels as defined in 49 CFR 523.6 or CFR 37.3 (bus).

[0059] A light-duty plug-in electric vehicle (LDEV) may refer to a vehicle with three or four wheels that is propelled by an electric motor powered by a rechargeable battery or other energy source, primarily intended for use on public streets, roads, and highways. A LEV may be defined as having a gross weight of less than 4.545 kg.

[0060] A wireless power charging system (WCS) may refer to a system for controlling between a Supply Device (or Ground Assembly, GA) and an EV device (or Vehicle Assembly, VA), including wireless power transfer, alignment, and communication.

[0061] Wireless power transfer (WPT) can refer to the transfer of electrical power from an alternating current (AC) power supply network, such as a utility or grid, to an electric vehicle through contactless means.

[0062] A utility provides electrical energy and can be defined as a collection of systems, typically including a Customer Information System (CIS), Advanced Metering Infrastructure (AMI), and a Rates and Revenue system. Utilities enable plug-in electric vehicles to access energy through price lists or discrete events. Utilities can also provide information on tariffs, metered power consumption intervals, and EV program qualifications for plug-in electric vehicles.

[0063] Smart charging can be described as a system where EVSE and / or electric vehicles (including plug-in hybrid electric vehicles) communicate with the power grid to optimize vehicle charge or discharge rates to grid capacity or time of day for cost-to-use ratios.

[0064] Automatic charging can be defined as the act of positioning a vehicle in a suitable location relative to a primary charger assembly capable of transmitting power and charging it either conductively or inductively. Automatic charging can be performed after obtaining the necessary authentication and authorization.

[0065] Interoperability can refer to the state in which components of a system can work together to achieve the intended function of the entire system. Information interoperability can refer to the ability of two or more networks, systems, devices, applications, or components to share information securely and effectively and easily with little or no user inconvenience.

[0066] An inductive charging system can refer to a system that electromagnetically transfers energy in the forward direction from the power supply network to an electric vehicle via a loosely coupled transformer. In this embodiment, the inductive charging system can correspond to an electric vehicle charging system.

[0067] An inductive coupler is a transformer that is formed by a primary device and a secondary device and transmits power through electrical isolation.

[0068] Inductive coupling can refer to the magnetic coupling between two coils. The two coils can refer to the primary coil / ground assembly coil and the secondary coil / vehicle assembly coil.

[0069] A supply power circuit (SPC) / ground assembly (GA) may refer to an assembly disposed on the primary / ground assembly or infrastructure side, including a primary coil / GA coil and other suitable components. The other suitable components may include at least one component for controlling impedance and resonant frequency, ferrite for reinforcing a magnetic path, and electromagnetic shielding materials. For example, the SPC or GA may include a power / frequency conversion device necessary to function as a power source of a wireless charging system, an SPC controller / GA controller, and wiring from the grid, and wiring between each unit and filtering circuits, a housing, etc.

[0070] An EV power circuit (EVPC) / vehicle assembly (VA) may refer to an assembly placed in a vehicle, including a secondary coil / VA coil and other suitable components. The other suitable components may include at least one component for controlling impedance and resonant frequency, ferrite for reinforcing the magnetic path, and electromagnetic shielding materials. For example, an EVPC or VA may include a rectifier / power converter necessary to function as a vehicle component of a wireless charging system, an EVPC controller / VA controller, and wiring for a vehicle battery, as well as wiring between each unit and filtering circuits, a housing, etc.

[0071] The aforementioned SPC may be referred to or distinguished as a ground assembly (GA), etc., and similarly, the EVPC may be referred to or distinguished as a vehicle assembly (VA), etc.

[0072] The aforementioned GA may be referred to as a primary device (PD), a primary device, etc., and similarly, the VA may be referred to as a secondary device (SD), a secondary device, etc.

[0073] The aforementioned GA may be referred to as a supply device, a power supply-side device, etc., and similarly, the VA may be referred to as an electric vehicle device (EV device), an electric vehicle-side device, etc.

[0074] A primary device may be a device external to the electric vehicle that provides contactless coupling to the secondary device. The primary device may be referred to as a primary-side device. When the electric vehicle receives power, the primary device may act as a power source that transmits power. The primary device may include a housing and all covers.

[0075] A secondary device may be a device mounted on an electric vehicle that provides contactless coupling to the primary device. The secondary device may be referred to as a secondary device. When the electric vehicle receives power, the secondary device can transfer power from the primary device to the electric vehicle. The secondary device may include a housing and all covers.

[0076] The supply power electronics may be part of the SPC or GA that regulates the output power level to the primary coil / GA coil based on information from the vehicle. The EV power electronics may be part of the EVPC or VA that monitors certain vehicle parameters during charging and initiates communication with the SPC or GA to control the output power level.

[0077] The supply power electronics described above may be referred to as ground assembly electronics (GA electronics), a ground assembly controller (GA controller), or a primary device communication controller (PDCC), and the electric vehicle power electronics (EV power electronics) may be referred to as vehicle assembly electronics (VA electronics), a vehicle assembly controller (VA controller), or an electric vehicle communication controller (VA controller).

[0078] The magnetic gap may refer to the vertical distance between the highest plane of the upper portion of the litz wire or the upper portion of the magnetic material of the primary coil / GA coil and the lowest plane of the lower portion of the litz wire or the magnetic material of the secondary coil / VA coil when they are aligned with each other.

[0079] Ambient temperature may refer to the ground level temperature measured in the atmosphere of a target subsystem that is not directly exposed to sunlight.

[0080] Vehicle ground clearance can refer to the vertical distance between the road or pavement and the lowest part of the vehicle's floor pan.

[0081] Vehicle magnetic ground clearance may refer to the vertical distance between the lowest plane of the floor of the Litz wire or the insulating material of the secondary coil / VA coil mounted on the vehicle and the road pavement.

[0082] Secondary coil surface distance / Vehicle assembly (VA) coil surface distance may refer to the vertical distance between the bottommost plane of the Litz wire or the magnetic material of the secondary coil / VA coil and the lowest outer surface of the secondary coil / VA coil. This distance may include additional items packaged with protective covering material and coil packaging material.

[0083] The secondary coil described above may be referred to as a VA coil, a vehicle coil, a receiver coil, etc., and similarly, the primary coil may be referred to as a ground assembly coil (GA coil), a transmit coil, etc.

[0084] An exposed conductive component may refer to a conductive component of an electrical device (e.g., an electric vehicle) that can be touched by a person and is not normally conductive but may become conductive in the event of a fault.

[0085] Hazardous live component may refer to a live component that may, under certain conditions, cause a hazardous electric shock.

[0086] Live component may refer to any conductor or conductive part that is electrically active in its basic use.

[0087] Direct contact can refer to contact between living beings, such as humans.

[0088] Indirect contact may refer to contact with exposed, conductive, live components due to an insulation failure (see IEC 61140).

[0089] Alignment may refer to a process of finding the relative position of a secondary device to a primary device for a specified efficient power transfer, and / or a process of finding the relative position of a primary device to a secondary device. In this specification, alignment may refer to, but is not limited to, the positional alignment of a wireless power transfer system.

[0090] Pairing may refer to the process of associating a vehicle (electric vehicle) with a single dedicated ground assembly (primary device) arranged to transfer power. In this specification, pairing may include the process of associating a charging spot or a specific SPC / ground assembly with an EVPC / vehicle assembly controller.

[0091] Correlation / Association may include the process of establishing a relationship between two peer communication entities.

[0092] Command and control communication may refer to communication between an electric vehicle power supply unit and an electric vehicle that exchanges information necessary to initiate, control, and terminate the wireless power transfer process.

[0093] High-level communication can handle all information beyond what command and control communication can handle. High-level communication data links can use, but are not limited to, power line communication (PLC).

[0094] Low power excitation may refer to, but is not limited to, activating the electric vehicle to detect the primary device for precision positioning and pairing, and vice versa.

[0095] A Service Set Identifier (SSID) is a unique 32-character identifier attached to the header of packets transmitted over a wireless LAN. The SSID identifies the basic service set (BSS) to which a wireless device is attempting to connect. Essentially, SSIDs distinguish multiple wireless LANs. Therefore, all access points (APs) and terminal / station devices attempting to use a specific wireless LAN can use the same SSID. Devices that do not use a unique SSID cannot join a BSS. Because SSIDs are visible in plaintext, they may not provide any security features to the network.

[0096] The ESSID (Extended Service Set Identifier) ​​is the name of the network you want to connect to. It's similar to the SSID, but can be a more extensive concept.

[0097] A BSSID (Basic Service Set Identifier) ​​is typically 48 bits long and is used to identify a specific BSS (Basic Service Set). For infrastructure BSS networks, the BSSID can be the MAC (Medium Access Control) of the AP device. For independent BSSs or ad hoc networks, the BSSID can be generated with any value.

[0098] A charging station may include at least one ground assembly and at least one ground assembly controller that manages the at least one ground assembly. The ground assembly may include at least one wireless communication device. A charging station may refer to a location equipped with at least one ground assembly, such as a home, office, public space, road, or parking lot.

[0099] In this specification, association may be used as a term meaning the procedure for establishing wireless communication between an Electric Vehicle Communication Controller (EVCC) and a Supply Equipment Communication Controller (SECC) that controls charging infrastructure.

[0100] Hereinafter, a preferred embodiment according to the present invention will be described in detail with reference to the attached drawings.

[0101] FIG. 1 is a drawing showing an example of an image-based object detection process for charging an electric vehicle to which one embodiment of the present invention is applied, in which the charging port of the electric vehicle is parked so that it is out of the operating range of a charging robot.

[0102] Referring to FIG. 1, an electric vehicle charging process can be performed by at least one component of an electric vehicle (100) and a charging station (charging station), and can be used to transmit power to the electric vehicle (100) wired or wirelessly.

[0103] An electric vehicle (100) according to an embodiment of the present invention may include a hybrid vehicle having both an electric motor and a general internal combustion engine, and may include not only an automobile but also a motorcycle, a cart, a scooter, and an electric bicycle.

[0104] Here, an electric vehicle (100) can be generally defined as a vehicle (automobile) that supplies current induced from a rechargeable energy storage device, such as a battery, as an energy source for an electric motor, which is a power device.

[0105] Additionally, the electric vehicle (100) may include a power receiving device / pad with a receiving coil to wirelessly charge the battery, and may also include a plug-in port to allow for wired battery charging. In this case, an electric vehicle (100) capable of wired battery charging may be referred to as a plug-in electric vehicle (PEV).

[0106] Here, the charging station can be connected to a power grid or power backbone and can provide alternating current (AC) or direct current (DC) power to a power transmitting device / pad containing a transmitting coil via a power link.

[0107] Additionally, the charging station can communicate with the power grid or an infrastructure management system or infrastructure server that manages the power grid via wired or wireless communication, and can perform wireless communication with the electric vehicle (100). Here, wireless communication may include Bluetooth, Zigbee, cellular, a wireless local area network, etc.

[0108] Additionally, for example, the charging station / charging station may be located in various locations, such as a parking lot attached to the home of an electric vehicle (100) owner, a parking area for charging electric vehicles at a gas station, a parking area at a shopping center or workplace, etc.

[0109] Here, the process of charging the battery of an electric vehicle (100) wired / wirelessly can be performed by first positioning the power receiving device / pad of the electric vehicle (100) in an energy field caused by the power transmitting device / pad, and allowing the transmitting coil of the power transmitting device / pad and the receiving coil of the power receiving device / pad to interact or couple with each other. As a result of the interaction or coupling, electromotive force is induced in the power receiving device / pad, and the battery can be charged by the induced electromotive force.

[0110] Additionally, the charging station / charging station and the power transmission device / pad may be referred to in whole or in part as a supply power circuit (SPC) or a ground assembly (GA), and the SPC or ground assembly may refer to the meaning defined above.

[0111] In addition, all or part of the electric vehicle's (100) power receiving device / pad and other internal components of the electric vehicle may be referred to as an electric vehicle power circuit (EVPC) or vehicle assembly (VA), where the EVPC or vehicle assembly may refer to the meaning defined above.

[0112] An electric vehicle charging system may include, but is not limited to, a conductive charging system using cables or a contactless wireless power transmission system. An electric vehicle charging system can be fundamentally defined as a system that charges a battery mounted on an electric vehicle (100) using power from a commercial power grid or an energy storage device. Such an electric vehicle charging system may take various forms depending on the type of electric vehicle.

[0113] For example, SAE TIR J2954, a leading standard for wireless charging, establishes industry-standard specification guidelines that define interoperability, electromagnetic compatibility, minimum performance, safety, and testing acceptance criteria for wireless charging of light-duty electric and plug-in electric vehicles. Those skilled in the art will readily understand that similar guidelines can also be applied to wired charging systems.

[0114] The wired / wireless charging system for electric vehicles can largely include the following three elements:

[0115] 1) GA coil for power connection and grid connection power converter, communication link with vehicle system

[0116] 2) VA coil with rectifier and filtering components and charging control power electronics for regulation / safety / shutdown if required, and communication link to base station side.

[0117] 3) Related modules required for secondary energy storage system, battery management system components and in-vehicle communication (CAN, LIN) required for battery SOC, charge rate and other necessary information.

[0118] Additionally, the VSE Field of WLAN does not contain detailed information such as maximum charging power, which may prevent users from properly connecting to the charger they actually want to charge at a charging station where one SECC and multiple EVSEs coexist.

[0119] In addition, after the vehicle is associated with the charger / SECC and WLAN Association, a Positioning procedure must be performed to ensure precise alignment between the vehicle and the EVSE. In the case of wireless charging (WPT), if the vehicle pad and the EVSE pad are not properly aligned, charging efficiency may decrease or charging may become impossible. In addition, after Positioning is complete, a procedure is required to check whether a physical connection with the EVSE to be connected is possible. If a physical connection with the EVSE to be charged is not possible, charging will not proceed. Therefore, a Pairing procedure must be performed between the vehicle and the EVSE after WLAN Association. For this purpose, a separate Positioning and Pairing method using a Point-to-Point Signal (P2PS) is defined in the IEC 61950-2 standard for wireless charging, but no separate Positioning and Pairing method has been defined in any standard for a robotic charging system based on an automatic coupling device (ACD).

[0120] The present invention proposes an image-based object detection procedure for accurate positioning, aligning, pairing and mating procedures between a charging manipulator for an electric vehicle (100) and a charging door / port of the electric vehicle (100), and a control procedure of a charging robot associated therewith.

[0121] Power transfer can be accomplished from a transmitting coil / primary coil (L1) to a receiving coil / secondary coil (L2). At this time, the resonance frequencies of the transmitting coil (L1) and the receiving coil (L2) can be configured to be similar or the same, and the receiving coil (L2) can be configured to be positioned at a close range to the electromagnetic field generated by the transmitting coil (L1).

[0122] Meanwhile, since power loss may increase as the distance between the transmitting coil (L1) and the receiving coil (L2) increases, setting the positions of the two may be an important factor.

[0123] At this time, the transmitting coil (L1) may be included in the power transmitting device / pad, and the receiving coil (L2) may be included in the power receiving device / pad. In addition, the transmitting coil may be referred to as a primary coil or a GA coil (Ground Assembly coil), and the receiving coil may be referred to as a secondary coil or a VA coil (Vehicle Assembly coil). Therefore, the determination of the positions between the power transmitting device / pad and the power receiving device / pad or the determination of the positions between the electric vehicle (100) and the power transmitting device / pad are also important factors.

[0124] The positional alignment between the power transmission device / pad in FIG. 1 and the power reception device / pad built into the electric vehicle (100) may correspond to the previously described term alignment, and therefore may be defined as the positional alignment between the SPC / GA and the EVPC / VA, and is not limited to the positional alignment of the power transmission device / pad and the power reception device / pad.

[0125] Referring again to FIG. 1, an exemplary location where a charging door / port (120) of an electric vehicle (100) to which one embodiment of the present invention is applied can be positioned is illustrated.

[0126] In an alternative embodiment of the present invention, a front bumper, a front fender panel, or a rear fender panel may be provided as a location where a charging door / port can be placed in an electric vehicle (100).

[0127] Although not illustrated in detail in FIG. 1, in the embodiments of the present invention, each charging door / port may be classified into a vehicle internal switch type in which a switch capable of opening and closing a door is located inside the vehicle, a vehicle external switch type in which a switch capable of opening and closing a door from outside the vehicle is located, and a push door type in which a door capable of opening and closing a door from outside the vehicle is operated by pressing pressure.

[0128] The in-vehicle switch type has a switch that can open and close the charging door / port located near the driver's seat inside the vehicle, and the driver or passenger can open and close the charging door / port by operating the switch. The external switch type generally has a switch that can mechanically open and close the charging door / port located outside the vehicle, and can generally be located near the charging door / port. However, according to a modified embodiment, the external switch may be manually operated or may be located at a predetermined distance from the charging door / port.

[0129] The push door type is a modified version of the vehicle exterior switch type, wherein the push switch can be located on the inside of the charging door / port. The push switch can be operated manually or mechanically.

[0130] There may be embodiments in which a separate cover is placed for the inlet, which is a charging socket installed in the vehicle. If the inlet cover is placed inside the charging door / port, the inlet cover may be implemented with a plastic or rubber component for insulation.

[0131] Referring again to FIG. 1, the position of the charging door / port (120) of an electric vehicle (100) to which one embodiment of the present invention is applied, a charging robot (200) having a charging manipulator, and an operating range (240) of the charging robot (200) are exemplarily illustrated.

[0132] The charging robot (200) may be a device of various types referred to as an ACD (Automated connection device, Automatic Charging Device, Autoconnect Charging Device), or may include an ACD.

[0133] In FIG. 1, for convenience of explanation, the charging robot (200) is illustrated as an ACD-S (Side) type equipped with a charging manipulator, but the idea of ​​the present invention is not limited to a specific embodiment, and in an alternative embodiment of the present invention, it can also be applied to an ACD-U (Underbody) or ACD-R (Roof) type.

[0134] In one embodiment of the present invention, the operating range (240) of the charging robot (200) may refer to an ACD mating space. In an alternative embodiment of the present invention, the operating range (240) of the charging robot (200) may refer to a range of space where the movable distance of the charging robot (200) and the ACD mating space are combined.

[0135] Referring to the process of identifying the type and specifications of the electric vehicle (100) to be described later, the operating range (240) of the charging robot (200) of FIG. 1 may be adjusted to determine whether mating is possible by taking into account the operating range of the manipulator, etc., of the electric vehicle (100), which is the ACD counterpart.

[0136] In FIG. 1, a state in which the charging port (charging door / port, 120) of an electric vehicle (100) is parked so that it is out of the operating range (240) of a charging robot (200) is illustrated, and therefore, an image-based object detection method for charging an electric vehicle according to an embodiment of the present invention can estimate the position and angle of the electric vehicle (100) based on an image, recognize that the charging port (120) is parked so that it is out of the operating range (240) of the charging robot (200), and guide the electric vehicle (100) to be parked again.

[0137] FIG. 2 is a drawing illustrating cameras placed in a charging station and a parking space as an example of an image-based object detection process for charging an electric vehicle to which one embodiment of the present invention is applied.

[0138] FIG. 3 is a diagram conceptually illustrating a process of obtaining an image by a camera placed in a parking space as an example of an image-based object detection process for charging an electric vehicle to which one embodiment of the present invention is applied.

[0139] Referring to FIGS. 2 and 3 together, one first camera (300) for object recognition may be placed for each parking space or parking area (400). As illustrated in FIG. 3, the first camera (300) may be placed such that the shooting range (320) of the first camera (300) includes (covers) the entire parking space (400) to which the first camera (300) corresponds.

[0140] As illustrated in FIG. 3, the first camera (300) can be installed on either the front or rear of the parking space (400).

[0141] In order to compare the operating range (240) of the charging robot (200) and the position of the charging door / port (120) of the electric vehicle (100), the electric vehicle (100) must be parked in a specific parking space (400). Therefore, the license plate / vehicle number recognition result of the electric vehicle (100) in the image obtained by the first camera (300), which is the main imaging device, can be used to estimate the position and angle of the electric vehicle (100) in the parking space (400) (for example, parked in a specific one of parking spaces A, B, and C).

[0142] In an alternative embodiment of the present invention, a second camera (340), which is an auxiliary camera that recognizes the entry of the electric vehicle (100) and the license plate / vehicle number when the electric vehicle (100) enters the charging station, may be further placed.

[0143] In an alternative embodiment of the present invention, the second camera (340) may perform an auxiliary role, such as recognizing the license plate / vehicle number of an electric vehicle (100) entering a charging station and cross-checking the license plate / vehicle number recognition result of the electric vehicle (100) within the image obtained by the first camera (300).

[0144] The license plate / vehicle number of the electric vehicle (100) recognized by the second camera (340) can be used to verify whether it matches the license plate / vehicle number recognition result of the electric vehicle (100) in the image obtained by the first camera (300), or can be used to obtain information in advance, such as the vehicle type, charging port type, and charging port location of the electric vehicle (100), by using the license plate / vehicle number recognized by the second camera (340) when the electric vehicle (100) enters.

[0145] FIGS. 4 and 5 are conceptual drawings illustrating examples of a two-dimensional reference coordinate system of a parking space to which one embodiment of the present invention is applied.

[0146] At this time, as shown in FIGS. 4 and 5, the x-axis can indicate the left-right direction of the vehicle or parking space, and the y-axis can indicate the front-back direction of the vehicle or parking space. Although not shown in FIGS. 4 and 5, the z-axis can indicate the up-down direction of the vehicle.

[0147] The coordinate system of a parking space can be based on a location on the boundary of the parking space. The coordinate system can be based on any of the four corners of an individual parking space, or it can be based on the midpoint between two adjacent corners of the individual parking space.

[0148] FIG. 6 is a flowchart illustrating an image-based object detection process for charging an electric vehicle according to one embodiment of the present invention.

[0149] Referring to FIG. 6, an object can be detected / segmented from an image obtained from a first camera (300) for object recognition that corresponds to each parking space (400) and includes the entirety of each parking space (400) within a shooting range (S1000).

[0150] At this time, in process S1000, an object is detected from the image, and segmentation may be performed to extract the shape of the detected object. In an alternative embodiment of the present invention, object detection and segmentation may be performed simultaneously. In one embodiment of the present invention, detection and / or segmentation may be performed using a known artificial neural network.

[0151] A classification process S1100 may be performed to classify the type of the detected / segmented object. In process S1100, it may be determined whether the class of the detected / segmented object is a car. Depending on the type of the detected / segmented object, subsequent processes may branch.

[0152] If the result object of the classification process S1100 is a vehicle, the position and angle of the vehicle can be estimated (S1300).

[0153] Independently of process S1000, the vehicle license plate can be recognized from an image obtained from the first camera (300) (S1200).

[0154] Vehicle license plates are generally rectangular or rectangular with rounded corners, making them easy to detect and segment. Since the numbers in a vehicle license plate are a combination of letters and numbers, the vehicle license plates can be easily identified using general OCR technology or artificial neural network-based text recognition technology.

[0155] In an alternative embodiment of the present invention, a vehicle license plate may be additionally recognized from an image obtained from the second camera (340) of FIG. 2. The vehicle license plate or number additionally recognized from the image of the second camera (340) may be used to assist process S1200 or to cross-check the recognition results of process S1200.

[0156] Vehicle information can be extracted by searching the vehicle number, which is the recognition result of process S1200, in the database (500) (S1220). The vehicle information may include the vehicle type, size, location of the charging port, etc.

[0157] In process S1300, information about the vehicle can be used to estimate the position and angle of the vehicle in the parking space (400).

[0158] Since the image obtained from the first camera (300) is biased toward the front or rear of the vehicle, information such as the size of the vehicle may be required to estimate the position and angle of the vehicle in the parking space (400).

[0159] The result of process S1300 may include two-dimensional coordinates (x, y) on the parking space (400) and an angle θ formed by the vehicle with the x-axis or y-axis on the parking space (400). The result of process S1300 may be described in FIG. 8, which will be described later.

[0160] In process S1300, after extracting the lowest point in the image of the vehicle segment, the position of the vehicle can be estimated based on the geometric relationship with the first camera (300) (the positional relationship between the first camera (300) and the parking space (400). This process can be supplemented with reference to FIG. 13, which will be described later.

[0161] In process S1300, the periphery of the license plate is recognized through text detection, and then a horizontal line is extracted to estimate the angle of the license plate within the image. Based on the angle of the license plate within the image and vehicle information (size, type), the angle θ that the vehicle forms with the x-axis or y-axis within the parking space (400) can be estimated. This process can be supplemented with reference to FIG. 14, which will be described later.

[0162] If the result of the classification process S1100 is not a vehicle, an additional process of determining whether the object is a person can be used to estimate the location of the person (S1620).

[0163] In process S1620, after extracting the lowest point in the image of the human segment, the human position (x, y) can be estimated based on the geometric relationship with the first camera (300) (the positional relationship between the first camera (300) and the parking space (400). This process can be supplemented with reference to FIG. 13, which will be described later.

[0164] By utilizing the vehicle's position / angle information of process S1300, the vehicle can be mapped onto a 2D map of the parking space (400). By utilizing the person's position information of process S1620, the person can be mapped onto a 2D map of the parking space (400).

[0165] FIG. 7 is a flowchart illustrating an image-based object detection process for charging an electric vehicle according to another embodiment of the present invention.

[0166] Referring to FIG. 7, the automobile classification process S1100 and the person classification process S1600 can be performed in parallel to classify the type of at least one object detected / segmented in the process S1000.

[0167] The processes other than process S1100 and process S1600 of Fig. 7 are the same or similar to the process of Fig. 6, so overlapping descriptions may be omitted.

[0168] For example, if two or more objects are detected in an image, each of the objects can be classified into a first object corresponding to a vehicle and a second object corresponding to a person / animal by processes S1100 and S1600, which are performed in parallel.

[0169] For the first object corresponding to the vehicle, the position coordinates (x, y) and angle θ on the 2D map of the parking space (400) can be estimated by process S1300.

[0170] For the second object corresponding to a person / animal, the location coordinates (x, y) on the 2D map of the parking space (400) can be estimated by process S1620.

[0171] In process S1000 of FIGS. 6 and 7, a technology capable of recognizing a car or person as an object in a single image and extracting the shape of the object may be utilized. For example, segmentation models such as CNN, U-NET, and V-NET may be utilized, and the concept of the present invention may be applied to various image-based object detection / segmentation technologies without restrictions on selecting a specific type of artificial neural network model.

[0172] In process S1200 of FIGS. 6 and 7, a text detection algorithm may be utilized to recognize a license plate, a key feature of a vehicle, in an image. For example, a CNN-based model may be utilized.

[0173] FIG. 8 is a diagram conceptually illustrating a process of estimating the position and angle of an electric vehicle on a two-dimensional reference coordinate system of a parking space, as an example of an image-based object detection process for charging an electric vehicle to which one embodiment of the present invention is applied.

[0174] Referring to FIG. 8, a two-dimensional reference coordinate system (x, y) having one of the four corners of the parking space as the origin and an angle θ formed by the parked direction of the vehicle with the y-axis are shown.

[0175] FIG. 9 is a flowchart conceptually illustrating an embodiment in which an image-based object detection process for charging an electric vehicle and a charging robot control process are linked according to one embodiment of the present invention.

[0176] The process of FIG. 9 may be performed by a computing system of a charging station, a charging robot, a controller of the charging robot, a controller of the EVSE, etc., or by cooperation and division of labor among these devices.

[0177] Referring to FIG. 9, the position and angle of the vehicle are estimated in process S1300, and it can be determined whether the position of the charging port derived based on the vehicle information and the position and angle of the vehicle is within the operating range of the charging robot (S1320).

[0178] At this time, the operating range of the charging robot can be given as a range such as [x_min, x_max], [y_min, y_max], for example.

[0179] If the location of the charging port is within the operating range of the charging robot, the charging robot can be controlled to approach the vehicle, align the manipulator to a connected or chargeable location, and start charging (S1400).

[0180] If the location of the charging port is not within the operating range of the charging robot, the vehicle is requested or guided to re-park (S1340), and processes S1000, S1100, S1300, etc. can be performed again for the image obtained after the vehicle is re-parked.

[0181] FIG. 10 is a flowchart conceptually illustrating a charging robot control process linked to an image-based object detection process for charging an electric vehicle according to one embodiment of the present invention.

[0182] Referring to Fig. 10, the control process S1400 of the charging robot is illustrated as an example.

[0183] The robot posture can be compensated using vehicle position / angle information (S1420).

[0184] The robot distal end and each joint driving angle can also be determined (S1440).

[0185] It can be determined whether the robot end angle and the socket angle correspond (match) (S1460).

[0186] If the robot end angle and the socket angle do not correspond (do not match), steps S1420 and S1440 may be performed again.

[0187] When the robot end angle and the socket angle correspond, the charger can be connected and charging can begin (S1500).

[0188] FIG. 11 is a flowchart illustrating a process for checking the safety of a charging process and / or a charging preparation process as an image-based object detection process for charging an electric vehicle according to another embodiment of the present invention.

[0189] Referring to Fig. 11, it can be determined whether the location of the person estimated in process S1620 is within the operating range of the charging robot (S1640).

[0190] If the estimated human location is not within the operating range of the charging robot, no special action may be required and continuous monitoring may be performed (repeat S1000, S1600, S1620).

[0191] If the estimated location of the person is within the operating range of the charging robot, safety actions can be performed (S1660).

[0192] Safety actions can include, but are not limited to, generating an alarm (visual or audible), stopping the charging robot when it is moving, and stopping charging during charging.

[0193] FIG. 12 is a flowchart illustrating an image-based object detection process for charging an electric vehicle according to another embodiment of the present invention.

[0194] Referring to Fig. 12, an alternative embodiment of the process for estimating the position and angle of a vehicle is illustrated. Descriptions of content overlapping with those in Figs. 6 and 7 in Fig. 12 may be omitted.

[0195] In process S1200, a license plate can be recognized from an image.

[0196] At this time, the license plate can be detected as a third object (S1202).

[0197] Text is detected from the detected license plate and the vehicle number can be identified (S1204).

[0198] A horizontal line of a license plate or text is identified, and an angle formed by the horizontal line of the license plate or text on the image can be estimated (S1210).

[0199] In process S1300, the angle of the vehicle can be estimated based on the angle formed by the horizontal line of the license plate or text on the image and the geometry of the first camera (300) and the parking space (400).

[0200] FIG. 13 is a diagram conceptually illustrating a process of estimating the position of an object by considering the position of a camera in an image-based object detection process for charging an electric vehicle according to one embodiment of the present invention.

[0201] Referring to Fig. 13, an object position estimation process is exemplarily illustrated.

[0202] The location of the vehicle can be estimated using the center point (u, v) of the license plate on the image.

[0203] Depth can be calculated using geometric information (camera installation angle and height, etc.) between the camera (300) and the ground.

[0204] f: focal length

[0205] y: cy(principal point) - y pixel coordinate value of the object

[0206] θ c : camera angle(tilt)

[0207] θ i : Angle between focal length and object position in the image plane

[0208] θ o : θ c - θ i

[0209] C: Height where camera (300) is installed

[0210] d: distance between camera(300) and object

[0211] D: object depth

[0212] When , depth D can be calculated as in mathematical equation 1 below.

[0213] [Mathematical Formula 1]

[0214] d = C / (cos θ o ), D = d*(cos θ i )

[0215] At this time, the location of the license plate can be converted into a 3D point as in mathematical expression 2 by utilizing the depth and camera intrinsic parameters.

[0216] [Equation 2]

[0217] X = (f x [uC x ]) / D, Y = (f y [vC y ]) / D, Z = D

[0218] FIG. 14 is a diagram conceptually illustrating a process of estimating an angle of an electric vehicle based on an angle of a license plate within an image in an image-based object detection process for charging an electric vehicle according to one embodiment of the present invention.

[0219] Referring to Fig. 14, an automobile angle estimation process is exemplarily illustrated.

[0220] You can crop the image to the area of ​​interest centered on the recognized license plate.

[0221] Edge detection (600) can be performed to find a horizontal line (620) from the cropped image. For example, a representative line can be searched for using the Ransac algorithm, and a straight line equation for the representative line can be derived.

[0222] The two intersections between the straight line above and the left and right ends of the license plate can be converted into 3D coordinates p1 and p2 using the method of Fig. 13.

[0223] The above p1 and p2 are points on the camera coordinate system tilted by the camera installation angle, and the rotation transformation of mathematical expression 3 can be applied to convert them to the reference coordinate system of FIG. 8.

[0224] [Equation 3]

[0225] P1 = R*p1

[0226] P2 = R*p2

[0227] P1 and P2 are the coordinates of p1 and p2 transformed into the reference coordinate system of Fig. 8, and R is a rotation transformation matrix.

[0228] Vector v passing through P1 and P2 12 This represents the angle of the car, so the vector v is parallel to the parking line. x Wow 12 The angle between the two is calculated as in mathematical formula 4, and the angle θ of the car is calculated as in mathematical formula 4. car can be calculated.

[0229] [Equation 4]

[0230] θ car = arccos [ (v 12 ㆍv x ) / ( ||v 12 ||ㆍ||v x || ) ]

[0231] (|| ||) means the norm of the vector.

[0232] The process of mapping the location of objects onto a 2D map is then described.

[0233] The position / angle estimation results can be converted into a 2D coordinate system with the lower right point of the parking space in Fig. 8 as the origin.

[0234] The rotation and positional relationship between the camera coordinate system and the reference coordinate system of Fig. 8 can be defined as in mathematical equations 5 and 6.

[0235] At this time, since it is a 2D coordinate in the form of a bird's-eye view, it can be given as z=0.

[0236] [Equation 5]

[0237]

[0238] [Equation 6]

[0239] t = [tx, ty, tz]

[0240] Here, R is a rotation transformation matrix, t is a position transformation vector, and ti (i=x, y, z) refers to the position transformation values ​​in each of the x, y, and z directions.

[0241] Using the vehicle information obtained in process S1220, the location of the charging port within the vehicle can be finally mapped onto a 2D map through the position / angle between the license plate and the charging port.

[0242] As a result of this mapping, the position, angle, and location of the charging port of the vehicle on the parking space can be mapped as shown in Fig. 1, and the operating range of the charging robot and the location of the vehicle charging port can be compared.

[0243] FIG. 15 is a flowchart conceptually illustrating an image-based object detection method for charging an electric vehicle according to one embodiment of the present invention.

[0244] Referring to FIG. 15, an image-based object detection method for charging an electric vehicle according to an embodiment of the present invention is a method executed by a computing system or controller on the side of a charging station that is linked to an electric vehicle power supply (EVSE) including a charging manipulator, and may include a step (S2100) of detecting a first object corresponding to an electric vehicle from an image obtained by a camera capable of obtaining an image of an electric vehicle from the outside of the electric vehicle; a step (S2200) of estimating a position and angle of the electric vehicle on a parking space based on position information of the first object in the image; a step (S2300) of detecting at least one second object other than the electric vehicle from the image; and a step (S2400) of estimating a position of at least one second object on the parking space based on the position of the at least one second object in the image.

[0245] The location information of the first object within the image may include a position and an angle. The location of the car may be estimated from the segmentation result of the car object, and the angle of the car may be estimated from the angle of the car license plate. In addition, car segmentation or detection box information may be partially utilized.

[0246] Step S2100 may refer to processes S1000 and S1100 of FIGS. 6, 7, and 12.

[0247] Step S2200 may refer to process S1300 of FIGS. 6, 7, and 12.

[0248] Step S2300 may refer to processes S1000 and S1600 of FIGS. 6, 7, and 12.

[0249] Step S2400 may refer to process S1620 of FIGS. 6, 7, and 12.

[0250] In the step (S2200) of estimating the position and angle of the electric vehicle, the position and angle of the electric vehicle can be estimated on a two-dimensional coordinate system based on the position on the boundary line of the parking space as a reference point. The two-dimensional coordinate system based on the position on the boundary line of the parking space as a reference point can be referenced to FIGS. 4 and 5.

[0251] The step of estimating the position and angle of the electric vehicle (S2200) may include a step of mapping the first object on a two-dimensional map corresponding to the parking space based on the position information of the first object in the image; and a step of estimating the position and angle of the first object on the two-dimensional map as the position and angle of the electric vehicle.

[0252] The location information of the first object in the image may include location information of the vehicle and angle information of the license plate.

[0253] According to an alternative embodiment of the present invention, the position and angle can be estimated using only information from either the license plate or the vehicle.

[0254] In an image-based object detection method for charging an electric vehicle according to one embodiment of the present invention, the step of detecting a first object corresponding to the electric vehicle may include a segmentation step of dividing an outline of the first object, and the position and angle of the first object within the image may be obtained based on the segmentation result of the first object.

[0255] In an alternative embodiment of the present invention, segmentation is not necessarily required, and the position of the vehicle can be estimated using the coordinates of the detection box.

[0256] An image-based object detection method for charging an electric vehicle according to one embodiment of the present invention may further include a step of detecting a third object corresponding to a license plate of the electric vehicle from an image (see S1202 of FIG. 12), and at this time, the step of estimating the position and angle of the electric vehicle (S2200) may include a step of assisting in extracting position information of a first object in the image based on the position and angle of the third object in the image.

[0257] At this time, the license plate can be considered as an object independent of the vehicle, even though the area of ​​the license plate within the image may be included within the area of ​​the vehicle object.

[0258] In an alternative embodiment of the present invention, the position and angle of a vehicle can be estimated using only the license plate, only the vehicle, or a combination of the angle of the license plate and the position of the vehicle.

[0259] Additionally, after estimating the vehicle's position and angle using only the vehicle, the angle of the license plate can be used to cross-validate the vehicle's angle.

[0260] An image-based object detection method for charging an electric vehicle according to one embodiment of the present invention may further include a step (S1202) of detecting a third object corresponding to a license plate of an electric vehicle from an image, and at this time, the step (S1300, S2200) of estimating a position and angle of the electric vehicle may include a step of estimating an angle of the electric vehicle in a parking space based on an angle of the third object in the image.

[0261] An image-based object detection method for charging an electric vehicle according to one embodiment of the present invention may further include a step of identifying a license plate of an electric vehicle from an image (S1200, S1204); and a step of extracting vehicle information of the electric vehicle from a database (500) using the license plate of the electric vehicle (S1220). At this time, in the step of estimating the position and angle of the electric vehicle (S1300, S2200), the vehicle information of the electric vehicle may be used to estimate the position and angle of the electric vehicle in a parking space.

[0262] An image-based object detection method for charging an electric vehicle according to one embodiment of the present invention may further include a step of estimating the position of a charging port of an electric vehicle based on the position and angle of the electric vehicle in a parking space; and a step (S1320) of determining whether the position of the charging port of the electric vehicle is within the operating range of a charging robot.

[0263] An image-based object detection method for charging an electric vehicle according to one embodiment of the present invention may further include a step (S1340) of guiding the electric vehicle to be re-parked if the location of the charging port of the electric vehicle is not within the operating range of the charging robot; and a step of guiding the charging robot to approach the electric vehicle if the location of the charging port of the electric vehicle is within the operating range of the charging robot.

[0264] The step (S1340) of guiding the electric vehicle to re-park may request the driver to re-park, or, if the vehicle is capable of autonomous parking, may request the EVCC / ECU to re-park.

[0265] In an image-based object detection method for charging an electric vehicle according to one embodiment of the present invention, at least one second object may be a person or an animal.

[0266] An image-based object detection method for charging an electric vehicle according to one embodiment of the present invention may further include a step of determining whether any one or more second objects is located within the operating range of a charging robot; and a step of executing a safety-related action if any one or more second objects is located within the operating range of the charging robot.

[0267] Safety actions can include, but are not limited to, generating an alarm (visual or audible), stopping the charging robot when it is moving, and stopping charging during charging.

[0268] FIG. 16 is a flowchart conceptually illustrating a control process of a charging robot linked to an image-based object detection method for charging an electric vehicle according to one embodiment of the present invention.

[0269] A charging robot (200) and / or a controller of the charging robot (200) related to an electric vehicle power supply (EVSE) that includes a charging manipulator according to one embodiment of the present invention and transmits power to an electric vehicle may include a memory that stores at least one command; a processor that controls the movement of the charging robot (200) or the charging manipulator by loading and executing at least one command from the memory; and a communication interface that transmits and receives information necessary for the process of transmitting power to the electric vehicle to and from the EVSE. The hardware configuration of the charging robot (200) and / or the controller of the charging robot (200) may refer to the configuration illustrated in FIG. 17, which will be described later.

[0270] Referring to FIGS. 16 and 17 together, in a charging robot (200) and / or a controller of the charging robot (200) according to one embodiment of the present invention, a processor can obtain a position and angle of an electric vehicle on a parking space estimated from an image obtained by a camera capable of obtaining an image of an electric vehicle from the outside of the electric vehicle (S2200), can obtain a position of at least one object other than the electric vehicle on the parking space detected from the image (S2400), and can determine whether charging of the electric vehicle is possible based on the position and angle of the electric vehicle on the parking space and the position of at least one object on the parking space (S2600, S1320).

[0271] In a charging robot (200) and / or a controller of the charging robot (200) according to one embodiment of the present invention, the position and angle of the electric vehicle on the parking space, and the position of at least one object on the parking space, can be expressed as a position and angle on a two-dimensional coordinate system with a position on a boundary line on the parking space as a reference point.

[0272] In a charging robot (200) and / or a controller of a charging robot (200) according to one embodiment of the present invention, if the location of the charging port of the electric vehicle, estimated based on the location and angle of the electric vehicle on the parking space, is not within the operating range of the charging robot (200), the processor may determine that charging of the electric vehicle is not possible (S2600, S1320).

[0273] In a charging robot (200) and / or a controller of the charging robot (200) according to one embodiment of the present invention, the processor can determine that charging of an electric vehicle is not possible if the location of at least one object in a parking space is within the operating range of the charging robot (200) (S2600, S1320).

[0274] In a charging robot (200) and / or a controller of the charging robot (200) according to one embodiment of the present invention, if charging of the electric vehicle is possible, the processor can control the movement of the charging robot (200) and the charging manipulator to move the charging robot (200) to the electric vehicle and connect the charging manipulator to the charging port of the electric vehicle (S1400, S1500).

[0275] In order to apply electric vehicle charging robots in real-world situations, a method for recognizing the surroundings of charging stations is required, and for this purpose, various facilities (such as cameras) can be utilized inside and outside the parking lot.

[0276] In actual robot applications, there may be situations where the robot cannot operate due to incorrect parking, and in addition, a dangerous situation may occur if a person approaches the robot while it is operating.

[0277] According to an embodiment of the present invention, in order to solve the above problem, a camera installed on the outside of a car or a charging robot and a robot linkage system are developed, thereby ensuring robot operation efficiency through recognition of the parking status and safety through determination of whether a person is approaching.

[0278] According to an embodiment of the present invention, cost can be reduced because situational awareness is performed using only one external camera without additional sensors mounted on the robot itself.

[0279] According to an embodiment of the present invention, safety can be increased by performing robot control by identifying risk factors such as robot collisions.

[0280] According to an embodiment of the present invention, an unmanned charging system can be operated by making external practical application of the robot charging infrastructure easier.

[0281] According to an embodiment of the present invention, by applying a robot intelligent situation recognition method, it is possible to secure a virtuous cycle structure for electric vehicle distribution by increasing the efficiency of charging infrastructure and enabling the provision of quality services.

[0282] FIG. 17 is a block diagram illustrating a generalized configuration of hardware of a device, a controller, a computing system, and / or a charging robot (200) for detecting an object based on an image or controlling a charging robot (200) in relation to an electric vehicle charging system, an electric vehicle power supply equipment (EVSE), and / or a charging manipulator according to one embodiment of the present invention.

[0283] Although omitted in the drawings in the embodiments of FIGS. 1 to 16, a processor and a memory are electronically connected to each component, and the operation of each component can be controlled or managed by the processor.

[0284] At least a part of the process of the image-based object detection method for charging an electric vehicle (100) according to one embodiment of the present invention and the control method of the charging robot (200) linked to the image-based object detection method may be executed by the computing system (3000) of FIG. 17. The computing system (3000) according to one embodiment of the present invention may include at least one processor (3100) and a memory (3200) that stores instructions that instruct the at least one processor (3100) to perform at least one step. At least a part of the steps of the method according to one embodiment of the present invention may be performed by the at least one processor (3100) loading and executing instructions from the memory (3200).

[0285] The processor (3100) may mean a central processing unit (CPU), a graphics processing unit (GPU), or a dedicated processor on which methods according to embodiments of the present invention are performed.

[0286] Each of the memory (3200) and the storage device (3400) may be configured with at least one of a volatile storage medium and a non-volatile storage medium. For example, the memory (3200) may be configured with at least one of a read-only memory (ROM) and a random access memory (RAM).

[0287] Additionally, the computing system (3000) may include a communication interface (3300) that performs communication via a wired / wireless network.

[0288] Additionally, the computing system (3000) may further include a storage device (3400), an input interface (3500), an output interface (3600), etc.

[0289] Additionally, each component included in the computing system (3000) can communicate with each other by being connected by a bus (3700).

[0290] A device including a processor (3100) according to one embodiment of the present invention may be, for example, a communicable desktop computer, a laptop computer, a notebook, a smart phone, a tablet PC, a mobile phone, a smart watch, smart glasses, an e-book reader, a portable multimedia player (PMP), a portable game console, a navigation device, a digital camera, a digital multimedia broadcasting (DMB) player, a digital audio recorder, a digital audio player, a digital video recorder, a digital video player, a PDA (Personal Digital Assistant), etc.

[0291] An image-based object detection device according to one embodiment of the present invention may be installed on the charging station side in relation to an electric vehicle charging system, an electric vehicle power supply equipment (EVSE), and / or a charging manipulator, and may include a processor (3100) that receives and executes at least one command from a memory (3200).

[0292] According to one embodiment of the present invention, a processor (3100) of an image-based object detection device for charging an electric vehicle is a method executed by a computing system or controller on the side of a charging station that is linked to an electric vehicle power supply (EVSE) including a charging manipulator, wherein a first object corresponding to an electric vehicle can be detected from an image obtained by a camera capable of obtaining an image of an electric vehicle from the outside of the electric vehicle, and a position and angle of the electric vehicle on a parking space can be estimated based on position information of the first object in the image, and at least one or more second objects other than the electric vehicle can be detected from the image, and a position of at least one or more second objects on the parking space can be estimated based on the position of the at least one or more second objects in the image.

[0293] When estimating the position and angle of an electric vehicle, the processor (3100) can estimate the position and angle of the electric vehicle on a two-dimensional coordinate system using a position on the boundary line of a parking space as a reference point.

[0294] When estimating the position and angle of an electric vehicle, the processor (3100) can map the first object on a two-dimensional map corresponding to a parking space based on the position information of the first object in the image, and can estimate the position and angle of the first object on the two-dimensional map as the position and angle of the electric vehicle.

[0295] When detecting a first object corresponding to an electric vehicle, the processor (3100) can segment the outline of the first object, and the position and angle of the first object within the image can be obtained based on the segmentation result of the first object.

[0296] The processor (3100) can detect a third object corresponding to the license plate of an electric vehicle from an image, and when estimating the position and angle of the electric vehicle, can assist in extracting position information of a first object in the image based on the position and angle of the third object in the image.

[0297] The processor (3100) can detect a third object corresponding to the license plate of the electric vehicle from the image, and when estimating the position and angle of the electric vehicle, can estimate the angle of the electric vehicle in the parking space based on the angle of the third object in the image.

[0298] The processor (3100) can identify the license plate of an electric vehicle from an image, extract vehicle information of the electric vehicle from a database using the license plate of the electric vehicle, and estimate the location and angle of the electric vehicle in a parking space using the vehicle information of the electric vehicle when estimating the location and angle of the electric vehicle.

[0299] The processor (3100) can estimate the location of the charging port of the electric vehicle based on the location and angle of the electric vehicle in the parking space, and can determine whether the location of the charging port of the electric vehicle is within the operating range of the charging robot.

[0300] The processor (3100) can guide the electric vehicle to be re-parked if the location of the charging port of the electric vehicle is not within the operating range of the charging robot, and can guide the charging robot to approach the electric vehicle if the location of the charging port of the electric vehicle is within the operating range of the charging robot.

[0301] In an image-based object detection device for charging an electric vehicle according to one embodiment of the present invention, at least one second object may be a person or an animal.

[0302] The processor (3100) can determine whether any one or more second objects are located within the operating range of the charging robot, and can execute a safety-related action if any one or more second objects are located within the operating range of the charging robot.

[0303] A charging robot or its controller linked to an image-based object detection method according to one embodiment of the present invention may include a processor (3100) that receives and executes at least one command from a memory (3200).

[0304] The operations of the method according to an embodiment of the present invention can be implemented as a computer-readable program or code on a computer-readable recording medium. A computer-readable recording medium includes any type of recording device that stores information readable by a computer system. Furthermore, a computer-readable recording medium can be distributed across network-connected computer systems, allowing the computer-readable program or code to be stored and executed in a distributed manner.

[0305] Additionally, the computer-readable recording medium may include hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. The program instructions may include not only machine language codes produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc.

[0306] While some aspects of the present invention have been described in the context of a device, they may also represent a description of a corresponding method, wherein a block or device corresponds to a method step or a feature of a method step. Similarly, aspects described in the context of a method may also be described as a corresponding block or item or a feature of a corresponding device. Some or all of the method steps may be performed by (or using) a hardware device, such as, for example, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, at least one or more of the most important method steps may be performed by such a device.

[0307] In embodiments, a programmable logic device (e.g., a field-programmable gate array) may be used to perform some or all of the functions of the methods described herein. In embodiments, the field-programmable gate array may operate in conjunction with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by some hardware device.

[0308] Although the present invention has been described above with reference to preferred embodiments thereof, it will be understood by those skilled in the art that various modifications and changes may be made to the present invention without departing from the spirit and scope of the present invention as set forth in the claims below.

Claims

1. An image-based object detection method for charging electric vehicles, A step of detecting a first object corresponding to the electric vehicle from an image obtained by a camera capable of obtaining an image of the electric vehicle from the outside of the electric vehicle; A step of estimating the position and angle of the electric vehicle on the parking space based on the position information of the first object in the image; A step of detecting at least one second object other than the electric vehicle from the image; and A step of estimating a position of the at least one second object on the parking space based on a position of the at least one second object within the image; Image-based object detection method for charging an electric vehicle including a 2. In paragraph 1, In the step of estimating the position and angle of the above electric vehicle, Estimating the position and angle of the electric vehicle on a two-dimensional coordinate system with the position on the boundary line of the above parking space as a reference point. Image-based object detection method for electric vehicle charging.

3. In paragraph 1, The step of estimating the position and angle of the above electric vehicle is: A step of mapping the first object onto a two-dimensional map corresponding to the parking space based on the location information of the first object within the image; and A step of estimating the position and angle of the first object on the two-dimensional map as the position and angle of the electric vehicle; An image-based object detection method for charging an electric vehicle, comprising:

4. In paragraph 1, The step of detecting the first object corresponding to the electric vehicle includes a segmentation step of dividing the outline of the first object, The position and angle of the first object within the image are obtained based on the segmentation result of the first object. Image-based object detection method for electric vehicle charging.

5. In paragraph 1, A step of detecting a third object corresponding to the license plate of the electric vehicle from the image; Including more, The step of estimating the position and angle of the above electric vehicle is: A step of assisting in extracting location information of the first object within the image based on the location and angle of the third object within the image; An image-based object detection method for charging an electric vehicle, comprising:

6. In paragraph 1, A step of detecting a third object corresponding to the license plate of the electric vehicle from the image; Including more, The step of estimating the position and angle of the above electric vehicle is: A step of estimating an angle of the electric vehicle on the parking space based on an angle of the third object in the image; An image-based object detection method for charging an electric vehicle, comprising:

7. In paragraph 1, A step of identifying the license plate of the electric vehicle from the image; and A step of extracting vehicle information of the electric vehicle from a database using the license plate number of the electric vehicle; Including more, In the step of estimating the position and angle of the above electric vehicle, Using the vehicle information of the above electric vehicle, the position and angle of the electric vehicle are estimated in the parking space. Image-based object detection method for electric vehicle charging.

8. In paragraph 1, A step of estimating the position of the charging port of the electric vehicle based on the position and angle of the electric vehicle in the parking space; and A step of determining whether the location of the charging port of the electric vehicle is within the operating range of the charging robot; An image-based object detection method for charging an electric vehicle, further comprising:

9. In paragraph 8, A step of guiding the electric vehicle to be parked again if the location of the charging port of the electric vehicle is not within the operating range of the charging robot; and A step of guiding the charging robot to approach the electric vehicle when the location of the charging port of the electric vehicle is within the operating range of the charging robot; An image-based object detection method for charging an electric vehicle, further comprising:

10. In paragraph 1, wherein at least one of the second objects is a person or an animal, A step of determining whether at least one of the second objects is located within the operating range of the charging robot; and A step of executing a safety-related action when any one of the at least one second object is located within the operating range of the charging robot; An image-based object detection method for charging an electric vehicle, further comprising:

11. A controller of a charging robot associated with an electric vehicle power supply unit (EVSE) that includes a charging manipulator and transmits power to an electric vehicle, Memory that stores at least one instruction; A processor controlling the movement of the charging robot or the charging manipulator by loading and executing at least one command from the memory; and A communication interface that transmits and receives information necessary for the process of transmitting power to the electric vehicle to the EVSE; Including, The above processor, Obtaining the position and angle of the electric vehicle in the parking space estimated from the image obtained by a camera capable of obtaining an image of the electric vehicle from the outside of the electric vehicle, Obtaining the location of at least one object other than the electric vehicle detected from the image on the parking space, Determining whether charging is possible for the electric vehicle based on the position and angle of the electric vehicle on the parking space and the position of at least one object on the parking space. Controller of the charging robot.

12. In paragraph 11, The position and angle of the electric vehicle on the parking space, and the position of the at least one object on the parking space, It is expressed as a position and angle on a two-dimensional coordinate system with the position on the boundary line of the above parking space as a reference point. Controller of the charging robot.

13. In paragraph 11, The above processor If the location of the charging port of the electric vehicle, estimated based on the location and angle of the electric vehicle in the parking space, is not within the operating range of the charging robot, it is determined that charging of the electric vehicle is not possible. Controller of the charging robot.

14. In paragraph 11, The above processor If the position of at least one object on the parking space is within the operating range of the charging robot, it is determined that charging of the electric vehicle is not possible. Controller of the charging robot.

15. In paragraph 11, The above processor If the electric vehicle is capable of being charged, the movement of the charging robot and the charging manipulator is controlled to move the charging robot to the electric vehicle and connect the charging manipulator to the charging port of the electric vehicle. Controller of the charging robot.

16. A charging robot associated with an electric vehicle power supply unit (EVSE) that includes a charging manipulator and transmits power to an electric vehicle, Memory that stores at least one instruction; A processor controlling the movement of the charging robot or the charging manipulator by loading and executing at least one command from the memory; and A communication interface that transmits and receives information necessary for the process of transmitting power to the electric vehicle to the EVSE; Including, The above processor, Obtaining the position and angle of the electric vehicle in the parking space estimated from the image obtained by a camera capable of obtaining an image of the electric vehicle from the outside of the electric vehicle, Obtaining the location of at least one object other than the electric vehicle detected from the image on the parking space, Determining whether charging is possible for the electric vehicle based on the position and angle of the electric vehicle on the parking space and the position of at least one object on the parking space. Charging robot.

17. In paragraph 16, The position and angle of the electric vehicle on the parking space, and the position of the at least one object on the parking space, It is expressed as a position and angle on a two-dimensional coordinate system with the position on the boundary line of the above parking space as a reference point. Charging robot.

18. In paragraph 16, The above processor If the location of the charging port of the electric vehicle, estimated based on the location and angle of the electric vehicle in the parking space, is not within the operating range of the charging robot, it is determined that charging of the electric vehicle is not possible. Charging robot.

19. In Article 16, The above processor If the position of at least one object on the parking space is within the operating range of the charging robot, it is determined that charging of the electric vehicle is not possible. Charging robot.

20. In paragraph 11, The above processor If the electric vehicle is capable of being charged, the movement of the charging robot and the charging manipulator is controlled to move the charging robot to the electric vehicle and connect the charging manipulator to the charging port of the electric vehicle. Charging robot.

Images