System and method for charging an electric vehicle using an image capture device

Abstract

A plug connection system for autonomous charging of an electric vehicle (EV) is provided. The method includes adjusting a configuration setting of an image capture device; capturing an image using the image capture device, wherein a portion of the image includes an optically distinct object associated with a charging port of the EV, and wherein the image includes a plurality of pixels; applying a pixel value threshold associated with a pixel metric to the image to filter the plurality of pixels of the image into a first subset of pixels; determining a location of the optically distinct object associated with the charging port of the EV within the image based on filtering the plurality of pixels into the first subset of pixels; and providing information to manipulate a robotic arm to a physical location associated with the location of the optically distinct object within the image.

Description

[0001] Cross-references to related applications

[0002] This patent application claims priority to U.S. Patent Application No. 16 / 920,095, filed July 2, 2020, for all purposes, which is incorporated herein by reference. Technical Field

[0003] This disclosure relates to an automatic electric vehicle charging system. Background Technology

[0004] Automatic charging of electric vehicles (EVs) is considered a significant business opportunity in the EV industry because it enables a high level of safety during charging, which in turn allows for the use of high charging power ratings (fast charging), improves the operational efficiency of charging fleet vehicles or vehicles at public charging stations, and increases customer comfort.

[0005] A key technological challenge in EV charging systems is identifying the physical location of the EV charging port (e.g., EV inlet) on the EV and creating accurate registration for automatic charging. Traditionally, EV charging systems have attempted to remedy this by using only 2D image processing, 3D vision systems, active references placed on the EV inlet, or QR codes positioned on or near the EV inlet. However, all these methods face significant drawbacks. For example, 2D image processing requires substantial computational bandwidth, especially for classification and segmentation, which can increase system cost by requiring a Tensor Processing Unit (TPU) or similar processing equipment to handle significant processing overhead. 3D vision systems may be computationally efficient on the main processor or controller, but at the cost of a significant increase in cost on the camera / sensor side of the EV charging system, and even then, remain highly sensitive to environmental conditions. Active references mounted on the EV (e.g., devices with their own light source and / or power supply) are challenged by high vehicle costs and vehicle-specific installations. For example, active references may include their own light source providing light / illumination. QR codes are easily damaged or soiled, potentially rendering them invalid. Therefore, there is a technological need for a low-cost and computationally efficient system for identifying the location of EV inlets to enable automatic charging. Summary of the Invention

[0006] A first aspect of this disclosure provides a plug connection system for autonomously charging an electric vehicle (EV). The plug connection system includes a robotic arm capable of controlled extension and retraction, wherein the robotic arm is adapted to carry a charging plug located at its distal end, wherein the charging plug is configured to be controllably movable and capable of insertion into a charging port of the EV. The plug connection system further includes an image capture device configured to capture one or more images, wherein the image capture device is configured with new image capture characteristics different from default image capture characteristics. The plug connection system also includes a control system. The control system includes one or more controllers and a non-transient computer-readable medium storing processor-executable instructions thereon. The processor-executable instructions, when executed by the one or more controllers, cause the following to occur: capture an image using the image capture device, wherein a portion of the image includes an optically distinguishable object associated with the charging port of the EV, and wherein the image includes a plurality of pixels; apply a pixel value threshold associated with a pixel metric to the image to filter the plurality of pixels of the image into a first subset of pixels; determine the position of the optically distinguishable object associated with the charging port within the image based on filtering the plurality of pixels into the first subset of pixels; and provide information to manipulate a robotic arm to a physical position associated with the position of the optically distinguishable object within the image.

[0007] According to the implementation of the first aspect, the plug connection system further includes a light emitter configured to emit illumination for the image capture device. The control system uses the illumination from the light emitter to capture an image.

[0008] According to the implementation of the first aspect, the image capturing device includes a light emitter. The control system captures multiple images associated with an optically distinguishable object and the charging port of an EV. The multiple images include at least one image captured using illumination from the light emitter and at least one image captured without illumination from the light emitter.

[0009] According to the implementation of the first aspect, the control system applies the pixel value threshold by filtering out pixels from the plurality of images using the pixel value threshold to determine a second pixel subset, wherein each pixel from the second pixel subset has a corresponding pixel metric that satisfies the pixel value threshold, and wherein the information is provided based on the second pixel subset to manipulate the robotic arm.

[0010] According to the implementation of the first aspect, the image acquisition device is located on the charging plug. The plug connection system further includes a second image capture device configured to provide a second image to the control system, and wherein information is provided to manipulate the robot arm further based on the second image.

[0011] According to an embodiment of the first aspect, the control system captures multiple images based on a set frequency. The multiple images include a first image captured in a first time instance and a second image captured in a second time instance. The first and second time instances are associated with the set frequency. The control system applies a pixel value threshold by applying a pixel value threshold to the first and second images. The control system provides information to manipulate the robotic arm to a physical position by: providing a first instruction to manipulate the robotic arm to a first physical position based on applying the pixel value threshold to the first image; and providing a second instruction to manipulate the robotic arm from the first physical position to a second physical position based on applying the pixel value threshold to the second image.

[0012] According to the implementation of the first aspect, the pixel value threshold is a brightness value threshold, and the control system applies the pixel value threshold by comparing the brightness value threshold with the brightness values ​​associated with multiple pixels and filtering multiple pixels of the image into a first pixel subset.

[0013] According to the implementation of the first aspect, the pixel value threshold is a color characteristic threshold associated with the red, green and blue (RGB) values ​​of the plurality of pixels, and the control system applies the pixel value threshold by filtering the plurality of pixels of the image into a first pixel subset based on comparing the color characteristic threshold with the RGB values ​​of the plurality of pixels.

[0014] According to the implementation of the first aspect, the image capture device is a two-dimensional (2D) camera, and the plug connection system further includes an optical distinguishing object that can be adapted to the charging port of an EV.

[0015] According to the implementation of the first aspect, the control system analyzes a first subset of pixels based on the known geometric contours associated with the charging port of the EV to determine the position of the optically distinguishable object associated with the charging port.

[0016] According to the implementation of the first aspect, the plug connection system further includes an invisible light emitter configured to emit invisible light illumination for the image capture device, wherein the invisible light illumination is associated with a light frequency in the invisible light range.

[0017] A second aspect of this disclosure provides a method for autonomously charging an electric vehicle (EV) using a plug-in connection system. The method includes: adjusting the configuration settings of an image capture device from default image capture characteristics to new image capture characteristics via a control system; capturing an image using the image capture device via the control system, wherein a portion of the image includes an optically distinguishable object associated with a charging port of the EV, and wherein the image comprises a plurality of pixels; applying a pixel value threshold associated with a pixel metric to the image via the control system to filter the plurality of pixels of the image into a first subset of pixels; determining, via the control system and based on filtering the plurality of pixels into the first subset of pixels, the position of the optically distinguishable object associated with the charging port of the EV within the image; and providing information via the control system to manipulate a robotic arm to a physical position associated with the position of the optically distinguishable object within the image.

[0018] According to the second aspect of the implementation, the robotic arm is capable of controlled extension and retraction, the robotic arm is adapted to carry a charging plug located at the distal end of the robotic arm, and the charging plug is configured to be controllably movable and capable of being inserted into the charging port of the EV.

[0019] According to the second aspect of implementation, the control system captures an image using an image capture device by capturing an image through illumination from a light emitter.

[0020] According to the second aspect, the illumination is invisible illumination associated with light frequencies in the invisible range.

[0021] According to the second aspect of implementation, the control system captures images by capturing multiple images associated with the optically distinguishable object and the charging port of the EV. The multiple images include: at least one image captured using illumination from the light emitter; and at least one image captured without illumination from the light emitter.

[0022] According to an embodiment of the second aspect, the control system captures images by capturing a plurality of images based on a set frequency. The plurality of images includes a first image captured in a first time instance and a second image captured in a second time instance. The first and second time instances are associated with the set frequency. The control system applies a pixel value threshold by applying a pixel value threshold to the first and second images. The control system provides information to manipulate the robotic arm to a physical position by: providing a first instruction to manipulate the robotic arm to a first physical position based on applying the pixel value threshold to the first image; and providing a second instruction to manipulate the robotic arm from the first physical position to a second physical position based on applying the pixel value threshold to the second image.

[0023] According to the second aspect of implementation, the pixel value threshold is a brightness value threshold, and the control system applies the pixel value threshold by comparing the brightness value threshold with the brightness values ​​associated with multiple pixels and filtering multiple pixels of the image into a first pixel subset.

[0024] According to the second aspect of the implementation, the pixel value threshold is a color characteristic threshold associated with the red, green and blue (RGB) values ​​of the plurality of pixels, and the control system applies the pixel value threshold by filtering the plurality of pixels of the image into a first pixel subset based on comparing the color characteristic threshold with the RGB values ​​of the plurality of pixels.

[0025] A third aspect of this disclosure provides a computer-readable medium having processor-executable instructions stored thereon. When executed by one or more controllers, the processor-executable instructions cause: adjusting the configuration settings of an image capture device from default image capture characteristics to new image capture characteristics; capturing an image using the image capture device, wherein a portion of the image includes an optically distinguishable object associated with a charging port of an electric vehicle (EV), and wherein the image includes a plurality of pixels; applying a pixel value threshold associated with a pixel metric to the image to filter the plurality of pixels of the image into a first subset of pixels; determining, based on filtering the plurality of pixels into the first subset of pixels, the position of the optically distinguishable object associated with the charging port of the EV within the image; and providing information to manipulate a robotic arm to a physical position associated with the position of the optically distinguishable object within the image. Attached Figure Description

[0026] Embodiments of this disclosure will now be described in more detail with reference to the exemplary accompanying drawings. This disclosure is not limited to exemplary embodiments. All features described and / or illustrated herein may be used individually or in different combinations in embodiments of this disclosure. The features and advantages of various embodiments of this disclosure will become apparent from the following detailed description, which includes the following figures:

[0027] Figure 1 A simplified block diagram depicting a charging environment for an electric vehicle (EV) according to one or more embodiments of the present disclosure is shown;

[0028] Figure 2a An exemplary electric vehicle (EV) charging environment with an EV charging system in a retracted, docked state, according to one or more embodiments of the present disclosure, is shown.

[0029] Figure 2b It shows Figure 2a An exemplary electric vehicle (EV) charging environment in which the EV charging system is in a charging state;

[0030] Figure 3 This is a schematic diagram of an exemplary control system according to one or more embodiments of the present disclosure;

[0031] Figure 4 The process for operating an EV charging system to charge an EV according to one or more embodiments of the present disclosure is illustrated;

[0032] Figure 5a and 5b Images captured by an image capture device and then filtered by a control system according to one or more embodiments of the present disclosure are depicted;

[0033] Figure 6 A graphical representation of applying a threshold to a captured image to identify an EV charging port, according to one or more embodiments of the present disclosure, is depicted; and

[0034] Figure 7 An exemplary configuration of an EV charging port according to one or more embodiments of the present disclosure is shown. Detailed Implementation

[0035] This disclosure describes a charging system and environment for an autonomous electric vehicle (EV) that offers advantages over existing technologies. For example, this disclosure provides an optically distinguishing object (e.g., a retroreflective or reflective material, adhesive, label, mark, and / or structure) that enables a low-cost 2D image capture system (e.g., a vision system) to classify and segment EV charging ports (e.g., EV inlets) with minimal processing. For example, pixels in an image can be represented by multiple individual characteristics or measurements such as red, green, and blue (RGB) color and / or brightness values. In some cases, the EV charging system may use a light emitter to emit light (e.g., a flash from a camera), which, in combination with a 2D image capture device and an optically distinguishing object having the same or similar geometry as the pins or slots of the EV charging port / charging device, can segment, classify, and / or identify EV charging ports on a vehicle. For example, pixels of the optically distinguishing object may have a unique color return profile. By rejecting or filtering pixels that do not have the desired signature of the object, the EV charging system is able to identify the location of the EV charging port with minimal processing.

[0036] In other words, by combining active lighting (e.g., flash) with a retroreflective or reflective object having a known color response and applied / adapted to the vehicle, the EV charging system can use a low-cost 2D sensor with a low-cost computing platform to determine the location of the EV charging port. For example, when flash is combined with a retroreflective object, the automatic white balance inherent in low-cost 2D image capture devices is balanced to wash out and darken other objects and structures within the image. When color values ​​are averaged in each pixel to give an approximation of brightness, there is a clear difference between the pixels on the retroreflective / reflective object and the rest of the objects / structures in the image. Although there may be spurious pixels with high brightness values, the EV charging system may be able to easily distinguish these pixels from the pixels representing the retroreflective / reflective object because the geometry of the spurious pixels may represent small patches or thin linear structures, while the retroreflective / reflective object may have a known geometry in which a large number of high-brightness pixels are grouped together. This will be described in further detail below.

[0037] Exemplary aspects of the charging system and charging apparatus according to this disclosure are further illustrated below in conjunction with exemplary embodiments as shown in the accompanying drawings. These exemplary embodiments illustrate some implementations of this disclosure and are not intended to limit its scope.

[0038] In all the accompanying drawings, the same reference numerals denote similar but not necessarily identical elements. The drawings are not necessarily drawn to scale, and the dimensions of some parts may be enlarged to show the examples more clearly. Furthermore, the drawings provide examples and / or implementations consistent with the specification; however, the description is not limited to the examples and / or implementations provided in the drawings.

[0039] Where possible, any term expressed in the singular form herein is intended to also include the plural form, and vice versa, unless otherwise expressly stated. Furthermore, as used herein, the terms “a” and / or “an” should mean “one or more,” although the phrase “one or more” is also used herein. Additionally, when something is referred to herein as being “based on” other things, it may also be based on one or more other things. In other words, unless otherwise expressly indicated, “based on” as used herein means “at least partially based on” or “at least partially based on.”

[0040] Figure 1 A simplified block diagram depicting an electric vehicle (EV) charging environment 100 according to one or more embodiments of the present disclosure is shown.

[0041] See Figure 1The EV charging environment 100 includes an electric vehicle (EV) 110 and an electric vehicle (EV) charging system 120. Among other components, systems, and / or entities, such as an engine and / or transmission (not shown), the EV 110 includes an EV charging port 112, such as an EV inlet. An optically distinguishing object 114 (e.g., a retroreflective object and / or a reflective object) is applied to (e.g., adapted to) the EV charging port 112. The object 114 may be and / or include adhesives, labels, markings, and / or structures designed with retroreflective, reflective, and / or other light-reflective materials. For example, the object 114 may include a retroreflective or reflective coating on its surface, or a patterned pattern of different retroreflective or reflective materials molded into the object 114 in a specific pattern, such that it is magnified and / or otherwise makes the EV charging port 112 more easily identifiable. The object 114 will be described in more detail below.

[0042] EV charging system 120 (e.g., plug / charging plug connection system) includes an automated robotic charging device 124 and a power supply 122. Robotic charging device 124 includes a robotic arm 128 and a control system 132. Robotic arm 128 includes an image capture device (e.g., a camera with a light emitter) 130 and is adapted to engage and carry a charging plug 131. In one embodiment, the charging plug 131 may be integrated with the robotic arm 128, while in another embodiment, the robotic arm 128 may be detached from the charging plug 131 but can be engaged to carry the charging plug 131. Additionally, in some examples, EV charging system 120 may optionally include an image capture device 134. When present, image capture device 134 may resemble image capture device 130 and capture images of the EV charging port 112 with object 114.

[0043] Power supply 122 may be an EV charging unit (EVCU) that provides a high voltage for charging. Power supply 122 may or may not be used with robot charging device 124. EV charging system 120 is configured to automatically and / or conveniently charge EV110 without human intervention or interaction, thereby providing a safe and convenient charging experience for the user or operator of EV110.

[0044] Power source 122 receives AC power (e.g., from mains power) and converts and regulates the mains power to power suitable for charging EV 110 (e.g., a DC voltage with sufficient rated current for fast charging of the EV). Power source 122 is electrically coupled to robotic charging device 124 to provide charging power to charging device 124. Robotic charging device 124 can then supply charging power to EV 110 in an automatic and operator-free manner. Control system 132 of the charging device can communicate with power source 122 (e.g., to provide loose or tight control of charging).

[0045] The EV charging system 120, particularly the control system 132, can detect when the EV 110 is within a predetermined proximity of the robotic charging device 124 and determine the physical location of the EV charging port 112 on the EV 110. Based on the control system 132's determination that the EV 110 and its EV charging port 112 are within reach of the robotic charging device 124, the control system 132 uses images captured by the image capture device 130 to manipulate (e.g., guide, orient, move, displace, and / or actuate) the robotic arm 128 and position its charging plug 131 close to the charging port 112. In some examples, both the charging plug 131 and the image capture device 130 may be integrated or otherwise operably included in the end effector of the robotic arm 128. In other examples, instead of Figure 1 The single image capture device 130 shown or in addition to Figure 1 In addition to the single image capture device 130 shown, the image capture device 130 or additional image capture devices 130 may be located at or around the EV charging system 120, for example, at its base. The robot charging device 124 then configures the charging plug 131 to be inserted into the EV charging port 112 and charges the EV 110. The dashed line 136 represents the extension of the robot arm 128 to the EV charging port 112 of the EV 110. Once charging is complete (or otherwise stopped), the robot charging device 124 removes the charging plug 131 from the EV charging port 112 and retracts the robot arm 128.

[0046] In some cases, the control system 132 manipulates the robot arm 128 by changing the physical position and / or orientation of the charging plug 131 to align it for insertion into the EV charging port 112 of the EV 110. For example, the control system 132 may first move the robot arm 128 within range close to the EV charging port 112, and then orient the charging plug 131 so that it is aligned for insertion into the EV charging port 112. In other cases, the control system 132 may dynamically move the robot arm 128 and orient the charging plug 131 in any sequence (including simultaneously) to provide a smooth movement and insertion of the charging plug into the charging port 112.

[0047] The control system 132 uses one or more images, or consecutive or series of images, captured by the image capture device 130 to manipulate the robotic arm 128 to align it with the EV charging port 112. For example, the control system 132 captures an image including the EV charging port 112 with an object 114. The image comprises multiple pixels, and the control system 132 filters out pixels based on pixel metrics or characteristics to identify the EV charging port 112. For example, the pixels representing the object 114 applied to the charging port 112 may have distinguishing features (such as brightness values) exceeding a certain threshold. By using this threshold, the control system 132 identifies the position of the object 114 within the captured image and uses this position to manipulate the robotic arm 128 and / or the charging plug 131 toward the EV charging port 112. This will be described in further detail below.

[0048] Figure 2a An exemplary electric vehicle (EV) charging environment 200 according to one or more embodiments of the present disclosure is shown, in which the EV charging system is in a retracted, docked state, while Figure 2b It shows Figure 2a An exemplary electric vehicle (EV) charging environment 200 is provided, in which the EV charging system is in a charging state. However, it should be understood that... Figure 2a and 2b The EV charging environment 200 shown in the EV charging system is only... Figure 1 Examples of EV charging environments 100 are provided, and additional / alternative embodiments of EV charging environments, systems, and / or devices are contemplated within the scope of this disclosure. For example, in additional and / or alternative embodiments, the EV charging environment 100 may include different robotic arm 128 assemblies, such as scissor linkage mechanisms for the robotic arm 128, or different charging arrangements for vehicles, such as vehicles with charging plugs on the vehicle chassis.

[0049] Reference Figure 2a and Figure 2b The EV charging environment 200 includes a robot charging device 124 and a power supply 122. The robot charging device 124 includes a robot arm 128 having a charging plug 131. The EV charging environment 200 also includes an EV 110. The EV 110 includes an EV charging port 112 on which an object 114 is applied.

[0050] In addition, the EV charging environment 200 includes two image capture devices 130 and 134. For example, image capture devices 130 and 134 may be attached to or included within the EV charging system 120. For example, image capture device 134 may be mounted, physically positioned, operatively coupled to, and / or secured to the base of robot charging device 124. Image capture device 130 may be mounted, physically positioned, operatively coupled to, and / or secured to robot arm 128, and specifically, mounted, physically positioned, operatively coupled to, and / or secured to or near charging plug 131 carried by the end effector of the robot arm. The positions of image capture devices 130 and 134 are merely exemplary, and in other examples, image capture devices 130 and 134 are appropriately positioned in other locations within the EV charging environment 100 or 200 to have a line of sight to the mobile charging plug 131 and the charging port of EV 110. Image capture devices 130 and 134 can be used as visibility and / or positioning sensors forming part of control system 132 for automatically controlling robot charging device 124 to manipulate charging plug 131 and orient it into EV charging port 112 to charge EV 110.

[0051] Additionally and / or optionally, in other examples, the EV charging environment 100 includes a different number of capture devices. For example, image capture device 134 may be optional, and in other examples, the EV charging environment 100 may include only image capture device 130 on the charging plug 131.

[0052] The EV charging environment 200 also shows a working area 204 and a non-working area 202. The working area 204 of the charging device 100 is the physical space (i.e., the area of ​​functional access) in which the robotic charging device 124 can manipulate its robotic arm 128 and engage its charging plug 131 with the EV charging port 112 of the EV 110. The non-working area 202 of the robotic charging device 124 is the physical space near the robotic charging device 124, in which the charging device 124 is configured such that it will not attempt to charge the EV 110 located within this space. The non-working area 202 may include spaces where the robotic charging device 124 cannot effectively insert its charging plug 131 into the EV charging port 112, whether due to mechanical impact or inability to achieve proper orientation, and / or a buffer space providing a safe walking distance for human occupants. Therefore, when the EV 110 is within the non-working area 202, the robotic charging device 124 may not attempt to move or otherwise engage with the EV 110.

[0053] The working area 204 is defined by the mechanical and kinematic capabilities of the robot charging device 124 and the arrangement of the EV charging system 120 within its installation space. Therefore, the working area 204 of the robot charging device 124 is (at least partially) defined by the range of motion that can be imparted to the robot arm 128 and the reachability of its charging plug 131. In other words, in Figure 2a and 2b The physical space defined as the work area 204 includes the reachable position of the top of the robot arm 124 (and therefore its end effector) and the position where the charging plug 131 can be inserted. The robot charging device 124 is capable of at least rotating (yaw), pitching, and / or extending / retracting its robot arm 128. In other words, the kinematics of the robot charging device 124 includes rotational and linear motion.

[0054] In some cases, the robot charging device 124 can also controllably orient and move the charging plug 131 independently of the arm 128. This provides an extended range of motion and orientation, enabling the charging device 124 to precisely position the charging plug 131 for insertion into the charging port 112.

[0055] In some examples, the robotic arm 128 and the charging plug 131 are separate components and / or devices. For example, the EV charging system 120 may not include the charging plug 131. In other words, the robotic arm 128 may be configured to allow the existing charging plug 131 within the control, movement, and / or manipulation environment 100 to be inserted into the charging port 112. In some cases, the charging plug 131 is located at the distal end of the robotic arm 128. For example, the charging plug 131 may be located at the tip of the robotic arm 128. Additionally and / or alternatively, the charging plug 131 may be located above and / or below the tip of the robotic arm 128. Additionally and / or alternatively, the charging plug 131 may be located at a distance (e.g., a few inches) from the tip of the robotic arm 128.

[0056] In some variations, the EV charging system 120 may also include a user feedback / user interface (UI) for communicating with the operator of the EV 110 or with a user of the EV charging environment 100. For example, the EV charging system 120 may include a UI / sensing display. The display may provide information to the user (e.g., location or charging feedback). For example, the display may notify the user in real time about the location and charging status of the EV 110. Regarding location / position feedback, the display may signal the operator when positioning / stopping the EV 110 (similar to what a user might experience while driving past a car wash). For example, the display may warn the user of forward, backward, or right / left angular movement. Regarding charging status, the display may warn the user of the charging status, such as charging in progress, charged, charging percentage, or remaining charging time. Clearly, other user feedback information is also within the scope of this disclosure.

[0057] In some variations, the display may be mounted on the charging device. However, the display may alternatively and / or additionally be located externally to the robotic charging device 124. For example, the display may be located at a remote installation point near the EV110 (such as a control point for paying for or interacting with the EV charging system 120). A projector may also project projected images conveying user feedback information. The robotic charging device 124 may also wirelessly transmit user feedback information (e.g., via Bluetooth, WiFi, etc.) to user devices (e.g., the EV110's mobile phone or display).

[0058] Figure 3 This is a schematic diagram of an exemplary control system according to one or more embodiments of the present disclosure. It should be understood that... Figure 3 The control system shown is merely an example, and additional / alternative embodiments of the control system 132 from environment 100 are also within the scope of this disclosure.

[0059] The control system 132 includes a controller 302. The controller 302 is not limited to any particular hardware, and its configuration can be achieved through any kind of programming (e.g., embedded Linux) or hardware design, or a combination of both. For example, the controller 302 can be formed by a single processor, such as a general-purpose processor with corresponding software implementing the control operations. On the other hand, the controller 302 can be implemented by special-purpose hardware, such as an ASIC (Application-Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), a DSP (Digital Signal Processor), etc.

[0060] Controller 302 is in electrical communication with memory 312. Memory 312 may be and / or include computer-usable or computer-readable media, such as, but not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor computer-readable media. More specific examples of computer-readable media (e.g., a non-exhaustive list) may include: electrical connections having one or more wires, tangible media such as portable computer disks, hard disks, time-dependent access memory (RAM), ROM, erasable programmable read-only memory (EPROM or flash memory), optical disc read-only memory (CD_ROM), or other tangible optical or magnetic storage devices. Memory 312 may store corresponding software, such as computer-readable instructions (code, scripts, etc.). When executed by controller 302, the computer instructions cause controller 302 to control control system 132 to provide operation of EV charging system 120 as described herein.

[0061] Controller 302 is configured to provide and / or receive information from image capture devices 130 and / or 134. For example, image capture devices 130 and / or 134 may capture one or more images, or consecutive images, of object 114, and may provide these images to controller 302. Controller 302 may use these images (alone or in combination with other elements of control system 132) to determine the physical location, orientation, and / or state of charging plug 131, robot arm 128, EV 110, and / or EV charging port 112. For example, controller 302 may use these images to determine the physical location of EV charging port 112 and provide instructions or commands to manipulate robot arm 128 such that robot arm 128 can be inserted into EV charging port 112. Image capture devices 130 and / or 134 may be physically located at charging device 124 or elsewhere in charging system 120. Figure 2a In a preferred embodiment, the image capture device 130 is located at the charging plug 131 (e.g., at the top, left, right, or bottom of the charging plug 131).

[0062] In some examples, image capturing devices 130 and / or 134 are 2D cameras or other 2D image capturing devices, such as charge-coupled devices (CCDs) or other types of electronic image acquisition devices. In some cases, image capturing devices 130 and / or 134 include a light emitter. For example, the light emitter may emit light in the visible or invisible range, such as a flash or other form of illumination. In some cases, the emitted light belongs to the infrared (IR) spectrum and / or ultraviolet (UV) spectrum, but other types of radiation may be used. Image capturing devices 130 and / or 134 may use light / illumination generated by the backscattering or reflection of radiation emitted by the emitter to capture an image of object 114.

[0063] Additional sensors 306 may optionally be included within the control system 132. These additional sensors 306 may combine (or serve as a backup) information (e.g., images) provided by the image capture devices 130 and / or 134 to provide information to the control system 132. For example, these additional sensors 306 may include light sensors and / or flash camera sensor systems. In other words, instead of a light emitter that is part of the image capture devices 130 and / or 134, the EV charging system 120 may include additional sensors that provide light / illumination for images captured using the image capture devices 130 and / or 134.

[0064] Additionally and / or optionally, the additional sensor 306 may optionally include another image capture device (2D or 3D), a lidar sensor, an RFID sensor, an ultrasonic sensor, a capacitive sensor, an inductive sensor, a magnetic sensor, etc., to refine the trajectory of the robot end effector when the robot end effector guide plug mates with the vehicle charging port after visual recognition of the location of the vehicle charging port using visual or video information as described herein. Typically, any sensor capable of providing a signal that enables the control system 132 to manipulate the charging plug 131 for easy and safe insertion into the charging port 112 of the EV 110 may be included in the control system 132.

[0065] In some variations, image capture devices 130 and / or 134, along with an additional sensor 306, form a flash-based photographic sensing system. A flash-based photographic system includes at least an image capture device (e.g., device 130), a light emitter (e.g., for emitting a flash), and an object 114. In operation, the light emitter cycles the flash, or provides constant light by continuously emitting light, which reflects the light away from the object 114, and the resulting image captured by the image capture devices 130 and / or 134 separates the object 114.

[0066] The control system 132 is configured to drive the motors 318 of the charging device 124. As used herein, the motors 318 include AC motors, DC motors, geared motors, linear motors, actuators, or any other electrically controllable devices for implementing the kinematics of the charging device. Therefore, the control system 132 is configured to automatically and continuously determine the physical state of the charging system 120 and automatically control the various motors 318 of the charging device 124 to manipulate the robotic arm 128, which includes a charging plug 131, so that the charging plug 131 can be inserted into the charging port 112, and to retract the charging plug 131 after stopping or halting the charging of the EV 110.

[0067] The control system 132 may further include a motor control unit (MCU) 314 (also referred to herein as a motor controller), for example as part of the controller 302 or as a separate device. The MCU 314 uses feedback from the motor sensor 320 (e.g., an encoder) to control the motor driver 316 to provide real-time control of the motor 318. Therefore, the MCU 314 receives instructions for controlling the motor 318 (e.g., receiving motor / actuator control signals from the controller 302) and parses these instructions in conjunction with the feedback signals from the motor sensor 320 to provide control signals to the motor driver 316 for precise and real-time control of the motor 318 (e.g., sending motor / actuator drive signals). The motor driver 316 converts the control signals transmitted by the MCU 314 into drive signals for driving the motor 318 (e.g., sending separate operating signals to the motor / actuator). In another embodiment, the MCU 314 is integrated with circuitry that directly controls the motor 318.

[0068] The MCU314 can be included as part of the controller 302 or a standalone processing system (e.g., a microprocessor). Therefore, like the controller 302, the MCU314 is not limited to any particular hardware, and the configuration of the MCU can be achieved through any type of programming or hardware design or a combination of both.

[0069] The control system 132 may include input / output (I / O) terminals 310 for sending and receiving various input and output signals. For example, the control system 132 may send / receive external communications to users, servers (e.g., billing servers), power units, etc., via the I / O terminals 310. The control system 132 may also control a user feedback interface via the I / O terminals 310 (or other means).

[0070] Figure 4 A process 400 for operating an EV charging system 120 to charge an EV 110 according to one or more embodiments of the present disclosure is shown. Process 400 can be executed by a control system 132, particularly by... Figure 3 The controller 302 shown executes the procedure. However, it will be appreciated that any of the boxes below can be executed in any suitable order, and that process 400 can be executed in any suitable environment, including EV charging environments 100, 200 and / or additional / alternative environments, and by any suitable controller or processor.

[0071] At box 402, the control system 132 adjusts the configuration settings of the image capture device (e.g., image capture devices 130 and / or 134) from default image capture characteristics to new image capture characteristics. For example, the default and new image capture characteristics could be white balance or color balance characteristics. White balance or color balance is a global adjustment of the intensity of a color (e.g., white) so that the image is rendered appropriately. For example, pixels in an image captured or acquired by a camera or other image capture device are transformed from their acquired or original values ​​to new values ​​suitable for color reproduction or display (e.g., new red, green, and blue (RGB) values ​​and / or brightness values). White balance features or settings are used for this transformation.

[0072] The default white balance characteristics can be either the factory default white balance setting or the original equipment manufacturer (OEM) white balance setting. The adjusted new white balance characteristics can be any new white balance setting that is not the OEM setting, and enable the control system 132 to better identify objects 114 within an image captured using the image capture device 130. For example, a white balance characteristic can be used such that the average value (e.g., average brightness value) of all pixels in the image is equal to a specific hue of white or another color. Therefore, by adjusting the white balance characteristics of the image capture device 130 to the new characteristics via the control system 132, the average value of the total pixels can also be adjusted so that the pixels representing object 114 stand out better from other objects or structures within the image. In other words, the new white balance characteristics may cause other structures or objects within the image to be washed out or become darker; however, because object 114 has a retroreflective coating or surface, the pixels of object 114 are unaffected, resulting in a clearer distinction between the pixels of object 114 and the rest of the pixels in the image.

[0073] In some cases, and as described above, the image capture device 130 is a 2D camera, and the default image capture characteristics may be OEM image capture settings associated with the 2D camera (e.g., OEM white balance settings).

[0074] In some variations, memory 312 stores one or more white balance characteristics. Then, at block 402, controller 302 retrieves the white balance characteristics from memory 312 and provides instructions to image capture devices 130 and / or 134 to set the white balance characteristics of image capture devices 130 and / or 134 to the white balance characteristics retrieved from memory 312. Additionally and / or alternatively, in some examples and even with the new white balance characteristics, control system 132 may have difficulty identifying object 114 from the captured image. In such examples, control system 132 may use one or more additional white balance characteristics to capture an image with object 114 (e.g., by adjusting the white balance characteristics of image capture devices 130 and / or 134 from the new white balance characteristics to the additional white balance characteristics). For example, controller 302 retrieves additional (e.g., second) white balance characteristics from memory 312 and provides instructions to image capture devices 130 and / or 134 to set the white balance characteristics of image capture devices 130 and / or 134 to the second white balance characteristics.

[0075] At block 404, control system 132 uses image capture devices (e.g., 130 and / or 134) to capture an image. The captured image comprises multiple pixels, and a portion of the captured image includes object 114 associated with EV charging port 112. The image capture device may use a new white balance feature from block 402 to capture the image.

[0076] For example, an operator of EV110 may seek to charge their vehicle using EV charging system 120. EV charging system 110 may capture one or more images of EV110 (e.g., EV charging port 112) and use these images to manipulate robotic arm 128 toward EV charging port 112. To better distinguish EV charging port 112 within the images, an object 114, such as a retroreflective adhesive, label, or material, may be applied to (e.g., adapted to) EV charging port 112. For example, an operator of EV110 may apply object 114 to EV charging port 112.

[0077] As described above, object 114 is and / or includes adhesives, labels, signs, and / or structures that may be designed with retroreflective, reflective, and / or other light-reflective materials. In some cases, object 114 is used to amplify the flash or light response (e.g., light / illumination response from a light emitter) within a local area of ​​the captured image (e.g., the area representing the EV charging port 112). Additionally and / or alternatively, certain patterns and / or colors may be applied to object 114 to enhance this response, particularly when it can amplify the response of the image capture characteristics set by control system 132 in block 402. As will be described below, by using object 114 on the EV charging port 112, control system 132 is able to use image capture features of the image capture device (e.g., white balance features) to darken many pixels other than object 114, which will facilitate the segmentation process by using one or more thresholds.

[0078] In some variations, object 114 may include and / or be a light transmissor (e.g., optical fiber) that collects light from a light emitter and re-emits the collected light at a localized point of focus. The light transmissor may be a flexible optical object (e.g., flexible optical fiber) that allows light to enter at one end. The light travels along a path that may not be straight and exits at the other end. In some cases, object 114 may include a plastic molding and / or a plastic molded part (e.g., molded into plastic using multi-material injection molding).

[0079] In some examples, the control system 132 uses a light emitter to capture images. For example, the light emitter may emit light or illumination (e.g., a flash) used by image capture devices 130 and / or 134 to capture images. As described above, image capture devices 130 / 134 may include a light emitter, and / or additional sensor 306 may include a light emitter. The control system 132 may provide instructions to the light emitter to provide light / illumination for capturing images. In some variations, the light or illumination emitted by the light emitter may be in the invisible spectrum (e.g., infrared (IR) light and / or ultraviolet (UV) light). For example, the light emitter may be an invisible light emitter configured to emit invisible light illumination (e.g., illumination with light frequencies in a range invisible to humans, such as IR or UV light).

[0080] In some cases, the control system 132 captures more than one image. For example, the control system 132 may use image capture device 130 to capture multiple images. Additionally and / or alternatively, the control system 132 may use another image capture device (e.g., image capture device 134) to capture one or more additional images. The control system 132 may use multiple images to identify the physical location of object 114 and manipulate robot arm 128 and / or charging plug 131.

[0081] Figure 5a Image 500 is shown, captured by an image capture device such as image capture device 130. This image simulates the presence of EV charging port 504 in an environment such as a residential garage, where many other objects are within the field of view of image capture devices 130 and / or 134. As shown, image 500 illustrates the optically distinguishable object 502 and EV charging port 504. Image 500 also shows additional objects, structures, and other background items. A particular object shown in image 500, background object 506, also has a highly reflective structure and characteristics. Additionally, image 500 includes structure 508, which is darkened due to the white balance characteristics adjusted as described above. Object 506 may have a structure and characteristics similar to the headlights or taillights of a vehicle such as EV110.

[0082] Image 500 will be used to describe the classification, segmentation, and / or localization processes described below. However, it should be noted that image 500 is merely an example, and image capturing devices 130 and / or 134 may capture additional / alternate background objects, structures, and / or items (e.g., Figure 1 , Figure 2a and / or Figure 2b Other images of the actual EV110.

[0083] At box 406, control system 132 applies a pixel value threshold associated with a pixel metric to the image to filter multiple pixels of the image into a first subset of pixels. For example, after capturing the image at box 404, control system 132 uses a threshold to filter out pixels in the image, such that control system 132 analyzes a portion or subset of pixels within the image rather than every single pixel in the image. By using a threshold, the computational processing required for the classification, segmentation, and / or localization of the EV charging port 112 within the image is drastically reduced due to the analysis of a smaller number of pixels. Furthermore, by using object 114 applied to the EV charging port 112, adjusting the image capture characteristics of the image capture device at box 402, and / or using light / illumination (e.g., flash), the threshold can be set more aggressively to filter out even more pixels from the image, which can further reduce computational processing.

[0084] The control system 132 can set a pixel value threshold and / or retrieve a pixel value threshold from memory 312. Pixel value thresholds and pixel metrics represent characteristics or attributes associated with a pixel. For example, a pixel metric could be a color characteristic representing the color of the associated pixel, such as red, green, and blue (RGB) values. Additionally and / or alternatively, a pixel metric could be a luminance or light intensity characteristic, such as a luminance value. Similarly, a pixel value threshold could also be a color characteristic (e.g., RGB values) and / or a light intensity characteristic (e.g., luminance value).

[0085] In some examples, memory 312 may include one or more set or predefined pixel value thresholds, and control system 132 may retrieve / use predefined thresholds from memory 312. In other examples, control system 132 may determine and / or set thresholds based on one or more captured images and / or pixels within images. For example, control system 132 may set a threshold such that a certain percentage of pixels (e.g., four percent) are retained after the threshold is applied.

[0086] The control system 132 filters pixels from an image based on a pixel value threshold. For example, the pixel value threshold could be a specific brightness value. The control system 132 can filter out pixels from the image with brightness values ​​below the specific pixel value threshold, such that only a subset of pixels in the image with brightness values ​​above the threshold are retained (e.g., a first subset of pixels).

[0087] Additionally and / or optionally, the pixel value threshold may be associated with the color characteristics of a pixel, such as the RGB values ​​of the pixel. The control system 132 may filter out pixels from the image that are below the color characteristic value to determine a first subset of pixels. In other words, each RGB value can be between 0 and 255. The color characteristic value may be a combination of three values ​​(e.g., the control system 132 combines each RGB value to determine the color characteristic). The control system 132 filters out pixels based on the combined values ​​of RGB values ​​below the pixel value threshold, such that only pixels with combined RGB values ​​above the pixel value threshold are retained. Additionally and / or optionally, the control system 132 may filter out pixels from the image based on individual RGB values ​​(e.g., the control system 132 filters out pixels with red values ​​below a threshold).

[0088] Figure 6A graphical representation 600 depicts the application of a threshold to a captured image to identify an EV charging port, and will be used to describe box 406 in more detail. For example, x-axis 602 represents pixel metrics (e.g., brightness and / or color characteristics), and y-axis 604 represents the pixel count of the image captured at box 402. Line 606 represents a pixel value threshold (e.g., brightness value or color characteristic value). Control system 132 can filter out pixels below (e.g., to the left) the threshold 606, such that only pixels above the threshold (e.g., to the right and also referred to as a first subset of pixels) are retained. As shown, by using the threshold, control system 132 can analyze only a portion of the pixels from the image, which can reduce the computational processing required to identify the EV charging port 112.

[0089] At block 408, control system 132 determines the location of object 114 associated with EV charging port 112 within the image based on filtering multiple pixels into a first subset of pixels. For example, after filtering multiple pixels at block 406 to determine the first subset of pixels, control system 132 determines or identifies the location of object 114 within the captured image. For example, control system 132 may perform one or more image processing algorithms (e.g., procedures, methods, and / or techniques) on the first subset of pixels (e.g., the remaining pixels after applying a threshold) to determine the location of object 114 and / or EV charging port 112.

[0090] In other words, instead of performing image processing algorithms such as edge detection on all pixels in the image, the control system 132 performs image processing algorithms only on a subset of pixels in the image (e.g., a first subset of pixels), which reduces the computational bandwidth required to identify object 114 and / or EV charging port 112. The image processing algorithm can be and / or includes any image processing algorithm that enables the control system 132 to identify object 114 in the image, EV charging port 112 in the image, the location of object 114 in the image, and / or the location of EV charging port 112 in the image.

[0091] In some examples, boxes 406 and / or 408 may be part of and / or attached to a control system 132 that performs one or more image classification, image segmentation, image localization, and / or image registration processes. Image classification is a method of classifying objects or structures within an image using contextual information, such as relationships between nearby pixels. Image segmentation is the process of dividing an image into segments (e.g., sets of pixels) to simplify and / or alter the presentation of the image into a more meaningful and / or easier-to-analyze set of objects. Image localization is the process of determining the location of an object / structure (e.g., object 114) within an image. Image registration is the process of transforming a dataset into another coordinate system (e.g., transforming the location of object 114 / EV charging port 112 within an image into a coordinate system that the robot charging device 124 can use to manipulate the robot arm 128 / charging plug 131 to charge EV 110).

[0092] For example, the control system 132 determines the position of the object 114 within the captured image by analyzing the remaining pixels (e.g., a first subset of pixels) using the known geometric contours (e.g., geometry / geometric shape) of the object 114. Figure 7 An exemplary configuration of an EV charging port 112 according to one or more embodiments of the present disclosure is shown. For example, object 114 may be an adhesive or label adapted to (e.g., placed on its surface) the EV charging port 112. The EV charging port 112 / object 114 includes certain geometries, such as two large circles 702 and four small circles 704. The control system 132 can use the known geometric contours of the EV charging port 112 (e.g., the two large circles 702 and the four small circles) to identify pixels representing the EV charging port 112 within a captured image.

[0093] Figure 7 The EV charging port 112 shown is merely an example; in other cases, EV charging port 112 may be another type of EV charging port that enables EV charging system 120 to charge EV 110. For example, in some cases, EV charging port 112 may be a charging port combining charging system type 1 or 2 (CCS1 or CCS2). Control system 132 may use the known geometric contours of the CCS1 or CCS2 charging port to determine the position of object 114 within the captured image.

[0094] Reference Figure 5a Image 500 may include EV charging port 502, optically distinguishable object 502, and another background object 506 (e.g., structurally similar to the headlights or taillights of EV110). By filtering pixels based on a threshold, control system 132 can remove most pixels, and the remaining pixels (e.g., a first subset of pixels) may include objects 502 and 506. For example, Figure 5bThe remaining pixels after threshold-based pixel filtering are depicted (e.g., pixels representing objects 502 and 506). The control system 132 can then determine the position of object 502 (e.g., an object applied to an EV charging port) by analyzing a first subset of pixels using the known contour of object 502. In other words, the control system 132 can determine that object 506 is background 506 because it does not have the same geometry as the charging port. The control system 132 can determine that object 502 is an object applied to the charging port because the known geometry of the charging port matches the geometry of object 502.

[0095] In other words, by using a threshold applied to the optical distinguishing objects and / or flashes at the charging port, the control system 132 may need to perform analysis only on a subset of pixels from the image (e.g., pixels representing objects 502 and 506) instead of the entire image, which would reduce the required computational bandwidth.

[0096] At block 410, control system 132 provides information (e.g., one or more signals, instructions, and / or commands) to manipulate the robotic arm 128 of EV charging system 120 to a physical location associated with the position of object 114 within the captured image. For example, based on the position of object 114 determined from block 408, control system 132 may manipulate (e.g., move / orient) robotic arm 128 and / or charging plug 131 to a physical location.

[0097] For example, based on the determined position of object 114, control system 132 sends control signals to manipulate robot arm 128 / charging plug 131 so that it can be inserted into charging port 112. In other words, the control signals include instructions to operate motor 318 to properly move and position charging plug 131 to charge EV 110. The control signals may also include instructions to insert charging plug 131 into charging port 112 and / or retract robot arm 128 / charging plug 131 after charging EV 110.

[0098] More specifically, the control system 132 determines motor control signals configured (when executed) to controllably operate the motor 318 to position and orient the arm 128 within the working area to a location accessible to the charging port 112 by the charging plug 131. The control system 132 then sends these motor control signals to perform the specified movement. The motor 318 may include multiple motors configured collectively to (ultimately) position the distal end of the charging plug 131 within the reach of the charging plug 131.

[0099] The control system 132 can also determine actuator control signals configured to adjust the orientation and / or position of the charging plug 131 to align it with the charging port 112. The control system 132 then sends these actuator control signals to perform a specified motion. The motor 318 includes actuators specifically designed for fine-tuning the orientation / position of the charging plug 131, and the actuator control signals are intended to control these actuators.

[0100] The control system 132 further determines additional motor control signals configured to operate the motor 318 to insert the charging plug 131 into the charging port 112. The control system 132 then sends these additional motor control signals to perform a specified movement. Subsequently, for example, after charging is complete, the control system 132 determines and sends additional motor control signals that, when executed, cause the motor 318 to retract the robot arm 128 to its fully retracted state.

[0101] MCU314 receives motor / actuator control signals and may also receive feedback signals. The feedback signals are provided by motor sensors 320 that detect the state / position of various motors / actuators in the charging device 124. Based on the feedback signals and the motor / actuator signals, MCU314 determines motor driver signals and actuator driver signals. Thus, the control signals can be high-level instructions for the operation (or resulting position) of the elements of the charging device 124, and MCU314 can interpret those high-level instructions (as notified by the feedback signals) to provide low-level control signals for individually driving the motors / actuators. MCU314 sends the motor driver signals and actuator driver signals directly to at least one motor driver 316. In some instances, MCU314 includes circuitry capable of operating appropriate voltages and currents for driving actuators coupled to its processing system (e.g., microcontroller, FPGA, ASIC, etc.), and thus can send the motor driver signals and actuator driver signals directly to the motor 318.

[0102] In some variations and referring to block 404, control system 132 receives multiple images from a single image capture device (e.g., device 130) and / or from multiple different image capture devices (e.g., devices 130 and 134). Control system 132 can use the multiple images to manipulate robot arm 128 and charge EV 110. For example, referring to blocks 406 and 408, control system 132 applies a pixel value threshold to each of the captured images and determines the corresponding position of object 114 within each image. In other words, control system 132 filters out pixels from multiple images and determines the remaining pixels (e.g., a second subset of pixels) after the filtering process. Control system 132 can determine the position of object 114 within these remaining pixels. Referring to block 410, control system 132 uses these determined positions to manipulate robot arm 128 and then charge EV 110.

[0103] In some cases, multiple images can originate from a single image capture device 130. For example, the control system 132 can capture each image at different times with different light / illumination (e.g., flash) and / or in the absence of light / illumination from a light emitter. The control system 132 can then subtract the thresholded images from each other to eliminate additional pixels that have been processed. This leverages simple global mathematics to refine the possible image registration before processor-enhanced applications must study and identify individual structures in the images.

[0104] In some examples, multiple images can come from multiple image capture devices. For example, the image capture devices can be located in different locations (e.g., device 130 can be at charging plug 131, and device 134 can be at the base of EV charging system 120, such as...). Figure 2a (As shown). The control system 132 can use images captured at different locations to manipulate the robot arm 128.

[0105] In some variations, the control system 132 may capture one or more images at different timing intervals or by using one or more frequencies (e.g., 10 Hz). The control system 132 can use these captured images at different timing intervals / frequencies to manipulate the robot arm 128. For example, the control system 132 can use these images to continuously determine the position / orientation / state of the robot arm 128 / charging plug 131 relative to the EV charging port 112. The control system 132 can then manipulate the robot arm 128 / charging plug 131 closer to the EV charging port 112 at each timing interval until the charging plug 131 can be inserted into / has been inserted into the EV charging port 112.

[0106] While embodiments of the invention have been detailed and described in the accompanying drawings and the foregoing description, such description should be considered illustrative or exemplary rather than restrictive. It should be understood that changes and modifications can be made by those skilled in the art within the scope of the appended claims. In particular, the invention covers additional embodiments having any combination of features from the different embodiments described above and below. For example, various embodiments of the motion subsystem, control subsystem, electrical subsystem, installation subsystem, and user interface subsystem can be used interchangeably without departing from the scope of the invention. Furthermore, the description of the invention herein relates to embodiments of the invention, and not necessarily all embodiments.

[0107] The terms used in the claims should be interpreted as having the broadest reasonable interpretation consistent with the foregoing description. For example, the use of the articles “a” or “the” when introducing an element should not be interpreted as excluding multiple elements. Similarly, the expression “or” should be interpreted as inclusive, such that the expression “A or B” does not exclude “A and B” unless it is clearly apparent from the context or the preceding description that it refers only to one of A and B. Furthermore, the expression “at least one of A, B, and C” should be interpreted as one or more of a set of elements consisting of A, B, and C, and should not be interpreted as requiring at least one of each of the listed elements A, B, and C, regardless of whether A, B, and C are as categories or otherwise related. In addition, the expressions “A, B, and / or C” or “at least one of A, B, or C” should be interpreted as including any singular entity from the listed elements, such as A, any subset from the listed elements, such as A and B, or the entire list of elements A, B, and C.

Claims

1. A plug connection system for autonomously charging an electric vehicle, the plug connection system comprising: A robotic arm capable of controlled extension and retraction, wherein the robotic arm is adapted to carry a charging plug located at the distal end of the robotic arm, wherein the charging plug is configured to be controllably movable and capable of being inserted into the charging port of the electric vehicle. An image capture device is configured to capture one or more images, wherein the image capture device is set to a new image capture characteristic that is different from the default image capture characteristic, wherein the default image capture characteristic is a white balance characteristic, and the new image capture characteristic is a white balance characteristic, wherein a default white balance characteristic is set, and the new white balance characteristic is different from the default white balance characteristic. as well as Control system, including: One or more controllers; and A non-transitory computer-readable medium having processor-executable instructions stored thereon, wherein the processor-executable instructions, when executed by the one or more controllers, cause: The image is captured using the image capture device, wherein a portion of the image includes an optically distinguishable object associated with the charging port of the electric vehicle, wherein the image includes a plurality of pixels, and wherein the optically distinguishable object is a retroreflective or reflective object; A pixel value threshold associated with a pixel metric is applied to the image to filter the plurality of pixels of the image into a first subset of pixels; The location of the optically distinguishing object associated with the charging port within the image is determined by filtering the plurality of pixels into a first subset of pixels; and Information is provided to manipulate the robotic arm to a physical location associated with the position of the optically distinguishable object within the image.

2. The plug connection system according to claim 1, further comprising: A light emitter configured to emit illumination for the image capturing device, and Capturing the image using the image capture device includes capturing the image using the illumination from the light emitter.

3. The plug connection system according to claim 2, wherein the image capturing device includes the light emitter, and Capturing the images includes capturing multiple images associated with the optically distinguishable object and the charging port of the electric vehicle, wherein the multiple images include: At least one image captured using the illumination from the light emitter; as well as At least one image captured without using the illumination from the light emitter.

4. The plug connection system according to claim 3, wherein applying the pixel value threshold includes: Pixels are filtered out from the plurality of images using the pixel value threshold to determine a second subset of pixels, wherein each pixel from the second subset of pixels has a corresponding pixel metric that satisfies the pixel value threshold, and The information is provided based on the second subset of pixels to manipulate the robotic arm.

5. The plug connection system according to claim 1, wherein the image capture device is located on the charging plug, and wherein the plug connection system further comprises: A second image capture device, configured to provide a second image to the control system, and The information provided to manipulate the robotic arm is further based on the second image.

6. The plug connection system of claim 1, wherein capturing the image comprises capturing a plurality of images based on a set frequency, wherein the plurality of images includes a first image captured in a first time instance and a second image captured in a second time instance, wherein the first time instance and the second time instance are associated with the set frequency. The application of the pixel value threshold includes applying the pixel value threshold to the first image and the second image, and Providing information to manipulate the robotic arm to the physical location includes: A first instruction is provided to manipulate the robotic arm to a first physical position based on applying the pixel value threshold to the first image; as well as A second instruction is provided based on applying the pixel value threshold to the second image to manipulate the robotic arm from the first physical position to the second physical position.

7. The plug connection system of claim 1, wherein the pixel value threshold is a brightness value threshold, and The application of the pixel value threshold includes: The plurality of pixels of the image are filtered into the first subset of pixels by comparing the brightness value threshold with the brightness values ​​associated with the plurality of pixels.

8. The plug connection system of claim 1, wherein the pixel value threshold is a color characteristic threshold associated with the red, green, and blue values ​​of the plurality of pixels, and The application of the pixel value threshold includes filtering the plurality of pixels of the image into the first pixel subset by comparing the color characteristic threshold with the red, green and blue values ​​of the plurality of pixels.

9. The plug connection system according to claim 1, wherein the image capturing device is a 2D camera, and wherein the plug connection system further comprises: The optically distinguishable object can be adapted to the charging port of the electric vehicle.

10. The plug connection system of claim 1, wherein determining the position of the optically distinguishing object associated with the charging port is based on analyzing the first subset of pixels using a known geometric profile associated with the charging port of the electric vehicle.

11. The plug connection system according to claim 1, further comprising: An invisible light emitter is configured to emit invisible light illumination for the image capturing device, wherein the invisible light illumination is associated with a light frequency in the invisible light range.

12. A method for a plug connection system for autonomous charging of an electric vehicle, comprising: The configuration settings of the image capture device are adjusted from the default image capture characteristics to new image capture characteristics by the control system. The default image capture characteristics are white balance characteristics, and the new image capture characteristics are white balance characteristics. The default white balance characteristics are set, and the new white balance characteristics are different from the default white balance characteristics. The control system uses the image capture device to capture an image, a portion of which includes an optically distinguishable object associated with the charging port of the electric vehicle, the image comprising multiple pixels, and the optically distinguishable object being a retroreflective or reflective object; The control system applies a pixel value threshold associated with a pixel metric to the image to filter the plurality of pixels of the image into a first subset of pixels; The position of the optically distinguishing object associated with the charging port of the electric vehicle within the image is determined by the control system and based on filtering the plurality of pixels into a first subset of pixels; as well as The control system provides information to manipulate the robotic arm to a physical position associated with the position of the optically distinguishable object within the image.

13. The method of claim 12, wherein the robotic arm is controllably extendable and retractable, wherein the robotic arm is adapted to carry a charging plug located at a distal end of the robotic arm, wherein the charging plug is configured to be controllably movable and capable of being inserted into the charging port of the electric vehicle.

14. The method of claim 12, wherein capturing the image using the image capturing device comprises capturing the image using illumination from a light emitter.

15. The method of claim 14, wherein the illumination is invisible illumination associated with light frequencies within the invisible range.

16. The method of claim 14, wherein capturing the image comprises capturing a plurality of images associated with the optically distinguishing object and the charging port of the electric vehicle, wherein the plurality of images includes: At least one image captured using illumination from the light emitter; as well as At least one image captured without using illumination from the light emitter.

17. The method of claim 12, wherein capturing the image comprises capturing a plurality of images based on a set frequency, wherein the plurality of images includes a first image captured in a first time instance and a second image captured in a second time instance, wherein the first time instance and the second time instance are associated with the set frequency. The application of the pixel value threshold includes applying the pixel value threshold to the first image and the second image, and Providing information to manipulate the robotic arm to the physical location includes: A first instruction is provided to manipulate the robotic arm to a first physical position based on applying the pixel value threshold to the first image; as well as A second instruction is provided based on applying the pixel value threshold to the second image to manipulate the robotic arm from the first physical position to the second physical position.

18. The method of claim 12, wherein the pixel value threshold is a brightness value threshold, and The application of the pixel value threshold includes: The plurality of pixels of the image are filtered into the first subset of pixels by comparing the brightness value threshold with the brightness values ​​associated with the plurality of pixels.

19. The method of claim 12, wherein the pixel value threshold is a color characteristic threshold associated with the red, green, and blue values ​​of the plurality of pixels, and The application of the pixel value threshold includes: The plurality of pixels in the image are filtered into the first pixel subset by comparing the color characteristic threshold with the red, green and blue values ​​of the plurality of pixels.

20. A non-transitory computer-readable medium having stored thereon processor-executable instructions, wherein the processor-executable instructions, when executed by one or more controllers, cause to perform the method of any one of claims 12 to 19.

Images