Method for determining the position and / or orientation of a socket in an electric vehicle for the purpose of automatic plugging-in of a connector - Patents.com

Abstract

The present invention relates to a method for determining the position and / or orientation of a socket in an electric vehicle for the purpose of automatic plugging-in of a connector, the method comprising: providing a digital camera image of an area in which the socket is likely to be present using a camera having configurable parameters; recognizing the position of a reference feature in the digital camera image using a feature recognition algorithm; and determining the position and / or orientation of the socket based on the reference feature in the digital camera image, characterized in that at least one camera parameter for providing the digital camera image is set such that the reference feature is recognizable to the feature recognition algorithm, the reference feature being part of a connection function of the socket.

Description

Detailed Description of the Invention

[0001] The present invention relates to a method for determining the position and / or orientation of an electric vehicle socket for the purpose of automatic plugging of a connector. Automatic plugging of an electric charging connector into a vehicle socket has become a new goal for owners of large fleets for the past few years. Many electric vehicles have a vehicle socket for manual plugging, such as a vehicle socket according to IEC 62196. The connector and vehicle socket combination typically has a close-fitting geometry. Automatic plugging of the connector therefore requires a certain degree of accuracy in the positioning and orientation of the connector. The accuracy with which the vehicle socket is positioned over various charging instances by a typical vehicle, such as a passenger car, in an automatic charging station is insufficient to facilitate automatic plugging without determining the position and orientation of the vehicle's socket. Factors that contribute to the lack of accuracy are, for example, the vehicle's kinematic accuracy when parking, and the variability in the height and orientation of the socket over multiple connection cycles of a vehicle over time, or between vehicles in general, due to, for example, suspension wear and settings.

[0002] The connector or connectors in the charging station are held by an actuation mechanism which is able to move the connector towards the socket of the stationary vehicle and adapt the position and orientation of the connector so that it can be plugged in with the help of said mechanism. To be able to do so, the position and orientation of the socket must be determined before plugging in. Obviously, this should also be done automatically. This can be achieved in different ways using different means.

[0003] However, when only the parts of the socket that contribute to the physical connection function, such as the conductive pins and the non-conductive body, are considered, it is difficult to find the location and orientation of the socket. The non-conductive parts of the socket usually consist of black features surrounded by black edges, which is a special complication that at any given time or moment, it may be in a poorly lit environment or situation, and at another position or moment, in a very bright environment. These factors make the determination of the location and orientation of the socket through feature recognition particularly complicated.

[0004] So far, some solutions have been proposed in the art. Korean Patent Publication KR20190113697A discloses a first light control unit including a first lamp and a first illuminance sensor, a second light controller installed on the opposite side of the first light controller with respect to the charging connector and including a second lamp and a second illuminance sensor, and a camera operable to photograph the first lamp and the second lamp. International Patent Application WO2021061354A1 discloses a system in which a charging head is connected to a charging inlet of a vehicle to provide an electric charge to recharge the battery of the electric vehicle. The charging head is attached to a connecting device, which moves the charging head to the charging inlet. A plurality of photodetectors are provided on the charging head to sense light emitted from the vehicle. German patent publication DE102011080456A1 discloses an arrangement for assisting in establishing plug connections of a computer terminal, for example for a blind user, with a detection unit for detecting the correct setting or insertion of the plug into the component and an output unit for outputting information to the user. US patent application publication no. 2013293366A1 discloses a communication unit which periodically transmits a request signal towards a defined range. When a transmitter is present within a range capable of receiving the request signal, the transmitter responds by sending out identification information.

[0005] Pan et al., XP006092988, describes an automatic recognition and localization system for electric vehicle charging ports in complex environments. The system obtains the pose of the charging port through image processing and performs an insertion motion in combination with a robotic arm to complete the charging gun insertion of the automatic charging link. Pan et al. describe the use of five circular features that mimic the five metal cores inside the socket. Pan et al. describe the use of a highly reflective material that marks the five circular features, whereby the recognition and localization of the charging port is learned based on those markers. The image is later processed in terms of brightness and noise reduction to better recognize the markers. The five circular features are either outside the socket geometry or are added to mimic the five metal cores inside the socket, but are not considered to have a connection function.

[0006] All the above mentioned systems have the drawback that they have to add means to the vehicle to facilitate the detection and estimation of the socket's position and orientation, such as introducing markers such as lights. This makes them unsuitable for vehicles that are not ready or optimized for these solutions. Furthermore, systems that try to determine the socket's position and orientation and use a camera to record its view encounter the difficulty that the socket consists of a black front or plane and a hole that is also black, and therefore very little information can be obtained directly from the raw data. Many solutions aimed at solving this problem try to improve the image by illuminating the socket with a fixed or dynamic light source. These solutions treat the dark socket as an object that needs illumination to make the camera work conventionally with conventional settings. This helps in some cases, but does not allow for a lot of variability.

[0007] It is the object of the present invention to propose a method and a system for determining the position and / or orientation of an electric vehicle socket for the purpose of automatic plugging-in of a charging station connector, which obviates the drawbacks of the prior art or at least constitutes a useful alternative to the prior art.

[0008] The present invention accordingly proposes a method for determining the position and / or orientation of a socket in an electric vehicle for the purpose of automatic plugging-in of a connector, the socket having a connection function, the method comprising providing a digital camera image of an area in which the socket is likely to be present using a camera having at least one configurable parameter, the method comprising the steps of recognizing, using a feature recognition algorithm, the position of a reference feature in the digital camera image, the reference feature being part of the connection function of the socket, determining the position and / or orientation of the socket based on the reference feature in the digital camera image, and setting at least one camera parameter for providing the digital camera image such that the reference feature is recognizable to the feature recognition algorithm in the digital camera image.

[0009] The reference features may be or form part of the connection features of the socket, which in this context may comprise features visible in the recording of the charging socket, for example with a documented geometry defined in an international standard such as IEC 62196 or another design specification. Based on the output of a feature recognition algorithm that detects these reference features, a pose estimation algorithm may determine the position and / or orientation of the socket. The reference features may generally be recognizable by their shape, contrast, color, etc. In this particular case, suitable aspects are typically slopes or sharp transitions (edges), which may constitute curves or corners or other complex visual features of holes or pins. The connection features may be formed from or comprise socket parts for electrically and / or mechanically and / or physically coupling the connector. Electrically insulating parts may be included. The (reverse) shape for receiving the connector in general, and the conductive pin and non-conductive body in particular, may be considered as connection features. External markers such as patches are generally not considered to have a connection function.

[0010] Several algorithms may be suitable for recognizing the reference features, including convolutional neural network algorithms or the "You Only Look Once" (YOLO) model, among others.

[0011] Determining the position and / or orientation of the socket can be done with the help of any suitable algorithm for pose estimation: SolvePnP or Ransac, among others, are applicable examples.

[0012] The method according to the invention has as a first advantage the possibility that it is suitable for all vehicles (sockets) without requiring any modifications on the vehicle side. In addition, it exploits the fact that improved information can be obtained from images, particularly from images that are particularly useful for feature recognition of dark objects, but not necessarily useful for other things. The digital camera image of the area where the socket is expected to be present can be from a camera specially installed on or near the assembly that handles the connector for the use of the method, or it can be from a camera monitoring a parking lot for charging electric vehicles, or in general any location where a socket may be required. The presence of the vehicle can be automatically announced by a detection system near the parking lot, an external system, or by communication with the vehicle, or by using a camera also used in the present invention.

[0013] In one embodiment of the present invention, modifying at least one camera parameter is implemented as modifying the exposure time of the camera. It should be noted that throughout this application, specific reference is made to digital cameras that output a single digital image, a series of digital images, or a stream of digital images. Terms derived from analog photography, film, or video technologies may be interpreted herein as their digital equivalents.

[0014] The longer the exposure time that the camera applies when acquiring the image, the "lighter" the image will be. In some light areas in the image, saturation may result, while in darker areas that were previously considered black or nearly black, this may result in lighter tones (e.g. lighter shades of grey). These values ​​have been found to provide better information for determining the position and / or orientation of the socket.

[0015] Varying the exposure time can generally involve overexposing the image, which means that some (especially light) colors will end up outside the range that can be displayed.

[0016] In a further embodiment, the at least one camera parameter is a linear amplification gain of an analog signal coming from a photosensitive cell comprised in the camera.

[0017] Increasing the amplifier gain magnifies the analog signal coming from the photosensitive cell, so that the resulting image becomes brighter without increasing the exposure time.

[0018] One particular embodiment of the gain is to provide a number of digital camera images with different gain corrections and form a resulting image where for each pixel the most appropriate corresponding pixel in one of the images is selected.

[0019] In another embodiment of the invention, modifying at least one camera parameter is implemented as modifying the size of the camera aperture.

[0020] By changing the size of the camera's aperture, the amount of light that hits the camera sensor can be changed, thereby changing the brightness of the digital camera, thus achieving the same effect on the digital image as previously described for exposure time.

[0021] In another embodiment, modifying at least one camera parameter is performed by performing gamma correction, where the digital signal is exponentially compressed or expanded by a factor gamma, which may be performed prior to storing the digital camera recording in a digital storage format, thereby non-linearly changing the sensitivity to light, which can be used to increase the sensitivity to the relative differences between darker tones compared to lighter tones.

[0022] In practice, a number of camera parameters are balanced such that they produce a digital camera image that is optimal for recognizing reference features.

[0023] The camera parameters used, and what values ​​they should take, can be determined iteratively, i.e. by analyzing acquired images and taking new images with modified settings, or it can be based on the input of an external sensor such as a light sensor, values ​​calibrated for constant lighting conditions, other information from an external system, or a combination thereof.

[0024] In one embodiment, the method according to the invention comprises determining a region of interest in the acquired camera image, the region of interest being an area where the socket is detected. In one embodiment, the method according to the invention comprises determining a region of interest in the digital camera image, the region of interest being an area where the socket is detected, the digital camera parameters being set based on information from said region of interest, in particular the information on which at least one camera parameter is set is limited to information from the region of interest. In some cases, adjusting the digital camera image is limited to adjusting the region of interest in the image, whereby the changes are particularly useful for the region of interest. In addition to that, this may save computation time, resulting in a faster determination process, and therefore also a faster plug-in sequence. In some cases, the information on which at least one camera parameter is set is limited to information from the reference features of the socket. In some cases, the information on which at least one camera parameter is set is limited to information from a substantially non-reflective area of ​​the socket. In some cases, the information on which at least one camera parameter is set is limited to information from a substantially non-conductive region of the socket. The non-conductive region of the socket includes the body of the socket and the case surrounding the conductive portion or pin of the socket. In the context of the present invention, substantially non-reflective refers to a region that does not substantially reflect optical radiation. In the context of the present invention, non-conductive refers to non-electrically conductive.

[0025] In this context, a region of interest (ROI) is considered to be an area in a digital camera image that includes a reference feature of the charging socket. The digital camera image may include at least one reference feature, more than one reference feature, or all reference features of the charging socket. Preferably, the ROI may be considered to be a set of pixels that constitute a convex hull that encloses all pixels that are considered to be part of the reference feature.

[0026] In a preferred embodiment, the step of providing a digital camera image can include providing a grayscale image or a grayscale representation of the image. In some cases, the grayscale image consists of pixels represented as bit values ​​over an interval determined by a representation such as a bit range, with values ​​increasing from dark to light, and at least one camera parameter is adjusted such that a predetermined target number of pixels have a bit value equal to or greater than the target bit value.

[0027] A digital image has many pixels. In a color image, each pixel is represented by three bit values. Various representations are possible, such as the amount or intensity of each of the colors Red, Green, and Blue (RGB), or the values ​​of Hue, Saturation, and Lightness / Brightness (HSL and HSB). Alternatively, a grayscale image, or a grayscale representation of a color image, requires only one bit value per pixel to represent the lightness of the pixel.

[0028] Although more information is contained in a color image than in a grayscale image, when simply looking for patterns or shapes, the information in a grayscale image is sufficient. The use of grayscale images has the advantage that it reduces the complexity of the model and therefore the need for complex hardware and software. However, in some cases, color images may be utilized within the scope of the present invention.

[0029] In some cases, color images may be converted to grayscale images through several methods, such as using an averaging of the RGB values, or weighting the RGB values ​​using the weights in the ITU-R BT 601 or ITU-R BT 709 recommendations, the latter being preferred.

[0030] When analyzing and adjusting the white balance and / or contrast in a digital image, various values ​​can be taken into account. One indicative combination can be visualized by creating a histogram of the recording in grayscale. This histogram shows the distribution of pixels along the grayscale.

[0031] In a 2592x1944 8-bit image, there are 5,038,848 pixels, each with a value on the interval (bit range) from 0 to 255, increasing from dark to light. The horizontal axis of the histogram shows the bit value along the bit range, while the vertical axis shows the number of pixels. Thus, a point on the diagram shows the number of pixels with a given bit value.

[0032] Two indicative values ​​based on which recording parameters can be adjusted are the number of pixels, given as a percentage of the total number of pixels, that have at least a certain bit value, given as a percentage of the bit range.

[0033] In a conventionally balanced image, the objective is often to find camera settings that result in a good balance for the lighter features in the image. A person skilled in the art would choose a large bit value and a low percentage to achieve that result; that is, the camera settings should be set so that a small number of pixels have a relatively high value. A typical value is that 13% of the pixels should have a bit value of 58% or higher.

[0034] In the present invention, the objective is to find a camera setting that provides a good balance for darker objects, because the charging socket is a black body and a black hole. In contrast, the present invention aims to provide an image with at least a large number of pixels with a relatively small bit value, which forces the darkest objects in the image to have at least a certain bit value, which allows the darker tones to be better distinguished. Thus, the characteristics of the charging socket are more easily detected. A side effect is that objects that are lighter than the socket may be considered overexposed.

[0035] The number of pixels at a specified bit value should preferably be greater than 75%, more preferably greater than 90%, and most preferably greater than 99%. The bit value should preferably be greater than 6.25%, more preferably greater than 12.5%, and most preferably greater than 25% of the bit range.

[0036] Another aspect of the invention is that whether a feature is recognizable or not is used to adjust camera settings. For a feature to be recognizable, there should be a significant change in absolute bit value between the pixels that represent the feature. That is, within the region of interest, the difference between the lightest and darkest pixels, excluding pixels that are conductive parts of the socket, should be greater than 20, more preferably greater than 30, or most preferably greater than 40.

[0037] The conductive parts of the socket are excluded because they are usually reflective. In the context of this invention, that means that the conductive parts are usually overexposed. Overexposure makes them difficult to distinguish.

[0038] Alternatively, when obtaining a sample of pixels that includes all or part of a distinctive aspect of a feature, such as an edge or line, and that is an area of ​​pixels that represents an area on or within the socket, the absolute bit value difference between the lightest and darkest pixels in the sample should preferably be greater than 10, more preferably greater than 20, or most preferably greater than 30. Such an area may have a size just sufficient to make the reference feature detectable within that area. Such an area may, by way of example, be 1 mm square or larger, and may, by way of example, be on the order of 3 mm by 3 mm.

[0039] The method according to the invention may further comprise controlling the position and / or orientation of the connector based on the determined position and / or orientation of the socket, more specifically inserting the connector into the socket.

[0040] The invention also relates to a device for determining the position and / or orientation of a socket of an electric vehicle, comprising a camera and a processor for carrying out the method described above. Obviously, the device may further comprise a connector for plugging into the socket, which connector may be coupled to a charging facility, which charging facility may also form part of the proposed solution according to the invention.

[0041] Such an apparatus may further comprise an actuation mechanism for moving the connector for the purpose of charging the electric vehicle, the actuation mechanism being adapted to control the position and / or orientation of the socket based on the determined position and / or orientation of the socket. Apparatuses that have proven highly suitable for performing such automatic plugging sequences are described in the applicant's patent applications, in particular in the numbers NL2023019, NL2024952, NL2025959, NL2026365, NL2026710 and NL2028169, which applications are incorporated herein by reference. All of the apparatuses described therein may be configured to perform the method according to the invention.

[0042] The invention will now be described in more detail with reference to the following figures. [Brief description of the drawings]

[0043] [Figure 1] FIG. 1 illustrates a socket scheme including features related to establishing a charging connection. [Diagram 2] FIG. 13 shows an image of a charging socket where 13% of the pixels have a bit value of at least 58% of the bit range, and a histogram to illustrate the distribution of the pixels. [Diagram 3] FIG. 13 shows an image of a charging socket where 50% of the pixels have a bit value of at least 50% of the bit range, and a histogram to illustrate the distribution of the pixels. [Figure 4] FIG. 13 shows an image of a charging socket where 75% of the pixels have a bit value of at least 25% of the bit range, and a histogram to illustrate the distribution of the pixels. [Diagram 5] FIG. 13 shows an image of a charging socket where 99% of the pixels have a bit value of at least 6.25% of the bit range, and a histogram to illustrate the distribution of the pixels. [Figure 6] FIG. 13 shows an image of a charging socket where 99% of the pixels have a bit value of at least 25% of the bit range, and a histogram to illustrate the distribution of the pixels. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0044] 1 shows a cross-sectional view of a charging socket, including fiducial features that are part of the socket's connection features. In this particular case, the gradients and sharp transitions (edges) that are recognizable to a feature recognition algorithm can constitute curves or corners of a front surface 1, holes 2, pins 3, or other defined curves 4.

[0045] FIG. 2 shows the results of using traditional settings to reach the target value when there are both light and dark objects in the image.

[0046] Figure 3 shows the result of using settings to reach a target value in the middle of the range, with the same object in the image as in Figure 2.

[0047] FIG. 4 shows the results of using the settings for reaching the preferred target with the same objects in the image as in FIG.

[0048] Comparing these figures, it can be seen that the conventional average setting favors the visibility of lighter objects, while the preferred setting favors the visibility of darker objects. In the histograms of Figures 2 and 3, this is seen as a disproportionate peak on the left, while the histogram of Figure 4 has a disproportionate peak on the right. That is, in conventional terms, Figures 2 and 3 are underexposed, while Figure 4 is overexposed. Moreover, visual inspection reveals that in Figures 2 and 3, the reference features of the connectivity function are not recognized or are not easily recognized. However, in Figure 4, they are recognizable.

[0049] FIG. 5 shows an image with only black objects when the camera parameters are configured to obtain a preferred target bit value and the most preferred target number of pixels.

[0050] FIG. 6 shows an image having only black objects when the camera parameters are configured to obtain the most favorable target bit value and the most favorable target number of pixels.

[0051] Comparing the histograms of Figures 5 and 6, it can be seen that at the most favorable target value, the pixels are more distributed across the bit values, indicating that there is more information available to the feature recognition algorithm. From the images and their histograms, it can be seen that the image according to the invention has clearly distinguishable features, while the other images do not. In addition, the image according to the invention with the lighter objects is overexposed on the lighter objects.

Claims

1. 1. A method for determining the position and / or orientation of a socket of an electric vehicle for the purpose of automatic plugging-in of a connector, said socket having a connection function, said method comprising: providing a digital camera image of an area where said socket is likely to be present using a camera having at least one configurable parameter; using a feature recognition algorithm to recognize the location of a reference feature in the digital camera image, the reference feature being part of the connection feature of the socket; determining a position and / or orientation of the socket based on the reference features in the digital camera image; A method comprising: setting at least one camera parameter to provide the digital camera image such that the reference feature is recognizable to the feature recognition algorithm within the digital camera image; A method characterized by:

2. The method of claim 1 , wherein the at least one camera parameter is an exposure time.

3. 2. The method of claim 1, wherein the at least one camera parameter is a linear amplification gain of an analog signal coming from a photosensitive cell included in the camera.

4. The method of claim 1 , wherein the at least one camera parameter is a camera aperture size.

5. The method of claim 1 , wherein the at least one camera parameter is gamma correction, and the digital signal is exponentially amplified by a factor gamma.

6. The method of claim 1 , wherein setting at least one camera parameter is based on external information, such as a signal from a light sensor, a value calibrated for a given lighting condition, or information from an external system.

7. The method of claim 1 , further comprising determining a region of interest in the digital camera image that includes at least one reference feature of the socket.

8. The method of claim 1 , wherein setting at least one camera parameter is based on a previously recorded digital camera image.

9. 9. The method of claim 8, comprising determining a region of interest in the previously recorded digital camera image, the region of interest being an area in which a socket is detected, and wherein information on which the at least one camera parameter is set is limited to information from the region of interest.

10. The method of claim 1 , wherein the step of providing a digital camera image includes providing a grayscale image or grayscaling the captured image.

11. 11. The method of claim 10, wherein the grayscale image comprises pixels represented as bit values ​​on an interval determined by a representation such as a bit range, with values ​​increasing from dark to light, and the at least one camera parameter is adjusted so that a predetermined target number of pixels have bit values ​​equal to or greater than a target bit value.

12. 12. The method of claim 11, wherein the at least one camera parameter is adjusted so that a predetermined target number of pixels is at least 75%, more preferably at least 90%, and most preferably at least 99%, and the target bit value is preferably at least 6.25%, more preferably at least 12.5%, and most preferably at least 25% of the maximum bit value.

13. 12. The method of claim 11, wherein the at least one camera parameter is adjusted such that within the region of interest, the absolute difference in absolute bit values ​​between the lightest and darkest pixels is greater than 20, more preferably greater than 30, and most preferably greater than 40, excluding pixels representing conductive parts of the connection features of the socket.

14. 12. The method of claim 11, wherein the at least one camera parameter is adjusted so that when obtaining a sample of pixels that includes all or part of an aspect of the reference feature, such as an edge or line, that is to be recognizable by the feature recognition algorithm, the absolute bit value difference between the lightest and darkest pixels in the sample is preferably greater than 10, more preferably greater than 20, and most preferably greater than 30.

15. The method of claim 1 , wherein the information on which the at least one camera parameter is set is limited to information from the reference features of the socket.

16. The method of claim 1 , wherein the information upon which the at least one camera parameter is set is limited to information from a substantially non-reflective area of ​​the socket.

17. The method of claim 1 , wherein the information upon which the at least one camera parameter is set is limited to information from a substantially non-conductive region of the socket.

18. The method of claim 1 , wherein the information on which the at least one camera parameter is set is limited to information from darker colored areas of the socket.

19. The method of claim 1 , comprising controlling a position and / or orientation of the connector based on the determined position and / or orientation of the socket.

20. 1. A device for determining the location and / or orientation of a socket of an electric vehicle, comprising: a camera having configurable parameters; processing means, such as a microprocessor, configured to perform the method of claim 1; an actuation mechanism for moving a connector for the purpose of charging an electric vehicle, the actuation mechanism adapted to control a position and / or orientation of the connector based on the determined position and / or orientation of the socket; An apparatus comprising:

Images