Method and system for vehicle presence detection

Abstract

For vehicle presence detection, a camera captures (1) an image of an environment of a charging station for electric vehicles. The captured image is analyzed, by a first machine learning model (2a) and by a second machine learning model (2b), wherein the first machine learning model classifies the captured image with regard to a presence of a vehicle, and wherein the second machine learning model estimates a distance between the camera and an object in the captured image. A decision logic (DL) processes (3) the output of the first machine learning model and the output of the second machine learning model and triggers a control process of the charging station (CPCS), if a vehicle has been detected by the first machine learning model and if according to the output of the second machine learning model, a distance between the camera and the detected vehicle is below a given threshold. As a result, reliable vehicle presence detection is achieved by utilizing solely an already existing 2D camera and a combination of Al-based computer vision models. Material and maintenance costs are reduced, since a 2D camera is usually already in place and no additional presence sensors such as ultrasonic sensors are required. As a consequence, drawbacks of ultrasonic sensors are avoided as they do not distinguish between different object classes, i.e. if it is a bicycle, a person or any other non-vehicle, that should not trigger the charging station's control process.

Description

[0001] Description

[0002] Method and system for vehicle presence detection

[0003] Technical Field

[0004] This application relates to a method and system for vehicle presence detection. More particularly, this application relates to vehicle presence detection as an initial part of the operation of an autonomous charging station for electric vehicles, in particular electric cars or heavy-load electric transport vehicles. The electrical vehicles themselves can be autonomous or non-autonomous.

[0005] Background Art

[0006] In the future, autonomous charging stations are becoming crucial for the automated charging of autonomous electric vehicle fleets at scale. In such systems the autonomous electric vehicle approaches the station. Once the system detects the presence of the vehicle, it starts the process of searching and plugging the charging inlet. The Siemens Autonomous Charging System prototype is an example of such a system.

[0007] For this system to work, the autonomous electrical vehicle has to park in a marked area to be charged. The Siemens Autonomous Charging System can charge all electric vehicles that have a standardized CCS charging connector. A robot that moves in all spatial axes connects to the vehicle’s CCS charging connector in less than a minute and charges the batteries with up to 300 kilowatts of power. After a certain amount of time - which depends on the capacity and state of charge of the batteries and the vehicle’s maximum permissible charging power - the robot breaks the connection, and the car can continue on its way fully charged.

[0008] In the coming years, heavy-load transport vehicles will also be increasingly electrified, because the limits imposed by the Ell on permissible fleet-wide CO2 emissions will go into effect in 2025. The logistics companies claim that the mandatory 45-minute breaks that drivers have to take every four-and-a-half hours are sufficient to charge the vehicles. However, heavy vehicles need to be charged with a one to three megawatt charging capacity, and that requires charging cables that are so thick and heavy that people will still need to rely on assistance to use them.

[0009] The detection of vehicle presence is an important step because the whole control process is triggered based on this signal. Currently, there is no standardized communication protocol established that enables the autonomous electric vehicle to communicate its presence to the charging station.

[0010] But also, if there is a standardized protocol established, it’s still beneficial to have an additional confirmation that the electric vehicle is right in front of the station in sufficient distance so that the system’s charging connector can reach the charging inlet reliably.

[0011] Summary of Invention

[0012] It is an object of the present invention to identify a problem in the prior art and to find a technical solution for this.

[0013] The objectives of the invention are solved by the independent claims. Further advantageous arrangements and embodiments of the invention are set forth in the respective dependent claims.

[0014] According to the method for vehicle presence detection, the following operations are performed by components, wherein the components are hardware components and / or software components executed by one or more processors: capturing, by a camera, an image of an environment of a charging station for electric vehicles, analyzing, by a first machine learning model and by a second machine learning model, the captured image, wherein the first machine learning model classifies the captured image with regard to a presence of a vehicle, and the second machine learning model estimates a distance between the camera and an object in the captured image, and processing, by a decision logic, the output of the first machine learning model and the output of the second machine learning model and triggering a control process of the charging station, if a vehicle has been detected by the first machine learning model and if according to the output of the second machine learning model, a distance between the camera and the detected vehicle is below a given threshold.

[0015] The system for vehicle presence detection comprises the following components, wherein the components are hardware components and / or software components executed by one or more processors: a camera, configured for capturing an image of an environment of a charging station for electric vehicles, a first machine learning model and a second machine learning model, configured for analyzing the captured image, wherein the first machine learning model classifies the captured image with regard to a presence of a vehicle, and the second machine learning model estimates a distance between the camera and an object in the captured image, and a decision logic, configured for processing the output of the first machine learning model and the output of the second machine learning model, and configured for triggering a control process of the charging station, if a vehicle has been detected by the first machine learning model and if according to the output of the second machine learning model, a distance between the camera and the detected vehicle is below a given threshold.

[0016] The following advantages and explanations are not necessarily the result of the object of the independent claims. Rather, they may be advantages and explanations that only apply to certain embodiments or variants.

[0017] The term "computer" should be interpreted as broadly as possible, in particular to cover all electronic devices with data processing properties. Computers can thus, for example, be personal computers, servers, clients, programmable logic controllers (PLCs), handheld computer systems, pocket PC devices, mobile radio devices, smartphones, or any other communication devices that can process data with computer support, for example processors or other electronic devices for data processing. Computers can in particular comprise one or more processors and memory units.

[0018] In connection with the invention, a "memory", "memory unit" or "memory module" and the like can mean, for example, a volatile memory in the form of random-access memory (RAM) or a permanent memory such as a hard disk, a solid state drive or a Disk.

[0019] The method and system, or at least some of their embodiments, offer a solution for reliable presence detection by utilizing solely an already existing 2D camera (which can also be used for the subsequent charging inlet search and detection process) and a combination of Al-based computer vision models.

[0020] The method and system, or at least some of their embodiments, reduce material and maintenance costs, since a 2D camera is usually already in place and no additional presence sensors such as ultrasonic sensors are required. Also, a comparatively cheap 2D camera (RGB or monochrome image needed only) can be used, and no expensive 3D depth camera is necessary. As a result, total charging time can also be reduced, by more reliable detection of vehicles and distinguishing the presence of non-vehicles, which prevents the triggering of the charging control process by mistake.

[0021] The method and system, or at least some of their embodiments, do not require additional presence sensors, i.e. ultrasonic sensors, to detect the presence of the vehicle. As a consequence, drawbacks of ultrasonic sensors are avoided as they do not distinguish between different object classes, i.e. if it is a bicycle, a person or any other non-vehicle, that should not trigger the charging station’s control process.

[0022] Description of Embodiments

[0023] In an embodiment of the method and system, the camera is a 2D camera, in particular an RGB camera or a monochrome camera.

[0024] In another embodiment of the method and system, the second machine learning model is a monocular depth estimation model, which outputs an estimated distance value for each pixel in the captured image.

[0025] In a further embodiment of the method and system, the detected vehicle is an electric car or a heavy-load electric transport vehicle.

[0026] The autonomous charging system contains the system for vehicle presence detection and a control unit, configured for executing the control process of the charging station.

[0027] The charging station for electric vehicles contains the autonomous charging system.

[0028] The computer program product comprises instructions which, when the program is executed by a computer, cause the computer to carry out the method.

[0029] The provisioning device for the computer program product stores and / or provides the computer program product.

[0030] Brief Description of the Drawing The foregoing and other aspects of the present invention are best understood from the following detailed description when read in connection with the accompanying drawings. For the purpose of illustrating the invention, the drawings show embodiments that are presently preferred. However, the invention is not limited to the specific instrumentalities disclosed. The embodiments may be combined with each other. Furthermore, the embodiments may be combined with any of the features described above. Unless stated otherwise, identical reference signs denote the same features or functionally identical elements between drawings. Included in the drawings are the following Figures:

[0031] Fig. 1 shows a first embodiment,

[0032] Fig. 2 shows another embodiment,

[0033] Fig. 3 shows a flowchart of a possible exemplary embodiment,

[0034] Fig. 4 shows a first case for image classification,

[0035] Fig. 5 shows a second case for image classification,

[0036] Fig. 6 shows a third case for image classification, and

[0037] Fig. 7 shows a fourth case for image classification.

[0038] In the following description, various aspects of the present invention and embodiments thereof will be described. However, it will be understood by those skilled in the art that embodiments may be practiced with only some or all aspects thereof. For purposes of explanation, specific numbers and configurations are set forth in order to provide a thorough understanding.

[0039] However, it will also be apparent to those skilled in the art that the embodiments may be practiced without these specific details.

[0040] The described components can each be hardware components or software components. For example, a software component can be a software module such as a software library; an individual procedure, subroutine, or function; or, depending on the programming paradigm, any other portion of software code that implements the function of the software component. A combination of hardware components and software components can occur, in particular, if some of the effects according to the invention are preferably exclusively implemented by special hardware (e.g., a processor in the form of an ASIC or FPGA) and some other part by software. Description of Examples

[0041] Fig. 1 shows one sample structure for computer-implementation of the invention which comprises:

[0042] (101) computer system

[0043] (102) processor

[0044] (103) memory

[0045] (104) computer program (product)

[0046] (105) user interface

[0047] In this embodiment of the invention the computer program 104 comprises program instructions for carrying out the invention. The computer program 104 is stored in the memory 103 which renders, among others, the memory 103 and / or its related computer system 101 a provisioning device for the computer program 104. The computer system 101 may carry out the invention by executing the program instructions of the computer program 104 by the processor 102. Results of invention may be presented on the user interface 105. Alternatively, they may be stored in the memory 103 or on another suitable means for storing data.

[0048] Fig. 2 shows another sample structure for computer-implementation of the invention which comprises:

[0049] (201) provisioning device

[0050] (202) computer program (product)

[0051] (203) computer network / lnternet

[0052] (204) computer system

[0053] (205) mobile device / smartphone

[0054] In this embodiment the provisioning device 201 stores a computer program 202 which comprises program instructions for carrying out the invention. The provisioning device 201 provides the computer program 202 via a computer network / lnternet 203. By way of example, a computer system 204 or a mobile device / smartphone 205 may load the computer program 202 and carry out the invention by executing the program instructions of the computer program 202.

[0055] In a variation of this embodiment, the provisioning device 201 is a computer-readable storage medium, for example a SD card, that stores the computer program 202 and is connected directly to the computer system 204 or the mobile device / smartphone 205 in order for it to load the computer program 202 and carry out the invention by executing the program instructions of the computer program 202.

[0056] Preferably, the embodiments shown in Figs. 3 to 7 can be implemented with a structure as shown in Fig. 1 or Fig. 2.

[0057] Fig. 3 shows a flowchart of a possible exemplary embodiment.

[0058] In a first step 1 , an image capturing IC is performed cyclically using an already existing 2D camera, for example an RGB or monochrome camera. That camera captures an image of an environment of a charging station for electric vehicles.

[0059] In a second step, the captured images are analyzed concurrently or consecutively by a first machine learning model 2a and by a second machine learning model 2b. Both machine learning models can be, for example, Al-based computer vision models.

[0060] The first machine learning model 2a is a vehicle classification model VM that classifies if a vehicle is present in the captured image or not, based on a confidence threshold. The vehicle classification model VM is trained on a dataset of images containing vehicles and non-vehicle objects. In other words, the vehicle classification model VM is able to detect an object in the captured image and to classify that object. For this purpose, a generic pretrained classification model can be used and fine-tuned with a custom dataset for the vehicle classification task.

[0061] Depending on the use case for the charging station, the first machine learning model 2a can be trained to detect an electric car and / or a heavy-load electric transport vehicle.

[0062] The second machine learning model 2b is a monocular depth estimation model DE that estimates the distance of the object in the captured image, at least if it was classified as a vehicle by the first machine learning model 2a. The task of the monocular depth estimation model DE is estimating the depth value (distance relative to the camera) of each pixel given the captured image, which is a single monocular image. The output of the monocular depth estimation model DE is a vector with an estimated distance value for each pixel in the captured image. A pre-trained model can be used, for example as known from the state of the art and described in “Monocular depth estimation”, available on the internet at https: / / huggingface.co / docs / transformers / tasks / monocular_depth_estimation on March 12, 2024. An object recognition mask identifying a contour and / or a shape of the detected object can be output by the vehicle classification model VM along with the classification result. That object recognition mask can be used to select pixels in the captured image belonging to the object and to average the distance values calculated by the monocular depth estimation model DE for these pixels, in order to derive a distance value for the detected object.

[0063] In another variant, the monocular depth estimation model DE can also operate in parallel to the vehicle classification model VM and provide output regardless of the output provided by the vehicle classification model VM. In that case, a background image with no object present can be used as a baseline. If an object is present in the image, then the monocular depth estimation model DE will output shorter estimated distance values for a significant number of pixels. The estimated distance values differing from the baseline can then be averaged to derive a distance value for the detected object.

[0064] In a third step 3, a decision logic DL processes the results of the first machine learning model 2a and the second machine learning model 2b and triggers a control process of the charging station CPCS if a vehicle was classified, with an estimated distance below a given threshold. Depending on the geometry of the charging station, the detected vehicle, and the use case, that given threshold can be, for example 50 cm or 1 m. Of course, specific use cases might require a different threshold, for example 10 cm or 1.5 m.

[0065] Fig. 4 - Fig. 7 show four different cases. For these examples, the expected vehicle classification result, visualized depth estimation result and the corresponding action of the decision logic are described in the following.

[0066] Fig. 4 shows an RGB image RGB-I as a captured source image and a corresponding visualization of a depth estimation DE, where no object is in front of the camera. Here the result of the vehicle classification model VM processing the RGB image RGB-I is “no vehicle”.

[0067] Therefore, the RGB image RGB-I does not need to be processed by the monocular depth estimation model and the action decided upon by the decision logic is “none”.

[0068] Fig. 5 shows an RGB image RGB-I as a captured source image and a corresponding visualization of a depth estimation DE, where a person is in front of the camera. Here the result of the vehicle classification model processing the RGB image RGB-I is “no vehicle”. Therefore, the RGB image RGB-I does not need to be processed by the monocular depth estimation model and the action decided upon by the decision logic is “none”. Fig. 6 shows an RGB image RGB-I as a captured source image and a corresponding visualization of a depth estimation DE, where a vehicle is in front of the camera in close distance. Here the result of the vehicle classification model processing the RGB image RGB-I is “vehicle”. Therefore, the RGB image RGB-I is also processed by the monocular depth estimation model DE depicted in Fig. 3, detecting close distance, and the action decided upon by the decision logic DL depicted in Fig. 3 is to trigger the control process of the charging station CPCS depicted in Fig. 3.

[0069] Fig. 7 shows an RGB image RGB-I as a captured source image and a corresponding visualization of a depth estimation DE, where a vehicle is in front of the camera in far distance. Here the result of the vehicle classification model VM processing the RGB image RGB-I is “vehicle”. Therefore, the RGB image RGB-I is also processed by the monocular depth estimation model, detecting far distance, and the action decided upon by the decision logic is “none”.

[0070] For example, the method can be executed by one or more processors. Examples of processors include a microcontroller or a microprocessor, an Application Specific Integrated Circuit (ASIC), or a neuromorphic microchip, in particular a neuromorphic processor unit. The processor can be part of any kind of computer, including mobile computing devices such as tablet computers, smartphones or laptops, or part of a server in a control room or cloud.

[0071] The above-described method may be implemented via a computer program product including one or more computer-readable storage media having stored thereon instructions executable by one or more processors of a computing system. Execution of the instructions causes the computing system to perform acts corresponding with the operations of the method described above.

[0072] The instructions for implementing processes or methods described herein may be provided on computer-readable storage media or memories, such as a cache, buffer, RAM, FLASH, removable media, hard drive, or other computer readable storage media. Computer readable storage media include various types of volatile and non-volatile storage media. The functions, acts, or tasks illustrated in the figures or described herein may be executed in response to one or more sets of instructions stored in or on computer readable storage media. The functions, acts or tasks may be independent of the particular type of instruction set, storage media, processor or processing strategy and may be performed by software, hardware, integrated circuits, firmware, micro code, and the like, operating alone or in combination. Likewise, processing strategies may include multiprocessing, multitasking, parallel processing, and the like.

[0073] The invention has been described in detail with reference to embodiments thereof and examples. Variations and modifications may, however, be effected within the spirit and scope of the invention covered by the claims. The phrase “A, B and / or C” as an alternative expression may provide that one or more of A, B and C may be used.

[0074] Independent of the grammatical term usage, individuals with male, female, or other gender identities are included within the term.

Claims

Claims1. A computer implemented method for vehicle presence detection, wherein the following operations are performed by components, and wherein the components are hardware components and / or software components executed by one or more processors: capturing (1), by a camera, an image of an environment of a charging station for electric vehicles, analyzing, by a first machine learning model (2a) and by a second machine learning model (2b), the captured image, wherein the first machine learning model (2a) classifies the captured image with regard to a presence of a vehicle, and the second machine learning model (2b) estimates a distance between the camera and an object in the captured image, and processing (3), by a decision logic (DL), the output of the first machine learning model (2a) and the output of the second machine learning model (2b) and triggering a control process of the charging station (CPCS), if a vehicle has been detected by the first machine learning model (2a) and if according to the output of the second machine learning model (2b), a distance between the camera and the detected vehicle is below a given threshold.

2. The method of claim 1 , wherein the camera is a 2D camera, in particular an RGB camera or a monochrome camera.

3. The method according to any of the preceding claims, wherein the second machine learning model (2b) is a monocular depth estimation model (DE), which outputs an estimated distance value for each pixel in the captured image.

4. The method according to any of the preceding claims, wherein the detected vehicle is an electric car or a heavy-load electric transport vehicle.

5. A system for vehicle presence detection, with a camera, configured for capturing (1) an image of an environment of a charging station for electric vehicles, with a first machine learning model (2a) and a second machine learning model (2b), configured for analyzing the captured image, whereinthe first machine learning model (2a) classifies the captured image with regard to a presence of a vehicle, and the second machine learning model (2b) estimates a distance between the camera and an object in the captured image, and with a decision logic (DL), configured for processing (3) the output of the first machine learning model (2a) and the output of the second machine learning model (2b), and configured for triggering a control process of the charging station (CPCS), if a vehicle has been detected by the first machine learning model (2a) and if according to the output of the second machine learning model (2b), a distance between the camera and the detected vehicle is below a given threshold.

6. An autonomous charging system, with a system for vehicle presence detection according to claim 5, and with a control unit, configured for executing the control process of the charging station (CPCS).

7. A charging station for electric vehicles, with the autonomous charging system according to claim 6.

8. A computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out a method according to one of the method claims.

9. A provisioning device for the computer program product according to the preceding claim, wherein the provisioning device stores and / or provides the computer program product.

Images