Method for determining a movement trajectory to a charging position of an at least partially electrically and at least partially autonomously operated motor vehicle, computer program product and assistance system

Abstract

The invention relates to a method for determining a movement trajectory (16) to a charging position (P1) of an at least partially autonomously operated motor vehicle (10), having the steps of: predetermining the charging position (P1) of the motor vehicle (10) relative to a charging point (24) by means of an electronic computing device (22) of the assistance system (12); detecting a charging point feature (26) assigned to the charging point (24) on a wall (28) by means of a detection device (18) of the assistance system (12), wherein the charging position (P1) is predetermined as a function of the charging point feature (26); emitting a two-dimensional pattern (30) onto the wall (28) by means of a pattern generating device (20) of the assistance system (12); determining a current position (P2) of the motor vehicle (10) as a function of the two-dimensional pattern (30) and as a function of the charging point feature (26) by means of the electronic computing device (22); and determining the movement trajectory (16) from the current position (P2) to the charging position (P1) by means of the electronic computing device (22). The invention further relates to a computer program product and to an assistance system (12).

Description

[0001] Method for determining a movement trajectory to a charging position of an at least partially electrically and at least partially autonomously operated motor vehicle, computer program product and assistance system

[0002] The invention relates to a method for determining a movement trajectory to a charging position of an at least partially electrically and at least partially autonomously operated motor vehicle using an assistance system of the motor vehicle. Furthermore, the invention relates to a computer program product and an assistance system.

[0003] So-called checkerboard pattern recognition is already known from the state of the art. This method uses visual perception and an optical evaluation algorithm to detect a relative position based on distortions of this checkerboard. Particularly in the loading area, this makes it possible to determine a movement trajectory that can be followed depending on the loading position (which is particularly predefined) and the current positions in order to reach the loading position.

[0004] Typically, a so-called pattern is used to determine the position using a camera, for example, a checkerboard pattern. This allows the camera to calculate all three spatial axes of the camera position, as well as all three viewing direction vectors or angles, from the image information, particularly position in the image, keystone distortion, and size in the image, thus determining a precise route for vertical approach. This requires the pattern to be attached and printed. In particular, the pattern is a very visually intense pattern. Therefore, there is a need to create a discreet way to approach the loading position.

[0005] Furthermore, if the charging socket itself were to be used for object detection, its small dimensions would make it difficult to detect any distortion accurately enough to perform the appropriate positioning. This requires a larger object, and a discreet option is still required.DE 102017 011 208 A1 relates to a method for the automated charging of an electrical energy storage device of an at least partially electrically operated and autonomous motor vehicle, comprising the steps of: detecting an optical pattern of a charging device by means of an optical detection device of the motor vehicle; determining a charging position, which has a predetermined relative position to the optical pattern, of the charging device as a function of the optical pattern by means of an evaluation device; autonomously moving the motor vehicle to the charging position; and automatically charging the electrical energy storage device with electrical energy in the charging position.

[0006] The object of the following invention is to provide a method, a computer program product, and an assistance system by means of which a movement trajectory to a loading position can be determined in a simple manner.

[0007] This problem is solved by a method, a computer program product, and an assistance system according to the independent patent claims. Advantageous embodiments are specified in the subclaims.

[0008] One aspect of the invention relates to a method for determining a movement trajectory to a charging position of an at least partially electrically and at least partially autonomously operated motor vehicle by means of an assistance system of the motor vehicle. The charging position of the motor vehicle relative to a charging station is specified by means of an electronic computing device of the assistance system. A charging station feature assigned to the charging station is detected on a wall by means of a detection device of the assistance system, wherein the charging position is specified as a function of the charging station feature. A two-dimensional pattern is emitted or projected onto the wall by means of a pattern generation device of the assistance system.A current position of the motor vehicle is determined as a function of the two-dimensional pattern and as a function of the charging station feature by means of the electronic computing device, and the movement trajectory from the current position to the charging position is determined by means of the electronic computing device.

[0009] Thus, based on the two-dimensional pattern, a movement trajectory can be reliably determined that moves the motor vehicle from its current position to the loading position. In particular, corresponding control commands for a longitudinal acceleration device and a lateral acceleration device of the motor vehicle can be generated, allowing the vehicle to move to the loading position at least partially or fully autonomously.

[0010] As already mentioned, the motor vehicle is designed, in particular, as an at least partially electrically powered motor vehicle or even as a fully electrically powered motor vehicle. In particular, the motor vehicle has an electrical energy storage device that can be charged via the charging station.

[0011] In other words, it is provided that the two-dimensional pattern, which is in particular a simple two-dimensional pattern, is projected onto the wall, and the current position can be determined based on the detection, for example, via a camera of the motor vehicle. For this purpose, the charging station feature is also used to determine the corresponding relative position. The movement trajectory is then determined in order to move from the current position to the charging position. In particular, corresponding control commands are then generated, which move the motor vehicle to the charging position at least partially autonomously.

[0012] In particular, the method can be used with a so-called front loader, where, for example, the charging station is arranged on the wall, in particular on the front wall. Furthermore, the method can also be used with a floor loader. In this case, wired charging or inductive charging can be provided. In particular, for example, an inductance coil can be provided in the floor, over which the motor vehicle drives and can thus be charged accordingly. The decisive factor here is the evaluation of the two-dimensional pattern and the evaluation of the charging station feature in order to drive to the correct charging position.

[0013] It should be noted at this point that the method is also applicable to motor vehicles with internal combustion engines. For example, precise positioning of the motor vehicle in a garage or a parking deck or a transport ship can be provided here. The motor vehicle is thus operated at least partially with assistance. Thus, a supplementary aspect or an alternative and independent aspect of the invention also relates to a method for determining a movement trajectory to a parking position of an at least partially autonomously operated motor vehicle by means of an assistance system of the motor vehicle. The parking position of the motor vehicle relative to a position assigned to the parking position, for example on a wall, is specified by means of an electronic computing device of the assistance system.A position feature assigned to the position is detected on a wall by a detection device of the assistance system, with the parking position being predetermined depending on the position feature. A two-dimensional pattern is emitted or projected onto the wall by a pattern generation device of the assistance system. The current position of the motor vehicle is determined depending on the two-dimensional pattern and the position feature by the electronic computing device, and the movement trajectory from the current position to the parking position is determined by the electronic computing device.

[0014] According to an advantageous embodiment, three points in the shape of a triangle are emitted as a two-dimensional pattern. In particular, the triangle is, for example, an isosceles triangle. In particular, this is only the emitted pattern. In other words, the triangle on the wall can have a different shape than the one emitted. If the motor vehicle approaches the loading position head-on and correctly, the triangle behaves congruently with the emitted triangle. If the current position differs from a head-on approach, the points have a different orientation to one another; in particular, the projected triangle is no longer congruent with the emitted triangle, and the movement trajectory can again be reliably determined depending on this.

[0015] A further advantageous embodiment provides for the three points to be projected as an equilateral triangle. In particular, the triangle is thus generated such that the three points are equidistant from one another. Based on the different distances between the projected triangles, a deviation can thus be reliably determined, allowing the movement trajectory to be reliably determined.

[0016] A further advantageous embodiment provides that the current position is determined depending on the relative position of the three points to one another. In particular, based on the relative position of the three points to one another, a corresponding angle of the motor vehicle relative to the wall onto which the three points are projected can be determined. This allows the movement trajectory to be reliably determined.

[0017] Furthermore, it has proven advantageous to determine the current position based on the distances between the three points. In particular, the distance of the motor vehicle or the feature generation device from the wall can be determined based on the distances. In particular, the distances between the three points become smaller as the motor vehicle approaches the wall. Based on this, the current position can be reliably determined.

[0018] In a further advantageous embodiment, a charging plug is detected as a charging station feature. In particular, the charging plug can be identified based on a feature analysis. The charging plug can then be used to evaluate the charging station feature accordingly. Alternatively, the charging station feature can also be determined using other predefined features. For example, the charging station feature can be visually specified and then captured in the image using a feature analysis. This allows the current position to be reliably determined.

[0019] A further advantageous embodiment provides for the two-dimensional pattern to be emitted by means of a laser device. In particular, the assistance system thus comprises the laser device. In particular, three points can be projected onto the wall using one or more lasers. This allows the two-dimensional pattern to be generated with high reliability.

[0020] In a further advantageous embodiment, it can be provided that the charging position is specified based on a learning process for the charging position. For example, a driver can drive the motor vehicle manually into the charging position. The surroundings can then be recorded using the recording device, in particular using the camera. In particular, it is provided that the two-dimensional pattern is also transmitted. The image with the charging station feature and the transmitted pattern can then be recorded and saved as the charging position. The comparison can then be carried out accordingly during future approach, and the movement trajectory can be determined. The presented method is, in particular, a computer-implemented method.Therefore, a further aspect of the invention relates to a computer program product with program code means that cause an electronic computing device, when the program code means are processed by the electronic computing device, to perform a method according to the preceding aspect. Furthermore, the invention therefore also relates to a computer-readable storage medium with the computer program product.

[0021] Yet another aspect of the invention relates to an assistance system for determining a movement trajectory to a charging position of an at least partially electrically and at least partially autonomously operated motor vehicle. The assistance system comprises at least one detection device, a pattern generation device, and an electronic computing device, wherein the assistance system is configured to perform a method according to the preceding aspect. In particular, the method is performed by means of the assistance system.

[0022] Yet another aspect of the invention also relates to a motor vehicle with an assistance system according to the preceding aspect. The motor vehicle is, in particular, operated at least partially autonomously or fully autonomously. Furthermore, the motor vehicle is operated at least partially electrically or fully electrically.

[0023] Advantageous embodiments of the method are to be regarded as advantageous embodiments of the computer program product, the assistance system, and the motor vehicle. The assistance system and the motor vehicle have material features for this purpose in order to be able to carry out corresponding method steps.

[0024] An electronic vehicle guidance system / assistance system can be understood as an electronic system designed to guide a vehicle fully automatically or autonomously, in particular without requiring driver intervention. The vehicle automatically performs all required functions, such as steering, braking, and / or acceleration maneuvers, monitoring and detecting road traffic, and responding accordingly. In particular, the electronic vehicle guidance system can implement a fully automatic or fully autonomous driving mode of the motor vehicle according to Level 5 of the SAE J3016 classification. An electronic vehicle guidance system can also be understood as an advanced driver assistance system (ADAS), which supports the driver in partially automated or semi-autonomous driving.In particular, the electronic vehicle guidance system can implement a partially automated or semi-autonomous driving mode according to levels 1 to 4 of the SAE J3016 classification. Here and below, "SAE J3016" refers to the corresponding standard in the April 2021 version.

[0025] The at least partially automated vehicle guidance may therefore include driving the vehicle according to a fully automated or fully autonomous driving mode of Level 5 according to SAE J3016. The at least partially automated vehicle guidance may also include driving the vehicle according to a partially automated or semi-autonomous driving mode according to Levels 1 to 4 according to SAE J3016.

[0026] Within the scope of the present disclosure, an object recognition algorithm, for example for detecting the pattern and the charging station feature, can be understood as a computer algorithm capable of identifying and locating one or more objects within a provided input data set, for example an input image, for example by defining corresponding bounding boxes or regions of interest (ROIs), and in particular by assigning a corresponding object class to each of the bounding boxes, wherein the object classes can be selected from a predefined set of object classes. The assignment of an object class to a bounding box can be understood as providing a corresponding confidence value or probability that the object identified within the bounding box belongs to the corresponding object class.For example, for a given bounding box, the algorithm may provide such a confidence value or probability for each of the object classes. Assigning the object class may, for example, involve selecting or providing the object class with the highest confidence value or probability. Alternatively, the algorithm may simply specify the bounding boxes without assigning a corresponding object class.

[0027] A computing unit / electronic computing device can be understood, in particular, as a data processing device that contains a processing circuit. The computing unit can therefore, in particular, process data to perform computing operations. This may also include operations for performing indexed access to a data structure, for example, a look-up table (LUT).

[0028] The computing unit may, in particular, contain one or more computers, one or more microcontrollers, and / or one or more integrated circuits, for example one or more application-specific integrated circuits (ASICs), one or more field-programmable gate arrays (FPGAs), and / or one or more single-chip systems (SoCs). The computing unit may also contain one or more processors, for example one or more microprocessors, one or more central processing units (CPUs), one or more graphics processing units (GPUs), and / or one or more signal processors, in particular one or more digital signal processors (DSPs). The computing unit may also include a physical or virtual network of computers or other of the aforementioned units.

[0029] In various embodiments, the computing unit includes one or more hardware and / or software interfaces and / or one or more memory units.

[0030] A memory unit can be a volatile data memory, for example a dynamic random access memory (DRAM) or a static random access memory (SRAM), or a non-volatile data memory, for example a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory or flash EEPROM, a ferroelectric random access memory (FRAM), a magnetoresistive random access memory,MRAM (magnetoresistive random access memory) or phase-change random access memory (PCRAM).

[0031] Algorithms for automatic visual perception, which can also be referred to as computer vision algorithms, machine vision algorithms, or machine vision algorithms, can be considered computer algorithms for automatically performing a visual perception task. A visual perception task, also referred to as a computer vision task, can be understood, for example, as a task for extracting visual information from image data. In particular, the visual perception task can, in principle, in some cases be performed by a human who is able to visually perceive an image corresponding to the image data. In the present context, however, visual perception tasks are also performed automatically, without the need for human assistance.

[0032] A computer vision algorithm may, for example, include an image processing algorithm or an image analysis algorithm that is or has been trained by machine learning and may, for example, be based on an artificial neural network, in particular a convolutional neural network. The computer vision algorithm may, for example, include an object detection algorithm, an obstacle detection algorithm, an object tracking algorithm, a classification algorithm, a semantic segmentation algorithm, and / or a depth estimation algorithm.

[0033] Corresponding algorithms can also be carried out analogously based on input data other than images that can be visually perceived by a human.

[0034] For example, point clouds or images from infrared cameras, lidar systems, etc., can also be evaluated using appropriately adapted computer algorithms. Strictly speaking, these algorithms are not algorithms for visual perception, since the corresponding sensors can operate in ranges that are not visually perceptible, i.e., imperceptible to the human eye, for example, in the infrared range. Therefore, such algorithms are referred to as algorithms for automatic perception in the context of the present invention. Thus, algorithms for automatic perception include algorithms for automatic visual perception, but are not limited to these with regard to human perception.Consequently, an algorithm for automatic perception according to this understanding can include a computer algorithm for automatically performing a perception task, which is or has been trained, for example, through machine learning and can, in particular, be based on an artificial neural network. Such generalized algorithms for automatic perception can also include object detection algorithms, object tracking algorithms, classification algorithms, and / or segmentation algorithms, for example, semantic segmentation algorithms.

[0035] If an artificial neural network is used to implement an algorithm for automatic visual perception, a commonly used architecture is a convolutional neural network (CNN). In particular, a 2D CNN can be applied to corresponding 2D camera images. CNNs can also be used for other automatic perception algorithms.

[0036] For example, 3D CNNs, 2D CNNs, or 1D CNNs can be applied to point clouds, depending on the spatial dimensions of the point cloud and the details of the processing.

[0037] The result or output of an automatic perception algorithm depends on the specific underlying perceptual task.

[0038] For example, the output of an object detection algorithm may contain one or more bounding boxes defining a spatial position and optionally an orientation of one or more corresponding objects in the environment and / or corresponding object classes for the one or more objects. An output of a semantic segmentation algorithm applied to a camera image may contain a pixel-level class for each pixel of the camera image. Similarly, an output of a semantic segmentation algorithm applied to a point cloud may contain a corresponding point-level class for each of the points. The pixel-level or point-level classes, respectively, may define an object type to which the respective pixel or point belongs.

[0039] Further features of the invention emerge from the claims, the figures, and the description of the figures. The features and combinations of features mentioned above in the description, as well as the features and combinations of features mentioned below in the description of the figures and / or shown alone in the figures, can be used not only in the respective specified combination, but also in other combinations or on their own.

[0040] The invention will now be explained in more detail using a preferred embodiment and with reference to the drawings. They show:

[0041] Fig. 1 is a schematic plan view of an embodiment of a

[0042] Motor vehicle with an embodiment of an assistance system; and Fig. 2 is a schematic perspective view according to an embodiment of the

[0043] procedure.

[0044] In the figures, identical and functionally identical elements are provided with the same reference symbols.

[0045] Fig. 1 shows a schematic plan view of an embodiment of a motor vehicle 10 with an embodiment of an assistance system 12. The assistance system 12 is designed for at least partially autonomous operation of the motor vehicle 10. For example, the assistance system 12 can generate control signals for a longitudinal and / or lateral acceleration device in order to be able to carry out corresponding autonomous operation of the motor vehicle 10. Furthermore, the motor vehicle 10 has an electrical energy storage device 14. The motor vehicle 10 is at least partially electrically operated or fully electrically operated.

[0046] The assistance system 12 is designed to determine a movement trajectory 16 to a loading position P1 for the motor vehicle 10. For this purpose, the assistance system 12 has at least one detection device 18, for example in the form of a camera, a pattern generation device 20, for example in the form of a laser device, and an electronic computing device 22.

[0047] In the method according to the invention for determining the movement trajectory 16, it is provided in particular that the charging position P1 of the motor vehicle 10 relative to a charging station 24 is specified by means of the electronic computing device 22. A charging station feature 26 assigned to the charging station 24 is detected on a wall 28 by means of the detection device 18, wherein the charging position P1 is specified as a function of the charging station feature 26. A two-dimensional pattern 30 is transmitted onto the wall 28 by means of the pattern generation device 20. A current position P2 of the motor vehicle 10 is determined as a function of the two-dimensional pattern 30 and as a function of the charging station feature 26 by means of the electronic computing device 22, and the movement trajectory 16 from the current position P2 to the charging position P1 is determined by means of the electronic computing device 22.

[0048] In particular, Fig. 1 shows that the charging station 24 can be designed as a front-mounted charging station. Alternatively, as shown later in Fig. 2, for example, the charging station 24 can also be designed as a floor-mounted charging station. Furthermore, the present exemplary embodiment provides, in particular, for a charging plug of the charging station 24 to be recorded as the charging station feature 26.

[0049] Fig. 2 shows a schematic perspective view according to an embodiment of the method as already at least partially illustrated in Fig. 1. In particular, the present embodiment shows that a floor charging station is provided as charging station 24. The charging station feature 26 is also formed on the wall 28.

[0050] Fig. 2 shows in particular that the two-dimensional pattern 30 emits three points 32 in the shape of a triangle. The three points 32 can in particular be emitted as an equilateral triangle, wherein a projection of the equilateral triangle on the wall 28 again differs from the emission of the equilateral triangle. Based on the relative positions of the three points 32 to one another, the current position P2 can in particular be determined. Furthermore, the current position P2 can be determined depending on the distances between the three points 32 to one another. In particular, based on the angles of the three points 32 to one another, a corresponding angle of the motor vehicle 10 relative to the loading position P1 can be determined. Based on the distances between the three points 32 to one another, a distance of the motor vehicle 10 to the wall 28 can in turn be determined.

[0051] In particular, it is thus provided that a projection is carried out using a laser device or focused light in the shape of an equilateral triangle, whereby in particular only the corner points are emitted, onto the wall 28 on which, for example, the charging socket is located. Through camera capture and image analysis, in particular through pattern recognition of the three points 32, together with the detection of the charging socket, the direction and position values ​​can now be precisely calculated in order to be able to approach the charging position P1 vertically and precisely.

[0052] The projection can be performed using laser diodes or focused light, for example. Alternatively, existing vehicle lighting can be used, for example, using activated apertures / stencils, shadow patterns, or possibly even three laser LEDs.

[0053] 10 motor vehicle

[0054] 12 Assistance system

[0055] 14 Electrical energy storage

[0056] 16 Movement trajectory

[0057] 18 Recording device

[0058] 20 Pattern generation device

[0059] 22 Electronic computing device

[0060] 24 charging stations

[0061] 26 Charging station features

[0062] 28 Wall

[0063] 30 Two-dimensional pattern

[0064] 32 points

[0065] P1 loading position

[0066] P2 Current Position

Claims

Patent claims 1. A method for determining a movement trajectory (16) to a charging position (P1) of an at least partially electrically and at least partially autonomously operated motor vehicle (10) by means of an assistance system (12) of the motor vehicle (10), comprising the steps: Specifying the charging position (P1) of the motor vehicle (10) relative to a charging station (24) by means of an electronic computing device (22) of the assistance system (12); Detecting a charging station feature (26) assigned to the charging station (24) on a wall (28) by means of a detection device (18) of the assistance system (12), wherein the charging position (P1) is predetermined as a function of the charging station feature (26); - emitting a two-dimensional pattern (30) onto the wall (28) by means of a pattern generating device (20) of the assistance system (12); Determining a current position (P2) of the motor vehicle (10) as a function of the two-dimensional pattern (30) and as a function of the charging station feature (26) by means of the electronic computing device (22); and Determining the movement trajectory (16) from the current position (P2) to the loading position (P1) by means of the electronic computing device (22).

2. Method according to claim 1, characterized in that three points (32) in the form of a triangle are emitted as a two-dimensional pattern (30).

3. Method according to claim 2, characterized in that the three points (32) are emitted as an equilateral triangle.

4. Method according to claim 3, characterized in that the current position (P2) is determined depending on a relative position of the three points (32) to each other.

5. Method according to claim 3 or 4, characterized in that the current position (P2) is determined as a function of distances between the three points (32) from one another.

6. Method according to one of the preceding claims, characterized in that a charging plug of the charging station (24) is detected as a charging station feature (26).

7. Method according to one of the preceding claims, characterized in that the two-dimensional pattern (30) is emitted by means of a laser device as the feature generation device (20).

8. Method according to one of the preceding claims, characterized in that the loading position (P1) is specified on the basis of a learning process for the loading position (P1).

9. Computer program product with program code means which cause an electronic computing device (22) to carry out a method according to one of claims 1 to 8 when the program code means are processed by the electronic computing device (22).

10. Assistance system (12) for determining a movement trajectory (16) to a charging position (P1) of an at least partially electrically and at least partially autonomously operated motor vehicle (10), with at least one detection device (18), a pattern generation device (20) and an electronic computing device (22), wherein the assistance system (12) is designed to carry out a method according to one of claims 1 to 8.

Images