Self-Location of a Vehicle in a Parking Facility

Abstract

The present invention relates to a method and a sensor device for self-positioning of a vehicle (2) in a parking facility. According to the method, a first pose of the vehicle (2) is determined. Environmental sensor data is generated by means of an environmental sensor device (4a, 4b), and landmarks (6a, 6b, 6c, 6d, 7, 7, 9a, 9d, 9c) in the environment are detected based on the environmental sensor data. The positions of the landmarks (6a, 6b, 6c, 6d, 7, 7, 9a, 9d, 9c) are determined, and a second pose of the vehicle (2) is determined accordingly. Stored corresponding rules are read, which assign the first pose to at least one preferred sensor type or at least one main landmark type, and a first part of the environmental sensor data is selected accordingly, which is generated by means of a first environmental sensor system (4a, 4b) of the environmental sensor device (4a, 4b) configured according to the first sensor type. In order to detect the landmarks (6a, 6b, 6c, 6d, 7, 7, 9a, 9d, 9c), a first landmark detection algorithm is applied to the first part of the environmental sensor data.

Description

Field of the Invention

[0001] The present invention relates to a method for a vehicle to perform self - positioning / autonomous positioning / automatic positioning in a parking facility, wherein a first pose of the vehicle is determined in a map coordinate system of a digital map stored on a storage medium, environmental sensor data representing the vehicle's environment is generated by means of an environmental sensor device of the vehicle, landmarks in the environment are detected based on the environmental sensor data, the positions of the landmarks are determined in the map coordinate system, and a second pose of the vehicle is determined in the map coordinate system based on the positions of the landmarks. The present invention also relates to a corresponding sensor device for a vehicle. Background Art

[0002] Autonomous driving vehicles continuously determine their position and orientation with the aid of suitable sensors and algorithms and can ensure that they are in a passable area without static obstacles by comparing with a digital map. For this purpose, the environment can be continuously detected during driving by means of a suitable environmental sensor system, such as a camera, a radar system, and a lidar system. The obtained digital images or data can be analyzed with the aid of suitable algorithms to identify prominent image contents, so - called features or landmarks, such as surfaces, walls, edges, ground marking lines, and intersections of ground marking lines, and to determine their positions.

[0003] Under given boundary conditions, the quality of the analysis results may vary depending on the type of sensor, so usually multiple sensor types are used simultaneously. The advantage is that the vehicle position and orientation can always be calculated even in parking facilities, such as parking garages, where the line of sight is often limited differently from public roads and highways.

[0004] The detection results can be compared with the information of the digital map by using one or more, if necessary, sensor - specific positioning algorithms. This information describes the landmark type of the landmarks and their positions in the map coordinate system of the digital map, i.e., especially within the parking facility. Based on the positions of the detected landmarks read from the digital map and considering the measured distance between the vehicle and the detected landmarks, the vehicle position and vehicle orientation in the map coordinate system are determined.

[0005] The operation of the environmental sensor system and the positioning algorithm increases the energy consumption that the vehicle's in - vehicle network must provide and requires high computing resources.

[0006] Document US2020 / 0200545A1 discloses a method for detecting landmarks, wherein the detection of the landmark type is limited to a part of the detected environmental data. Thus, for example, in the image area where a stationary vehicle is recognized, the search for ground marking lines, etc. is abandoned. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to reduce the demand for computing resources by means of environmental sensor data in vehicle self-positioning.

[0008] This technical problem is solved by the corresponding subject matter of the independent claims. Advantageous improvements and preferred embodiments are the subject matter of the dependent claims.

[0009] The present invention is based on the idea that a previously stored corresponding rule is read based on a first pose of the vehicle, and the corresponding rule makes the first pose correspond to a type of sensor or a type of landmark. Based on this correspondence, a first part of the environmental sensor data is selected, and the first part is generated by means of a first environmental sensor system configured according to the first sensor type. A corresponding first landmark detection algorithm is applied to detect landmarks based on the first part of the environmental sensor data.

[0010] According to an aspect of the present invention, a method for vehicle self-positioning in a parking facility is provided, wherein a first pose of the vehicle is determined in a map coordinate system of a digital map stored on a storage medium of the vehicle in particular. In particular, at least one computing unit reads a corresponding rule stored on the storage medium, and the corresponding rule makes the first pose correspond to at least one preferred sensor type or at least one main landmark type, in particular the main landmark type in the environment of the first pose. Environmental sensor data showing the vehicle environment is generated by means of an environmental sensor device of the vehicle. In particular, at least one computing unit selects, according to the read corresponding rule, that is, in particular according to the correspondence between the first pose and at least one preferred sensor type or at least one main landmark type, a first part of the environmental sensor data generated by a first environmental sensor system formed according to the first sensor type of the environmental sensor device. In particular, at least one computing unit detects landmarks in the vehicle environment based on the environmental sensor data and determines the positions of the landmarks in the map coordinate system. Here, in order to detect landmarks, a first landmark detection algorithm is used for the first part of the environmental sensor data. In particular, at least one computing unit determines a second pose of the vehicle in the map coordinate system based on the positions of the landmarks and optionally based on the first pose. The vehicle is in particular a motor vehicle, such as a car.

[0011] Self - localization can in particular be understood as the determination of a second pose by the vehicle itself, in particular by at least one computing unit. The first pose can, for example, be determined by the vehicle, in particular by at least one computing unit, beforehand. Here, the first pose is determined in a known manner, that is, for example, by using the vehicle's first environmental sensor system and / or second environmental sensor system and / or other environmental sensor systems. The first pose can also be determined based on geographical coordinates, which are determined by means of a receiver for signals of a global navigation satellite system (GNSS, such as GPS, GLONASS, Galileo, and / or Beidou). Alternatively, the first pose can also be specified in a different way and provided to at least one computing unit.

[0012] Here and hereinafter, unless otherwise stated, the pose includes the position, which, in the case of the vehicle pose, can additionally include the orientation, in particular in a map coordinate system. The first pose of the vehicle in particular includes the first position and the first orientation of the vehicle in the map coordinate system. The second pose of the vehicle in particular includes the second position and the second orientation of the vehicle in the map coordinate system. Here, the vehicle has the first pose at a first time point and the second pose at a second time point, and the second time point is after the first time point.

[0013] The parking facility can, for example, be a parking garage, a parking lot, or other parking areas. The parking facility has a plurality of parking spaces in which a vehicle, in particular a motor vehicle, such as a car, can be parked. For example, it can be a parking facility for valet parking, where the vehicle is brought by a human driver or user into the entrance area of the parking facility. Then, the driver or user can leave the vehicle, and then the vehicle can park autonomously.

[0014] Therefore, this method is a method for self - localization of a fully autonomous vehicle (also known as an autonomous driving vehicle). However, in other embodiments, the vehicle does not have to be configured for fully autonomous driving. Thus, for example, self - localization can be used for semi - autonomous driving functions or assisted driving.

[0015] A landmark can be understood as representing a feature and / or pattern in the environment that can be recognized and can correspond to at least one location information or position information. For example, a landmark can be a feature point or an object arranged at a specific position in the environment.

[0016] A landmark can correspond to a type of landmark, in particular by means of one or more geometric and / or semantic properties of the landmark. Thus, for example, road markings, lane markings, other ground marking lines, building edges or corners, masts, columns, traffic signs, indication signs or other signs, buildings, vegetation elements, buildings or parts thereof, parts of a traffic control system, two-dimensional codes (e.g., QR codes or barcodes), alphanumeric expressions, etc. can accordingly be defined as types of landmarks. Here, a landmark can also correspond to multiple types of landmarks.

[0017] Making the first pose correspond to at least one preferred sensor type or at least one main landmark type can be understood as: when the vehicle is at the first position of the first pose, making the environment of the vehicle correspond to at least one preferred sensor type or at least one main sensor type.

[0018] The environment sensor device includes at least two environment sensor systems, namely, a first environment sensor system and a second environment sensor system. The second environment sensor system is configured according to a second sensor type, and the second sensor type is different from the first sensor type.

[0019] Generally, when generating environment sensor data, the first environment sensor system and the second environment sensor system of the environment sensor device can be activated, and if necessary, other environment sensor systems can also be activated. Thus, in addition to the first part, the environment sensor data usually also includes a second part generated by means of the second environment sensor system.

[0020] However, the second part of the environment sensor data is not used for detecting landmarks. In other words, the determination of landmarks is independent of the second part of the environment sensor data.

[0021] At least one computing unit includes a first detection module, which is a software module and implements and / or stores a first landmark detection algorithm. Thus, the first detection module is activated to detect landmarks.

[0022] At least one computing unit especially further includes a second detection module, which is a software module and implements or stores a second landmark detection algorithm. The second landmark detection algorithm is designed to detect landmarks based on the second part of the environment sensor data generated by means of the second environment sensor system configured according to the second sensor type of the environment sensor device. However, the second detection module is not activated to detect landmarks, or rather the second detection module is deactivated for the purpose of detecting landmarks.

[0023] A landmark detection algorithm can generally be understood as follows: It identifies one or more landmarks based on sensor data of one or more environmental sensor systems, in particular, and determines the position of the landmarks relative to the corresponding environmental sensor system, in particular relative to a vehicle, and optionally classifies the landmarks. The landmark detection algorithm can be designed in particular as an algorithm for automatic perception, for example as an algorithm for automatic visual perception.

[0024] The first landmark detection algorithm and the second landmark detection algorithm differ in that they are designed to detect landmarks based on different sensor data, i.e., in particular based on sensor data of environmental sensor systems constructed according to different sensor types. The first landmark detection algorithm can be designed, for example, to detect landmarks based on camera images or lidar point clouds, while the second landmark detection algorithm can be designed, for example, to detect landmarks based on radar data, or vice versa. However, other cases and sensor types or sensor data types are also possible.

[0025] An automatic visual perception algorithm, which can also be called a computer vision algorithm or a machine vision algorithm, can be regarded as a computer algorithm for automatically performing visual perception tasks. Visual perception tasks, which are also called computer vision tasks, can be understood, for example, as tasks for extracting information from image data. Visual perception tasks can in principle be carried out in particular by a person who can visually perceive an image corresponding to the image data. However, in the current case, the visual perception tasks are also performed automatically without human assistance.

[0026] A computer vision algorithm can, for example, include an image processing algorithm or an algorithm for image analysis, which has been trained by or through machine learning and can, for example, be based on an artificial neural network, in particular a convolutional neural network. A computer vision algorithm can, for example, include an object recognition algorithm, an obstacle recognition algorithm, an object tracking algorithm, a classification algorithm, and / or a segmentation algorithm.

[0027] The corresponding algorithms can also be implemented similarly based on input data different from images perceptible to human vision. For example, point clouds or images from infrared cameras, lidar systems, etc. can also be evaluated with the aid of corresponding matching computer algorithms. Strictly speaking, the corresponding algorithms are not algorithms for visual perception, because the corresponding sensors can operate in areas where vision cannot perceive, i.e., areas imperceptible to the naked eye, such as in the infrared range. Therefore, within the scope of the present invention, such algorithms are referred to as automatic perception algorithms. Thus, automatic perception algorithms include automatic visual perception algorithms, but are not limited to human perception. Therefore, based on this understanding, automatic perception algorithms can include computer algorithms for automatically performing perception tasks, which are trained, for example, by or have been trained by machine learning, and in particular can be based on artificial neural networks. Such a generalized automatic perception algorithm can also include object detection algorithms, object tracking algorithms, classification algorithms, and / or segmentation algorithms, such as semantic segmentation algorithms.

[0028] When using artificial neural networks to implement automatic visual perception algorithms, a commonly used architecture is the convolutional neural network CNN. Two-dimensional CNN can be particularly applied to corresponding two-dimensional camera images. CNN can also be used for other automatic perception algorithms. For example, three-dimensional CNN, two-dimensional CNN, or one-dimensional CNN can be applied to point clouds, depending on the spatial dimension of the point cloud and the details of the processing.

[0029] The result or output of the automatic perception algorithm depends on the specific underlying perception task. For example, the output of an object recognition algorithm can include one or more bounding boxes that define the spatial position, and optionally the orientation, of one or more corresponding objects in the environment, and / or define the corresponding object categories of one or more objects. A semantic segmentation algorithm applied to a camera image can include the category of the pixel plane for each pixel of the camera image. Similarly, a semantic segmentation algorithm applied to a point cloud can include the category of the corresponding point plane for each point. For example, the category of the pixel plane or the point plane can define the object type to which the corresponding pixel or point belongs.

[0030] At least one preferred sensor type particularly includes a first sensor type rather than a second sensor type. At least one sensor type corresponds to one or more sensor types which, according to experience, such as determined by means of a previous analysis of driving, are particularly suitable for self-localization in the corresponding surrounding area of a first pose or a first position. Here, for example, it can be pre-determined which specific sensor types are involved by detecting and classifying landmarks and various other landmarks in a parking facility, so that the landmarks can respectively correspond to the corresponding landmark types and the preferred sensor types associated therewith. In particular, each landmark type corresponds to at least one sensor type. In this way, when the corresponding rule read from the storage medium makes the first pose correspond to at least one main landmark type, at least one computing unit can derive the corresponding preferred sensor type or at least one corresponding preferred sensor type.

[0031] Here, the corresponding rule can be stored, for example, as part of a digital map, especially as an additional map layer or an additional map layer. Thus, for example, for the entire area of a parking facility, the corresponding correspondence can be preset for any position or pose.

[0032] Activating the first detection module can be understood as: it also includes maintaining the activation of the first detection module in the case where the first detection module has been activated at the corresponding time point. Similarly, deactivating the second detection module can also include maintaining the deactivation of the second detection module in the case where the second detection module has been deactivated at the corresponding time point.

[0033] An environmental sensor system can generally be understood as: a sensor system capable of generating environmental sensor data or sensor signals that depict, present, or otherwise reproduce the environment of a vehicle or an environmental sensor system. For example, a camera, a radar system, a lidar system, or an ultrasonic sensor system can be understood as an environmental sensor system.

[0034] Accordingly, the sensor type can be understood as the specific design of the corresponding environmental sensor system, such as being designed as a camera, a radar system, a lidar system, or an ultrasonic sensor system. Depending on the implementation of the method, there can also be more detailed distinctions of different sensor types. For example, different cameras can be distinguished (such as cameras operating in the visible light range or the infrared range, etc.), different radar systems that are particularly sensitive at short or long distances can be distinguished, and different lidar systems can be distinguished (such as laser scanners or flash lidar systems, etc.). In other implementations, a coarser classification can also be considered between different sensor types, for example, according to the detected physical phenomena. Thus, for example, an optical sensor system can be distinguished from a sensor system sensitive to radio waves or a sensor system sensitive to ultrasonic waves, etc. Combinations of different classifications are also feasible.

[0035] The activation of the first detection module and / or the deactivation of the second detection module do not have to occur suddenly or simultaneously. Rather, it is also possible to set a superimposition such that during a transition period, both detection modules, namely the first detection module and the second detection module, are activated and used for self-localization. The activation of the first detection module and / or the deactivation or superimposition of the second detection module also do not have to occur immediately after reading the corresponding rule. For example, it is also possible to actively read the corresponding rule at an earlier time point, so that the vehicle computing unit has more time for planned activation, deactivation, or superimposition.

[0036] Finally, the corresponding rule is also not necessarily the only condition and / or the only basis for activating the first detection module or deactivating the second detection module. Other boundary conditions, such as the instantaneous speed of the vehicle or the required accuracy or minimum accuracy for self-localization, can especially be included in the judgment.

[0037] Therefore, by considering the correspondence of at least one preferred sensor type, directly by making the first pose correspond to at least one preferred sensor type by means of the corresponding rule or indirectly by making the first pose correspond to at least one main landmark type, it is possible to activate only the following detection modules especially during self-localization: the detection module that can provide a relatively high availability for self-localization with a higher probability because the corresponding landmark or feature exists in the corresponding environment of the first pose. Compared with the continuous parallel operation of all available detection modules, the method according to the invention saves computing resources. These saved computing resources can be advantageously used for other computing tasks. Thus, precisely for the limited computing power of autonomous vehicles or highly automated vehicles and the embedded systems often used therein, the present invention has particularly advantageous effects.

[0038] According to at least one embodiment of the method, a second landmark detection algorithm designed for landmark detection based on a second part of the environmental sensor data is deactivated according to a corresponding rule. In other words, the second detection module is deactivated according to the corresponding rule.

[0039] The deactivation of the second detection module does not necessarily mean the deactivation of the second environmental sensor system. In particular, the first environmental sensor system and the second environmental sensor system can be activated, and optionally, additional environmental sensor systems of the environmental sensor device can be activated when necessary, in order to generate environmental sensor data. In other words, both the first part and the second part of the environmental sensor data are generated. The first part is used for landmark detection, but the second part is not used for landmark detection. However, the second part of the environmental sensor data can alternatively be used for other purposes, such as for one or more safety functions, such as an emergency braking function, a distance control function, a lane keeping assist function, etc.

[0040] For example, a driving function or an assisted driving function for automatically or semi-automatically guiding the vehicle can be executed according to the second part of the environmental sensor data.

[0041] The first part and the second part of the environmental sensor data can also be interpreted as corresponding data streams. Thus, the data streams are generated to a certain extent continuously or quasi-continuously in the sense of successive individual images or frames.

[0042] In particular, other algorithms, such as other automatic perception algorithms, can be applied to the second part of the environmental sensor data. According to the results of the other algorithms, at least one computing unit can generate at least one control signal for at least semi-automatic guidance of the vehicle. The at least one control signal can be transmitted, for example, to one or more actuators of the vehicle, which can enable at least semi-automatic guidance of the vehicle.

[0043] According to at least one embodiment, the first environmental sensor system includes or consists of an optical sensor system. For example, the second environmental sensor system includes or consists of a radar system.

[0044] Here, the optical sensor system can be understood as a sensor system that is based on the detection of light, where the light can include visible light and electromagnetic waves in the infrared or ultraviolet spectral range. In other words, the optical sensor system includes at least one optical detector. In particular, a camera or a lidar system is an optical sensor system.

[0045] Such an embodiment is particularly advantageous if there are visible landmarks or landmarks detectable by means of infrared light in the environment of the first pose, which can be used for self-localization and in particular for determining the second pose, however, which cannot be detected by the radar system or cannot be detected with sufficient reliability. For example, this typically applies to lane markings, parking lot markings or other marking lines, or intersections of lane marking lines, etc. In addition, this also applies to the following landmarks: whose semantic content is required to define or uniquely identify the landmark. For example, the meaning of traffic signs or indication signs or warning signs, etc. can be determined using a camera or other optical sensor system, if necessary by subsequent segmentation or detection algorithms, which is hardly possible or not possible at all by the radar system.

[0046] Therefore, in such an embodiment according to the invention, the second detection module for detecting landmarks based on radar system data is deactivated because it does not provide a significant advantage for self-localization.

[0047] According to at least one embodiment, in which the first environmental sensor system comprises or consists of an optical sensor system, the landmark comprises at least one ground marking line or at least one intersection of at least one ground marking line. In other words, at least one ground marking line or at least one intersection of at least one ground marking line is detected as a landmark.

[0048] According to at least one embodiment, the first environmental sensor system comprises or consists of a radar system. For example, the second environmental sensor system comprises or consists of an optical sensor system.

[0049] Such an embodiment is particularly suitable if in the environment of the first pose there are mainly landmarks that cannot be detected by the optical sensor system or cannot be detected reliably by the optical sensor system, but can be detected by the radar system. For example, metal structures that may be completely or partially covered by other objects can be detected reliably by the radar system, while the optical sensor system cannot. For example, such metal structures can be integrated in or on a wall or other building part.

[0050] According to at least one embodiment, in which the first environmental sensor system comprises or consists of a radar system, the landmark comprises at least one metal structure or a building wall or a part of a building wall. In other words, at least one metal structure or a building wall or a part of a building wall is detected as a landmark.

[0051] According to at least one embodiment, the instantaneous speed of the vehicle is determined, for example, by means of a speed sensor of the vehicle. A first part of the environmental sensor data is determined according to the instantaneous speed, in particular the first detection module is activated according to the instantaneous speed.

[0052] For example, the second detection module is deactivated according to the instantaneous speed.

[0053] By additionally considering the instantaneous speed, the following situation can be considered: the determined sensor system or the environmental sensor data generated by it is particularly reliable when the vehicle is stationary or at a low speed, while this may not be the case at higher speeds. Therefore, the reliability of self-localization can be further improved.

[0054] According to at least one embodiment, a first part of the environmental sensor data is determined according to a preset positioning accuracy. In particular, the first detection module is activated according to the preset positioning accuracy.

[0055] For example, the second environmental sensor system is deactivated according to the preset positioning accuracy.

[0056] The positioning accuracy can be, for example, the target accuracy for positioning or the target accuracy for determining the second pose, or the minimum accuracy preset for the positioning of the second pose.

[0057] In this embodiment, it is possible to avoid deactivating the environmental sensor system, especially the second environmental sensor system, because: this second environmental sensor system may only make a small contribution to self-localization, but it can cause a higher overall positioning accuracy. Therefore, these embodiments can achieve a trade-off between power consumption and positioning accuracy.

[0058] According to at least one embodiment, an analysis drive is performed by the vehicle in a parking facility to determine corresponding rules, wherein during the analysis drive, the first detection module is activated and the second detection module is activated.

[0059] Here, the analysis drive is performed especially before determining the first pose and the second pose. Therefore, during the analysis drive, the vehicle can determine the types and positions of landmarks and other possible landmarks in the parking facility, and thus generate or update corresponding rules. In other words, an additional map layer with corresponding rules is generated in this way. The analysis drive does not have to be a driving process specifically performed to determine the corresponding rules, but can be the normal use of the vehicle in the parking facility. In this way, through the analysis drive, or if necessary through multiple analysis drives, the map layer with corresponding rules can be supplemented by other corresponding rules and gradually established, so that the present invention can be applied more and more widely.

[0060] The analysis drive can additionally or alternatively be performed by another vehicle in the parking facility. In this case, during the analysis drive, in particular, the corresponding additional first detection module and the corresponding additional second detection module of the other vehicle are activated. The above explanations regarding the environmental sensor device, the first environmental sensor system and the second environmental sensor system, and the first detection module and the second detection module can be similarly transferred to the corresponding additional environmental sensor device of the other vehicle, the corresponding first detection module and the second environmental sensor system of the additional environmental sensor device, and the additional first detection module and the second detection module.

[0061] In particular, additional environmental sensor data is generated by means of the environmental sensor device during the analysis drive of the vehicle. The first landmark detection algorithm and the second landmark detection algorithm are applied to the additional environmental sensor data to determine the correspondence rules.

[0062] In an alternative embodiment, during the analysis drive of the vehicle, additional environmental sensor data is generated by means of the environmental sensor device. The first landmark detection algorithm and the second landmark detection algorithm are applied to the additional environmental sensor data to determine the correspondence rules.

[0063] Instead of determining the correspondence rules by means of at least one computing unit or at least one additional computing unit, the correspondence rules can also be determined by a computing unit outside the vehicle, such as a cloud computing unit or a cloud server, based on the additional environmental sensor data, and stored, and in particular transmitted to the vehicle.

[0064] According to at least one embodiment, the first environmental sensor system is activated according to the correspondence rules, and the second environmental sensor system is deactivated according to the correspondence rules. The environmental sensor data is generated by means of the first environmental sensor system and in particular independently of the second environmental sensor system. In other words, the environmental sensor data contains a first part but does not contain a second part.

[0065] Therefore, the energy for running the second environmental sensor system can be saved.

[0066] It should be noted that in different embodiments, in addition to the first environmental sensor system and the second environmental sensor system, the environmental sensor device may further include one or more additional environmental sensor systems. For example, when the second environmental sensor system is deactivated, one or more additional environmental sensor systems may remain active. In other words, if the second environmental sensor system has been deactivated, the environmental sensor data does not have to consist only of the first part. More precisely, the environmental sensor data may include one or more additional parts, which are generated by means of one or more additional environmental sensor systems. According to the corresponding rules, one or more additional parts can be used to detect landmarks. Alternatively, landmarks can also be detected based only on the first part of the environmental sensor data.

[0067] According to another aspect of the invention, there is provided a sensor device for a vehicle, in particular a motor vehicle, such as an autonomous vehicle. The sensor device has: an environmental sensor device configured to generate environmental sensor data representing the vehicle's environment; a storage medium that stores a digital map; and a control system. The control system is configured to determine a first pose of the vehicle in the map coordinate system of the digital map. The control system is configured to detect landmarks in the environment based on the environmental sensor data and determine the positions of the landmarks in the map coordinate system. The control system is configured to determine a second pose of the vehicle in the map coordinate system based on the positions of the landmarks. The environmental sensor device includes a first environmental sensor system configured according to a first sensor type, which is configured to generate a first part of the environmental sensor data. The control system is configured to read the corresponding rules stored on the storage medium, which associate the first pose with at least one preferred sensor type or at least one main landmark type. The control system is configured to select the first part of the environmental sensor data to detect landmarks according to the corresponding rules and apply a first landmark detection algorithm to the first part of the environmental sensor data.

[0068] Here, the control system may include one or more computing units. In particular, the control system may include at least one computing unit of the vehicle, which has been described with reference to different embodiments of the method according to the invention, and vice versa.

[0069] Other embodiments of the sensor device according to the invention directly refer to different embodiments of the method according to the invention, and vice versa. The sensor device according to the invention can in particular be configured to carry out the method according to the invention or the sensor device carries out such a method.

[0070] According to another aspect of the invention, there is provided an electronic vehicle guidance system for a vehicle, which includes the sensor device according to the invention.

[0071] An electronic vehicle guidance system can be understood as an electronic system that is configured to guide a vehicle fully automatically or completely autonomously, especially without the need for driver intervention in the control. The vehicle automatically performs all necessary functions, such as steering, braking, and / or acceleration operations, observing and detecting road traffic, and responding accordingly. The electronic vehicle guidance system can in particular enable fully automatic or fully autonomous motor vehicle driving modes according to level 5 of the classification according to SAE J 3016. The electronic vehicle guidance system can also be understood as an advanced driver assistant system (ADAS), which supports the driver during semi-automatic or semi-autonomous driving. The electronic vehicle guidance system can in particular enable semi-automatic or semi-autonomous driving modes according to levels 1 to 4 of the classification according to SAE J 3016. Here and hereinafter, "SAE J 3016" refers to the corresponding standard in the June 2018 version.

[0072] Thus, at least semi-automatic vehicle guidance can include guiding the vehicle according to fully automatic or completely autonomous driving modes according to level 5 of the classification according to SAE J 3016. At least semi-automatic vehicle guidance can also include guiding the vehicle according to semi-automatic or semi-autonomous driving modes according to levels 1 to 4 of the classification according to SAE J 3016.

[0073] The computing unit can in particular be understood as a data processor, so that the computing unit can in particular process data to perform computational operations. This can also include operations for performing indexed access to data structures, such as a look-up table LUT (look-up table).

[0074] The computing unit can in particular comprise one or more computers, one or more microcontrollers, and / or one or more integrated circuits, such as one or more application-specific integrated circuits ASIC (application-specific integrated circuit), one or more field-programmable gate arrays FPGA, and / or one or more system-on-a-chip SoC (system on a chip). The computing unit can also comprise one or more processors (such as one or more microprocessors), one or more central processing units, CPU (central processing unit), one or more graphics processing units GPU (graphics processing unit), and / or one or more signal processors, in particular one or more digital signal processors DSP. The computing unit can also comprise a physical or virtual combination of a computer or other of the above units.

[0075] In various embodiments, the computing unit includes one or more hardware interfaces and / or software interfaces and / or one or more memory units. According to another aspect of the present invention, a motor vehicle having a sensor device according to the present invention and / or an electronic vehicle guidance system according to the present invention is also proposed.

[0076] The present invention also includes combinations of the features of the above-described embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Embodiments of the present invention will be described below. Among them:

[0078] Figure 1 A schematic diagram showing an exemplary embodiment of a sensor device according to the present invention is shown. DETAILED DESCRIPTION OF THE INVENTION

[0079] The embodiments described below are preferred embodiments of the present invention. The components described in these embodiments correspondingly constitute separate features of the present invention that can be considered independently of each other, and they also correspondingly improve the present invention independently of each other. Therefore, they can also form components of the present invention individually or in a manner different from the shown combinations. In addition, the illustrated embodiments can also be supplemented by other features of the features of the present invention that have been described.

[0080] In Figure 1 a motor vehicle 2, in particular an autonomous motor vehicle, is schematically shown, which has an exemplary embodiment of a sensor device 1 according to the present invention.

[0081] The sensor device 1 includes at least two environmental sensor systems 4a, 4b constructed according to different sensor types. For example, the first environmental sensor system 4a can be an optical sensor system, such as a camera, and the second environmental sensor system 4b is a radar system. However, the present invention is not limited to the combination of these two sensor types described above, but any different sensor types can be used.

[0082] The sensor device 1 also has a control system 3, which has a storage medium 5. The control system 3 can include one or more computing units of the vehicle and can be used to control the environmental sensor systems 4a, 4b and to evaluate the environmental sensor data generated by the environmental sensor systems 4a, 4b.

[0083] The motor vehicle 2 is particularly located in a parking facility. Exemplarily, a plurality of different landmarks 6a, 6b, 6c, 6d, 7, 8, 9a, 9c, 9d in the parking facility are shown. Here, the landmarks 6a, 6b, 6c, 6d, 7, 8, 9a, 9b, 9c can correspond to different landmark types. For example, the landmarks 6a, 6b, 6c, 6d can be, for example, ground marking lines, etc., for example for delimiting parking spaces. The landmark 7 can be, for example, a pillar, a traffic sign, etc. The landmark 8 can be a wall or other building part or other building structure. The landmarks 9a, 9b, 9c can be, for example, metal structures integrated into the wall, such as steel beams, etc.

[0084] Depending on the landmark type, the adaptability of the respective different environmental sensor systems 4a, 4b for detecting the corresponding landmarks 6a, 6b, 6c, 6d, 7, 8, 9a, 9b, 9c is different, and thus for the self-positioning of the vehicle 2, the different environmental sensor systems are more or less differently adaptable or valuable.

[0085] By means of the sensor device 1, for example, a method according to the invention for the self-positioning of the vehicle 2 in a parking facility can be carried out. For this purpose, a first pose of the vehicle 2 can first be determined in the map coordinate system of a digital map stored on a storage medium 5. This can be achieved, for example, based on environmental sensor data generated by means of two environmental sensor systems 4a, 4b. The environmental sensor data can be compared with the digital map such that the position and / or orientation of the vehicle 2 can be determined in the map coordinate system. For this purpose, the control system 3 can in particular evaluate the environmental sensor data in order to detect one or more of the landmarks 6a, 6b, 6c, 6d, 7, 8, 9a, 9b, 9c. Since the positions of the landmarks 6a, 6b, 6c, 6d, 7, 8, 9a, 9b, 9c are also stored in the digital map, the motor vehicle 2 can be positioned accordingly.

[0086] In the control system 3, a first detection module corresponding to the first environmental sensor system 4a is provided, and this first detection module implements a first landmark detection algorithm. Similarly, in the control system 3, a second detection module corresponding to the second environmental sensor system 4a is provided, and this second detection module implements a second landmark detection algorithm. Here, the first landmark detection algorithm is designed to detect landmarks based on environmental sensor data generated by means of the first environmental sensor system 4a, that is, for example, based on camera images. The second landmark detection algorithm is designed to detect landmarks based on environmental sensor data generated by means of the second environmental sensor system 4b, that is, for example, based on radar data.

[0087] In addition to the position information regarding landmarks 6a, 6b, 6c, 6d, 7, 8, 9a, 9b, 9c, the digital map also has information regarding the corresponding landmark types of landmarks 6a, 6b, 6c, 6d, 7, 8, 9a, 9b, 9c. For example, there may be an additional map layer that stores corresponding correspondence rules for each position in a parking facility, and the correspondence rules make the corresponding position correspond to the main landmark type in the environment around the corresponding position. Based on the determined first pose of the vehicle 2, the control system 3 can correspondingly read the correspondence rules from the map, and then, for example, activate the first detection module according to the correspondence rules, while the second detection module is deactivated according to the correspondence rules.

[0088] Therefore, for example, the control system 3 can select a first part of the environmental sensor data according to the correspondence rules, and the first part is generated by means of the first environmental sensor system 4a. Then, the first detection module can use the first part of the environmental sensor data to detect landmarks and determine the second pose of the vehicle 2 based on the landmarks.

[0089] In the example outlined above, where the first environmental sensor system 4a is a camera and the second environmental sensor system 4b is a radar system, the correspondence rules can, for example, make the determined pose of the vehicle 2 correspond to the landmark types of landmarks 6a, 6b, 6c, 6d, that is, for example, ground marking lines. Since ground marking lines can be detected with high reliability by means of an optical sensor system, such as a camera, and recognized by means of a first landmark detection algorithm, while it is difficult or impossible to detect ground marking lines by virtue of radar data, the second detection module can be correspondingly deactivated without causing a significant loss in positioning accuracy. In this way, the computational resources freed up can be used for other purposes.

[0090] It should be emphasized that the described scenarios are only example scenarios, and in other cases, other decisions regarding activating and / or deactivating the corresponding detection modules can be made.

[0091] Therefore, especially during the driving of the vehicle 2 through a parking facility, it can be continuously checked by comparing with the additional map layer: in which areas of the parking facility which detection modules should be advantageously activated or should remain activated, and which detection modules can be deactivated.

[0092] An autonomous driving vehicle must continuously determine its position and orientation by means of suitable sensors and algorithms, and by comparing the position and orientation with a digital map, ensure that the vehicle is in a passable area without static obstacles. For this purpose, the environment is continuously detected during driving by means of suitable sensors (such as cameras, radars, and lasers). The generated digital images are analyzed by means of suitable algorithms to identify prominent image content, that is, so-called features or landmarks, such as wall surfaces, edges, lines, and line intersections, and determine their positions.

[0093] Since the quality of the analysis results varies depending on the sensor type under given boundary conditions, multiple sensor types are commonly used in autonomous vehicles. The advantage is that the position and orientation of the vehicle can be calculated at any location and at any time, even in parking spaces where the line of sight is often restricted compared to public roads and highways. Thus, for example, in an area with many ground markings, sufficient lines and line intersections can be visually detected with the aid of a camera system, and based on this, the vehicle position and / or the vehicle orientation can be calculated. However, in an area with fewer ground markings but more metal structures, for example, their edges and surfaces can be well detected by radar and used by the positioning algorithm to determine the current vehicle position and / or the vehicle orientation.

[0094] The positioning algorithm can compare the detection results with the information in the digital map, which describes the types of landmarks and their positions within the parking space. Based on the positions of the detected landmarks read from the digital map and taking into account the measured distances between the vehicle and the detected landmarks, the vehicle position and orientation within the parking space are determined.

[0095] According to different embodiments, it is proposed that during autonomous or assisted driving through a parking facility, based on the landmark entries in the digital map, the map entries of the main expected types in the upcoming section are read from the digital map, and when entering this section, for example, only the corresponding landmark detection algorithm for detecting visual features or landmarks is run, while the landmark detection algorithm for detecting the edges and surfaces of metal structures by radar is turned off.

[0096] So if the upcoming section is a section with few visual landmarks but many metal structures, after switching from the upcoming section to the next section, the landmark detection algorithm for visually detecting features or landmarks should be turned off, and the landmark detection algorithm for identifying the edges and surfaces of metal structures with the help of radar signals should be activated.

[0097] If the main expected type of visual landmarks is additionally determined for a section mainly with visual landmarks, and the main expected type of radar landmarks is correspondingly determined for a section mainly with radar landmarks, the already reduced computational power requirements compared to continuously running two landmark detection algorithms in parallel can be further reduced.

[0098] For example, the possible result may be that there are expected to be a very large number of ground markings in the form of parking space boundaries or corresponding line intersections, while vertical columns and uprights only appear in small numbers. For sections dominated by radar landmarks, for example, there may be a large number of small metal structures with clear edges, while there are only a few large metal structures without clear boundaries. If this information also exists in the digital map, then in sections dominated by visual features or landmarks, not only can radar landmark detection be turned off, but also the detection algorithm for unexpected visual landmarks can be turned off. Correspondingly, in sections dominated by metal structures, not only can visual landmark detection be turned off, but also the detection algorithm for unexpected radar landmarks can be turned off. Here, the activation and deactivation of vision- and radar-based positioning and detection modules can be achieved not only in a hard on-and-off manner, but also in a way with soft fading. To avoid jumps in the positioning result and thus the calculated vehicle position, hard switching or soft fading with substitution can occur at such points or areas where there are still sufficient landmarks of the currently used type, but there are also already sufficient landmarks of the type to be applied in the next section. The switching points as well as the start and end of the fading area can also be obtained from the digital map.

[0099] Different embodiments of the present invention also include automatically generating a digital map or supplementing the digital map with additional information about relevant landmark types or sensor types, for example based on the cloud.

[0100] To this end, during the passage of vehicles in a given fleet through a parking facility, the main features and landmarks can first be detected, and then the corresponding landmark types and the corresponding landmark positions can be determined. Then, for example, after leaving the parking facility, an analysis can be made of which landmark types frequently appear in which areas of the parking facility or which landmark types are the main parts.

[0101] On the other hand, it is proposed that after leaving the parking facility, an analysis is made of at which locations it can be assumed that when using the automatically generated extended digital map for self-positioning later, the jumps caused by the activation and deactivation of the corresponding environmental sensor system in the calculation of the vehicle position and orientation can be ignored. To also be able to achieve a soft introduction and cut-off of the corresponding environmental sensor system, rather than a hard on-and-off, an analysis can also be made after leaving the parking facility of where the fading process starts and ends, so that the possible jumps in the calculation of the vehicle position and orientation are low during later use.

[0102] Thus, additional information can be utilized in a digital parking lot map to achieve, for example, the generation of an additional metadata layer based on the cloud. Here, the additional information can relate to the main landmark types within a determined area, descriptions regarding area boundaries, favorable activation points of the environmental sensor system in a front partial area, favorable deactivation points of the environmental sensor system in a traversed partial area, and / or the start and end of a favorable overlay area for the environmental sensor system.

[0103] For example, a vehicle that autonomously travels through a parking lot can be implemented, which only activates the radar system and / or an algorithm for detecting radar landmarks in an area dominated by radar landmarks, and only activates the optical sensor system and / or an algorithm for detecting visual landmarks in an area where visual landmarks are concentrated. Thus, the required computing power, cost, and / or energy can be reduced.

[0104] List of reference numerals

[0105] 1 Sensor device

[0106] 2 Motor vehicle

[0107] 3 Control system

[0108] 4a, 4b Environmental sensor system

[0109] 5 Storage medium

[0110] 6a, 6b, 6c, 6d Landmark

[0111] 7, 8 Landmark

[0112] 9a, 9b, 9c Landmark.

Claims

1. A method for a vehicle (2) to perform self - positioning in a parking facility, wherein, - determining a first pose of the vehicle (2) in a map coordinate system of a digital map stored on a storage medium (5); - generating environmental sensor data representing the environment of the vehicle (2) by means of an environmental sensor device (4a, 4b) of the vehicle (2); - detecting landmarks in the environment based on the environmental sensor data and determining the positions of the landmarks in the map coordinate system; - determining a second pose of the vehicle (2) in the map coordinate system based on the positions of the landmarks; characterized in that - reading a corresponding rule stored on the storage medium (5), the corresponding rule correlating the first pose with at least one preferred sensor type or at least one dominant landmark type; - selecting a first part of the environmental sensor data according to the corresponding rule, the first part being generated by a first environmental sensor system of the environmental sensor device (4a, 4b) configured according to a first sensor type; - applying a first landmark detection algorithm to the first part of the environmental sensor data for detecting landmarks; - deactivating a second landmark detection algorithm according to the corresponding rule, the second landmark detection algorithm being designed to detect landmarks based on a second part of the environmental sensor data, the second part being generated by a second environmental sensor system of the environmental sensor device (4a, 4b) configured according to a second sensor type.

2. The method according to claim 1, characterized in that Implementing an assisted driving function or a driving function for fully or semi - automatically controlling the vehicle based on the second part of the environmental sensor data.

3. The method according to claim 1 or 2, characterized in that performing an analysis drive by the vehicle (2) in the parking facility, wherein, during the analysis drive, additional environmental sensor data is generated by means of the environmental sensor device (4a, 4b), and the first landmark detection algorithm and the second landmark detection algorithm are applied to the additional environmental sensor data to determine the corresponding rule; or - performing an analysis drive by another vehicle in the parking facility, wherein, during the analysis drive, additional environmental sensor data is generated by means of an additional environmental sensor device of the another vehicle, and the first landmark detection algorithm and the second landmark detection algorithm are applied to the additional environmental sensor data to determine the corresponding rule.

4. The method according to claim 1, characterized in that - activating the first environmental sensor system according to the corresponding rule and deactivating the second environmental sensor system according to the corresponding rule; and - generating environmental sensor data by means of the first environmental sensor system.

5. The method according to claim 1 or 2, characterized in that - the first environmental sensor system is configured as an optical sensor system (4a); - detecting at least one ground marking line or at least one intersection of at least one ground marking line as the landmark.

6. The method according to claim 1 or 2, characterized in that - the first environmental sensor system is configured as a radar system (4b); - detecting at least one metal structure as the landmark, or detecting a building wall or a part of a building wall as the landmark.

7. The method according to claim 1 or 2, characterized in that, - determining the instantaneous speed of the vehicle (2); - determining a first part of the environmental sensor data based on the instantaneous speed.

8. The method according to claim 1 or 2, characterized in that, Determining a first part of the environmental sensor data according to a preset positioning accuracy.

9. A sensor device (1) for a vehicle (2), the sensor device (1) having: an environmental sensor device (4a, 4b) configured to generate environmental sensor data representing the environment of the vehicle (2); a storage medium (5) storing a digital map; and a control system (3) configured to, - determining a first pose of the vehicle (2) in the map coordinate system of the digital map; - detecting landmarks in the environment based on the environmental sensor data and determining the positions of the landmarks in the map coordinate system; - determining a second pose of the vehicle (2) in the map coordinate system based on the positions of the landmarks, characterized in that, - the environmental sensor device (4a, 4b) includes a first environmental sensor system configured according to a first sensor type, the first environmental sensor system being configured to generate a first part of the environmental sensor data; - the control system (3) is configured to read a corresponding rule stored on the storage medium (5), the corresponding rule making the first pose correspond to at least one preferred sensor type or at least one main landmark type; - the control system (3) is configured to select a first part of the environmental sensor data for detecting landmarks according to the corresponding rule, and apply a first landmark detection algorithm to the first part of the environmental sensor data, deactivating a second landmark detection algorithm according to the corresponding rule, the second landmark detection algorithm being designed to detect landmarks based on a second part of the environmental sensor data, the second part being generated by a second environmental sensor system of the environmental sensor device (4a, 4b) configured according to a second sensor type.

Images