Method for operating a parking assistance system, computer program product, parking assistance system and vehicle

Abstract

The invention relates to a method for operating a parking assistance system (110) of a vehicle (100), which is designed to capture and store a to-be-trained trajectory in a training mode (MOD0) and to follow the stored trajectory by the vehicle (100) in a follow-me mode (MOD1). In order to determine whether the stored optical feature used to orient the vehicle in the follow-me mode needs to be updated, the parameter distribution of the optical feature is compared. If the similarity of the compared distributions is below a predetermined threshold, an update is performed.

Description

Technical Field

[0001] The present invention relates to a method for operating a parking assistance system, a computer program product, a parking assistance system, and a vehicle. Background Technology

[0002] It is known that parking assistance systems can learn a specific trajectory to be followed, where, in training mode, the vehicle is manually driven along the trajectory to be followed later. During this training process, environmental data is acquired and stored via vehicle sensors, which enables the vehicle to be located in subsequent following steps. This can be achieved, for example, using VSLAM, where camera images are acquired and evaluated, and the vehicle's current position is determined accordingly.

[0003] In this approach, it is crucial that the stored environmental data is current; otherwise, localization will be impossible. Because the environment changes over time—for example, due to the removal, addition, or relocation of moving objects, or due to ongoing construction work in the surrounding area—environmental data can lose its fluidity. To continue successful localization, the stored environmental data must be updated.

[0004] DE 10 2017 115 991 A1 discloses a method for a driver assistance system for operating a motor vehicle, wherein, during the training phase of the driver assistance system, the trajectory is stored when the driver manually manipulates the motor vehicle along a trajectory, and multiple object features are stored based on images provided by at least one camera of the motor vehicle. During the operation phase of the driver assistance system, the motor vehicle is semi-autonomously manipulated along the stored trajectory based on the stored trajectory and the stored object features. In the operation mode, multiple object features are detected, and the detected object features are assigned to stored object features. Based on this assignment, it is determined whether it is necessary to store the object features and / or the trajectory again. Summary of the Invention

[0005] In this context, the purpose of the present invention is to improve the operation of parking assistance systems.

[0006] According to the first aspect, a method for a parking assistance system for operating a vehicle is proposed. The parking assistance system is configured to capture and store a training trajectory in a training mode, and is configured to follow the stored trajectory by vehicle in a following mode. The training mode includes:

[0007] A1) Manually drive the vehicle along the trajectory.

[0008] A2) Receive at least one image of the vehicle environment during manual driving.

[0009] A3) Determine multiple optical features in the received image, wherein each optical feature is characterized by at least one parameter, and

[0010] A4) Store the dataset containing the identified features.

[0011] Following modes include:

[0012] B1) Receive at least one current image of the vehicle environment during vehicle following operations.

[0013] B2) Determine the optical features in the received current image.

[0014] B3) Determine a first distribution of at least one parameter based on the stored dataset, and determine a second distribution of that parameter based on the determined optical features of the current image.

[0015] B4) Determine the similarity between the first and second distributions, and

[0016] B5) If the determined similarity is less than or equal to the predetermined update threshold, then update the stored dataset.

[0017] The advantage of this method is that it updates the stored dataset using optical features used to locate the vehicle in following mode only when the statistical significance of the necessary update is determined. On one hand, this avoids the possibility of updating even with minor environmental changes, thus not consuming the computational power required for this, which must be provided by the parking assistance system or another computing unit within the vehicle. This means that the processing power can be used for other processes, which helps, for example, increase the safety, reliability, and / or speed of other operating processes. On the other hand, this method creates a reliable metric based entirely on statistics to reliably assess whether an update to a particular dataset is useful, i.e., whether it significantly contributes to improving vehicle localization.

[0018] In training mode, the vehicle is moved manually, specifically by the user. This means the user always has control over the vehicle. However, this does not preclude the possibility of using the vehicle's remote control and / or self-steering and / or autonomous driving systems, where even sensor-assisted decisions regarding changes in driving direction can be initiated and / or executed by the vehicle.

[0019] The received images are particularly those received by an in-vehicle camera (e.g., a front-facing camera). They can also be images composed of multiple images from different cameras and / or images acquired at different times. The received images can particularly include an extended spectral range; for example, the image may include optical information in the near-infrared and UV ranges. For example, the image contains spectral information between 2500 nm and 150 nm. Furthermore, the image may contain information within one or more well-defined spectral ranges, acquired, for example, using appropriate band filters or line filters, which can optimize the contrast used to determine the optical features in the corresponding image. The received images are particularly digital images that can be represented as a two-dimensional pixel matrix, where the pixel matrix may include multiple planes, each containing information, for example, from a specific spectral range. For example, in the case of a color image, three layers corresponding to the three captured color channels, particularly red, green, and blue (RGB), may be provided.

[0020] Optical features identified in an image may possess specific properties, such as particular contrast between adjacent pixels and / or across multiple pixels, or specific shapes, such as circles, angles, elongations, or wavy shapes. These optical features can be determined using different image processing methods and / or image transformations, and these methods and transformations can be combined in different orders. Furthermore, neural networks can be used, particularly for object classification of visible objects in the image.

[0021] Specifically, a particular feature is characterized by multiple parameters. These parameters specifically include: the feature's location in the image, defined by at least two coordinates, such as x and y values; the feature's "color"; the feature's shape; the feature's extent, which can be specified, for example, by the number of pixels it covers; the feature's classification, etc. For example, the "color" of a particular feature can be specified by specifying its intensity at a specific wavelength or its intensity (luminance information) for a specific filter. For example, the intensity is determined by the value of a pixel matrix entry associated with the pixel. The number of possible values ​​for the parameters ranges from binary ("0" or "1") to quasi-continuous, with no upper or lower bound. "Quasi-continuous" is because the data is processed digitally in the current case, which is why the parameter values ​​are quantized, even if the corresponding parameter itself is continuous in nature.

[0022] In a single image, it is preferable to identify at least 50 and up to 5000 optical features. It should be noted that a larger number of optical features requires a correspondingly larger amount of memory to store the dataset. On the other hand, while the accuracy of localization increases with the number of optical features, this improvement in accuracy plateaus as the number increases. Preferably, 100 to 500 optical features are identified for each image and stored in the dataset.

[0023] For example, a dataset may consist of a list or table of optical features, each assigned a corresponding parameter value. Not all determined optical features must include a value for every possible parameter; they may have values ​​that mark the parameter as "undefined".

[0024] Image reception and optical feature determination are performed in the same manner, specifically in training mode and following mode, using the same image processing methods. However, the accuracy when performing single or multiple computational operations may vary, for example, depending on available system resources. The possibility of adding new and / or other image processing steps and methods during the lifespan of the parking assistance system, as part of system updates, etc., cannot be ruled out. After a system update, these images and optical features will again be used in the same manner in training mode and following mode. This ensures that the same quality and / or type of results are achieved in both training mode and following mode.

[0025] If optical features have already been determined for the currently received image, the next step is to determine the first and second distributions of at least one parameter. In this case, the determined distributions are specifically probability distributions. Alternatively, the parameter values ​​can be said to be randomly distributed across all optical features of the given image, thus forming a random variable. The distribution of this random variable is a feature of the given image.

[0026] The determined distribution can be one-dimensional or multi-dimensional. For example, the local distribution of optical features in a two-dimensional image can be determined as a two-dimensional distribution. Multi-dimensional distributions are not limited to the same type of parameters (e.g., position coordinates), but can also be determined based on the parameter "position coordinates", the parameter "color", and / or other and / or additional parameters.

[0027] By comparing two distributions, their similarity can be determined. Distribution similarity corresponds to, for example, the intersection, common set, or overlap of the distributions. In the case of multidimensional distributions, similarity can be determined separately for each of the different dimensions (parameters) of the distribution.

[0028] Then, based on the obtained similarity of the distributions, it is determined whether to update the stored dataset. Specifically, the obtained similarity can be compared with an update threshold, where if the obtained similarity is lower than the update threshold, an update is performed.

[0029] If corresponding first and second distributions are determined for multiple parameters and their similarities are established, the overall similarity can be determined based on the obtained multiple similarities. The similarity values ​​of distributions for different parameters can be considered to varying degrees. For example, the similarity of the distribution at the y-position (vertical position) can be considered to a greater extent than the similarity of the distribution at the x-position (horizontal position), and vice versa.

[0030] Parking assistance systems are specifically designed for partially or fully autonomous vehicle operation, where, for example, in follow mode, the vehicle automatically follows a training trajectory. Semi-autonomous driving is understood to mean, for example, that the parking assistance system controls the steering equipment and / or the automatic gear selection system. Fully autonomous driving is understood to mean, for example, that the parking assistance system additionally controls the drive and braking systems. Vehicle orientation and / or positioning are performed, in particular, based on a comparison of determined optical features with a stored dataset. Based on the absolute and / or relative arrangement of the determined optical features with respect to each other, the displacement or relative position of the vehicle's current position relative to the corresponding position during training mode can be determined, and the vehicle can be controlled accordingly based on and along the training trajectory.

[0031] According to one embodiment of the method, the parameters of the optical features include the corresponding position of each feature in the image, the classification of the corresponding feature, the color of the corresponding feature, the geometry of the corresponding feature, the contrast value of the corresponding feature, etc.

[0032] As mentioned above, the parameter "color" represents, for example, the intensity (brightness) at a specific wavelength, within a specific band, and / or at multiple wavelengths. Furthermore, the parameter "color" can include the ratio of two or more intensities at different wavelengths.

[0033] Contrast value parameters can include pure intensity contrast, but can also include color contrast.

[0034] Specific optical characteristics are uniquely characterized by a specification of assigned or associated parameter values. For example, parameter values ​​can be arranged in a parameter vector, where the position in the vector identifies the parameter.

[0035] According to another embodiment of the method, the similarity between the first and second distributions is determined based on Bhattacharyya distance and / or KL (Kullback-Leibler) distance.

[0036] According to another embodiment of the method, steps A2)-A4) are performed on multiple locations along the training trajectory to store a corresponding dataset for each location. In the following mode, steps B3)-B5) are performed based on some stored datasets, in which the distance between the corresponding location and the current location of the vehicle is less than or equal to a predetermined distance threshold.

[0037] In this embodiment, the training trajectory is assigned multiple datasets with defined optical features, each dataset determined based on images captured at different locations along the trajectory. The current vehicle position is a useful indicator for comparing whether two distributions are meaningful. This is because if the locations of the images used to determine the optical features now or in the past differ too much, then the distributions are assumed to be dissimilar, since the images may show different parts or regions of the environment. Performing updates in this case would be detrimental, and this can be reliably avoided by comparing positions. This embodiment can also be referred to as a method for selecting datasets or distributions to be compared.

[0038] The vehicle's location can be determined, in particular, using a location sensor such as a GPS. Alternatively or additionally, an odometer can be used to determine the corresponding location. It should be noted that, in this example, the term "location" also includes the vehicle's orientation, which can be determined by a magnetic field sensor, for example, orientation relative to the Earth's magnetic field and / or an artificial horizon.

[0039] In various embodiments, in the following mode, only the datasets that are stored have the smallest distance between the corresponding position and the current position of the vehicle compared to other datasets of stored trajectories are used.

[0040] According to another embodiment of the method, steps A2)-A4) are performed on multiple locations along the training trajectory to store a corresponding dataset for each location. In the following mode, steps B3) and B4) are performed on all stored datasets, and step B5) is performed on certain datasets, where the similarity between the first and second distributions of these datasets is higher than a predetermined similarity threshold.

[0041] In this embodiment, the training trajectory is assigned to multiple datasets with defined optical characteristics, each dataset determined based on images captured at different locations along the trajectory. In this embodiment, the similarity of the distributions is used as a basis for determining whether images of the environment under a given distribution show comparable portions or regions from that environment. This embodiment can be particularly combined with location-based selection methods.

[0042] Therefore, in this embodiment, there are two predetermined thresholds for similarity: an update threshold and a similarity threshold.

[0043] According to another embodiment of the method, the similarity corresponding to the predetermined similarity threshold is lower than the predetermined update threshold.

[0044] This means that datasets with similarity values ​​above the similarity threshold but below the update threshold are updated.

[0045] For example, the similarity threshold has a value between 65% and 75%, and the update threshold has a value between 80% and 95%. If the similarity is between 75% and 80%, it is determined that the corresponding dataset needs to be updated. A value of 100% means that the distributions of the two comparisons are the same, and a value of 0% means that the distributions of the two comparisons have no overlap or commonality.

[0046] According to another embodiment of the method, step B5) is performed only on a dataset that has the highest similarity between the first and second distributions compared to all datasets of trajectories.

[0047] According to another embodiment of the method, a first random process for determining a first distribution of at least one parameter is performed based on the corresponding timestamps of images received in the training mode, and wherein a second random process for determining a second distribution of the parameter is performed based on the corresponding timestamps of images received in the following mode, and wherein step B5 is additionally and / or alternatively performed based on the similarity between the first random process and the second random process.

[0048] In this embodiment, the temporal evolution of the parameter distribution is determined along the trajectory, coupled to the vehicle's position along the trajectory via vehicle speed, and decisions related to updates are associated with this. Alternatively, in this case, time is treated as an additional parameter, allowing the temporal evolution of, for example, the position coordinate distribution to be represented as a two-dimensional distribution.

[0049] According to another embodiment of the method, the dataset update in step B5) is performed based on the optical features determined in step B2).

[0050] According to another embodiment of the method, the dataset update in step B5) includes: replacing the dataset with the current dataset and / or replacing at least one optical feature contained in the stored dataset and / or updating at least one parameter of the optical feature contained in the stored dataset.

[0051] According to the second aspect, a computer program product comprising commands is proposed, which, when a computer runs the program, cause the computer to perform the method according to the first aspect.

[0052] A computer program product, such as a computer program device, may be provided or supplied, for example, by a server on a network, in the form of a storage medium, such as a memory card, USB stick, CD-ROM, or DVD; or in the form of a downloadable file. This may be done, for example, by transmitting a corresponding file containing the computer program product or computer program device over a wireless communication network.

[0053] According to a third aspect, a parking assistance system for vehicles is proposed. This parking assistance system is configured to capture and store a training trajectory in a training mode, and is configured to follow the stored trajectory by vehicle in a following mode. The parking assistance system includes:

[0054] A receiving unit is configured to receive at least one image of the vehicle's environment in training mode when the vehicle is traveling along a training trajectory.

[0055] A first determining unit is configured to determine multiple optical features in the received image, wherein each optical feature is characterized by at least one parameter, and

[0056] Storage unit for storing a dataset including the determined optical features.

[0057] The receiving unit is configured to receive at least one current image of the vehicle environment while the vehicle is traveling along a trajectory in following mode, and the first determining unit is configured to determine optical features in the received current image. The parking assistance system also includes:

[0058] The second determining unit is configured to determine a first distribution of at least one parameter based on a stored dataset, and to determine a second distribution of that parameter based on determined optical features of the current image.

[0059] Comparison unit, used to determine the similarity between the first distribution and the second distribution, and

[0060] An update unit is used to update the stored dataset when the determined similarity is less than or equal to a predetermined update threshold.

[0061] This parking assistance system has the same advantages as the method described according to the first aspect. The embodiments and definitions set forth according to the method of the first aspect are applicable to this parking assistance system with necessary modifications.

[0062] Each unit of this parking assistance system can be implemented in hardware and / or software. In the case of hardware implementation, the corresponding unit can be, for example, a computer or microprocessor. In the case of software implementation, the corresponding unit can be in the form of a computer program product, function, routine, algorithm, partial program code, or runnable object. Furthermore, each unit mentioned herein can also be part of a higher-level control system of the vehicle, such as a central control system and / or an ECU (engine control unit).

[0063] According to the fourth aspect, a vehicle is proposed having at least one camera for detecting and outputting images of the vehicle environment, and having a parking assistance system according to the third aspect.

[0064] The vehicle is, for example, a car or even a truck. Preferably, the vehicle includes multiple sensor units configured to capture the vehicle's driving status and its surrounding environment. Examples of such sensor units are image capture devices, such as cameras, radar (radio detection and ranging) or lidar (light detection and ranging), ultrasonic sensors, position sensors, wheel angle sensors, and / or wheel speed sensors. The sensor units are configured to output sensor signals to, for example, a parking assistance system or driver assistance system that performs partially or fully autonomous driving based on the captured sensor signals.

[0065] Other possible embodiments of the invention include combinations of features or embodiments not explicitly mentioned in the descriptions above or below with reference to exemplary embodiments. In such cases, those skilled in the art will also add individual aspects as improvements or additions to the corresponding basic form of the invention. Attached Figure Description

[0066] Further advantageous configurations and aspects of the invention are the subject of the dependent claims and the exemplary embodiments of the invention described below. The invention will now be explained in more detail based on preferred embodiments with reference to the accompanying drawings.

[0067] Figure 1 A schematic diagram illustrating an exemplary embodiment of a vehicle is shown from a bird's-eye view.

[0068] Figure 2 A schematic diagram of a received image containing multiple optical features is shown;

[0069] Figure 3 Three schematic diagrams are shown as examples of the corresponding distributions as corresponding parameters;

[0070] Figure 4 An example of comparing two distributions to determine similarity is shown;

[0071] Figure 5 A schematic block diagram illustrating an exemplary embodiment of a parking assistance system is shown; and

[0072] Figure 6 A schematic flowchart illustrating an exemplary embodiment of a method for operating a parking assistance system is shown.

[0073] Unless otherwise stated, identical or functionally equivalent elements in the figures have the same reference numerals. Detailed Implementation

[0074] Figure 1 A schematic diagram of vehicle 100 is shown from a bird's-eye view. Vehicle 100 is, for example, a car arranged in environment 200. Car 100 has a parking assistance system 110, for example, in the form of a control unit. Furthermore, car 100 includes a front-facing camera 120, which in this example is arranged on the upper edge of the windshield. Camera 120 is configured to capture images of environment 200. The captured images are output to, for example, parking assistance system 110, which receives and further processes the images, as referenced below. Figure 2-6 Detailed description.

[0075] The parking assistance system 110 is configured to drive the car 100 semi-autonomously or even fully autonomously. In addition... Figure 1 In addition to the camera 120 shown, the vehicle 100 may also have a variety of other sensor devices. Examples of these sensor devices are ultrasonic sensors, lidar, radar, GPS receivers, accelerometers, receivers for receiving electromagnetically transmittable data signals, microphones, etc.

[0076] Figure 2 A schematic diagram of a received image IMG is shown, containing multiple optical features F1-F8. In this example, the image IMG is an image with two spatial dimensions, i.e., it comprises a two-dimensional pixel array. In various embodiments, it may also be an image with three spatial dimensions, additionally including depth information for each pixel. Each pixel of the image IMG is uniquely determined by its corresponding coordinates within the image IMG, where the corresponding coordinates are two positional coordinates, x and y. The coordinates x and y relate to an image coordinate system (not shown), the origin of which is, for example, located at one of the corners of the image IMG. Thus, the exemplary optical features F1-F8 shown have two positional parameters corresponding to the x and y coordinates of the respective optical feature. It should be noted that in this example, for clarity, the optical features F1-F8 are shown only as points in the image IMG, but specific optical features may also have ranges.

[0077] In addition to the position coordinates x and y, optical features F1-F8 are also characterized by a third parameter p, which is, for example, the color value of optical features F1-F8. In this example, a specific optical feature can therefore be uniquely characterized by the specification of the three parameters x, y, and p. For example, optical feature F1 can be represented by the specification F1(x1, y1, p1), where x1, y1, and p1 are the corresponding values ​​of the corresponding parameters of optical feature F1.

[0078] It should be noted that the specific optical features F1-F8 can be characterized by more than three parameters. It should also be noted that, preferably, in the image IMG, far more than eight optical features F1-F8 are determined; for example, 200 to 500 optical features are determined.

[0079] Specifically, each optical feature F1-F8 is a feature structure in the received image IMG, which can be determined, for example, in terms of contrast.

[0080] Figure 3 Three schematic diagrams are shown as examples of the corresponding distributions P(x), P(y), and P(p) given parameters x, y, and p. Each distribution represents a probability distribution or a frequency distribution, where the vertical axis P represents probability or frequency, and the corresponding horizontal axes x, y, and p represent the values ​​of the corresponding parameters x, y, and p.

[0081] In this example, distributions P(x) and P(y) are represented as (quasi-)continuous distributions, and distribution P(p) is represented as a discrete distribution. Since the corresponding values ​​of parameters x, y, and p are specifically quantized during the determination process by the determining unit of the parking assistance system, all distributions are, for example, discrete, but can also be referred to as quasi-continuous distributions. Here, data simplification can also be advantageous, for example, by performing a "discarding" process, where all values ​​within a certain interval are assigned to the average (e.g., in the case of a reduced bit depth of the parameters).

[0082] Figure 4 A schematic diagram of two distributions P1(x) and P2(x) containing parameter x is shown. Figure 4 This is used to illustrate the meaning of the similarity between two distributions P1(x) and P2(x). For example, the first distribution P1(x) is a first distribution of parameter x, which is determined based on the stored dataset, while distribution P2(x) is a second distribution of the same parameter x, which is based on the received current image IMG (see [link to IMG]). Figure 2 The determined optical characteristics F1-F8 (see) Figure 2The similarity between two distributions P1(x) and P2(x) can be represented as the intersection U(P1; P2) of the two distributions P1(x) and P2(x). For example, the number of equal parameter values ​​x in the two distributions P1(x) and P2(x) can be counted and divided by the total number of optical features F1-F8 to determine the similarity. Advantageously, the similarity between two distributions P1(x) and P2(x) is determined based on Bach's distance and / or KL distance. Other known similarity measures can also be used.

[0083] Figure 5 A schematic block diagram of an exemplary embodiment of a parking assistance system 110 is shown, for example, as... Figure 1 The parking assistance system 110 of vehicle 100 is configured in training mode MOD0 (see...). Figure 6 The training trajectory is captured and stored in the vehicle-following mode MOD1 (see [link]). Figure 6 In the training mode MOD0, the vehicle 100 follows a stored trajectory. The parking assistance system 110 includes a receiving unit 111 for receiving the environment 200 of the vehicle 100 (see [reference]) when the vehicle 100 is traveling along the training trajectory in training mode MOD0. Figure 1 At least one image of an IMG (see Figure 2 The first determining unit 112 is used to determine multiple optical features F1-F8 in the received image IMG (see...). Figure 2 Each optical feature F1-F8 is characterized by at least one parameter x, y, p (see...) Figure 2 or Figure 3 ); and storage unit 113 for storing a dataset including the determined optical features F1-F8.

[0084] The receiving unit 111 is further configured to receive at least one current image (IMG) of the environment 200 of the vehicle 100 while the vehicle 100 is traveling along a trajectory in following mode (MOD1), and the first determining unit 112 is further configured to determine optical features F1-F8 in the received current image (IMG). The parking assistance system 110 also includes a second determining unit 114 for determining a first distribution P(x), P(y), P(p), P1(x) of at least one parameter x, y, p based on a stored dataset (see...). Figure 3 or Figure 4 ), and used to determine the second distributions P(x), P(y), P(p), P2(x) of parameters x, y, and p based on the determined optical features F1-F8 of the current image IMG (see Figure 3 or Figure 4The comparison unit 115 is used to determine the similarity between the first distribution P(x), P(y), P(p), P1(x) and the second distribution P(x), P(y), P(p), P2(x); and the update unit 116 is used to update the stored dataset when the determined similarity is less than or equal to a predetermined update threshold.

[0085] Figure 6 An example of operating a parking assistance system 110 is shown (e.g. Figure 5 A schematic flowchart illustrating an exemplary embodiment of a parking assistance system 110 for vehicle 1. The method includes a training mode MOD0 and a following mode. In training mode MOD0, a trajectory to be trained is captured and stored; in following mode, the vehicle follows the stored trajectory.

[0086] Training mode MOD0 specifically includes steps S1-S4, wherein in the first step S1, vehicle 100 is manually driven along a trajectory, and in the second step S2, the environment 200 of vehicle 100 is received during the manual driving step (see...). Figure 1 At least one image of an IMG (see Figure 2 In the third step S3, multiple optical features F1-F8 in the received image IMG are determined (see...). Figure 2 The corresponding optical features F1-F8 are determined by at least one parameter x, y, p (see...). Figure 2 or Figure 3 The data is characterized, and in the fourth step S4, the dataset including the determined optical features F1-F8 is stored.

[0087] The following mode specifically includes steps S5-S9, wherein in step S5, at least one current image IMG of the environment 200 of the vehicle 100 is received during the following process; in step S6, optical features F1-F8 in the received current image IMG are determined; and in step S7, a first distribution P(x), P(y), P(p), P1(x) of at least one parameter x, y, p is determined based on the stored dataset (see [link to documentation]). Figure 3 or Figure 4 Furthermore, based on the determined optical features F1-F8 of the current image IMG, the second distributions P(x), P(y), P(p), and P2(x) of parameters x, y, and p are determined (see [link to image]). Figure 3 or Figure 4 In the eighth step S8, the similarity between the first distribution P(x), P(y), P(p), P1(x) and the second distribution P(x), P(y), P(p), P2(x) is determined, and in the ninth step S9, if the determined similarity is less than or equal to a predetermined update threshold, the stored dataset is updated.

[0088] It should be noted that the training mode MOD0 for a specific trajectory is executed only once, while the following mode MOD1 can be executed frequently as needed, based on the training trajectory.

[0089] It should also be noted that the following mode MOD1 may include other steps related to, for example, the control of the vehicle 100 by the parking assistance system 110.

[0090] Although the invention has been described based on exemplary embodiments, it can be modified in various ways.

[0091] Reference tag list

[0092] 100 vehicles

[0093] 110 Parking Assist System

[0094] 111 Receiving Unit

[0095] 112 Determine the unit

[0096] 113 storage units

[0097] 114 Determine the unit

[0098] 115 Comparison Unit

[0099] 116 Update Unit

[0100] 120 cameras

[0101] F1-F8 Optical Features

[0102] IMG images

[0103] MOD0 Training Mode

[0104] MOD1 Follow Mode

[0105] p parameter

[0106] p1-p8 parameter values

[0107] P probability

[0108] P(p) probability distribution

[0109] P(x) probability distribution

[0110] P1(x) First Distribution

[0111] P2(x) Second Distribution

[0112] P(y) probability distribution

[0113] S1-S9 Method Steps

[0114] U(P1;P2) intersection

[0115] x-coordinate

[0116] x1-x8 x position (parameter example)

[0117] y-coordinate

[0118] y1-y8 y-position (parameter example)

Claims

1. A method for operating a parking assistance system (110) for a vehicle (100), the parking assistance system being configured to capture and store a training trajectory in a training mode (MOD0) and to follow the stored trajectory by the vehicle (100) in a following mode (MOD1), wherein the training mode (MOD0) includes: A1) Manually drive (S1) the vehicle (100) along the said trajectory. A2) Receive (S2) at least one image (IMG) of the environment (200) of the vehicle (100) during manual driving. A3) Determine multiple optical features (F1-F8) in the received image (IMG) by (S3), wherein each optical feature (F1-F8) is characterized by at least one parameter (x, y, p), and A4) Storage (S4) includes the dataset of the determined optical features (F1-F8). And the following mode (MOD1) mentioned therein includes: B1) During the following process, receive (S5) at least one current image (IMG) of the environment (200) of the vehicle (100), B2) Determine (S6) the optical features (F1-F8) in the received current image (IMG), B3) Based on the stored dataset, determine (S7) a first distribution (P(x), P(y), P(p); P1(x)) of at least one parameter (x, y, p), and determine a second distribution (P(x), P(y), P(p); P2(x)) of the parameter (x, y, p) based on the determined optical features (F1-F8) of the current image (IMG). B4) Determine (S8) the similarity between the first distribution (P(x), P(y), P(p); P1(x)) and the second distribution (P(x), P(y), P(p); P2(x)), and B5) If the determined similarity is less than or equal to the predetermined update threshold, then update the dataset stored in (S9); In the training mode (MOD0), steps A2)-A4) are performed on multiple locations along the trajectory to be trained, thereby storing a corresponding dataset for each location. In the following mode (MOD1), steps B3) and B4) are performed on all the stored datasets, and step B5) is performed on datasets where the similarity between the first distribution (P(x), P(y), P(p); P1(x)) and the second distribution (P(x), P(y), P(p); P2(x)) of these datasets is higher than a predetermined similarity threshold.

2. The method as described in claim 1, characterized in that, The parameters (x, y, p) of the optical features (F1-F8) include the corresponding position of each feature in the image (IMG), the classification of the corresponding feature, the color of the corresponding feature, the geometry of the corresponding feature, and / or the contrast value of the corresponding feature.

3. The method as described in claim 1 or 2, characterized in that, The similarity between the first distribution (P(x), P(y), P(p); P1(x)) and the second distribution (P(x), P(y), P(p); P2(x)) is determined based on Bach's distance and / or KL distance.

4. The method as described in claim 1 or 2, characterized in that, In the training mode (MOD0), steps A2) - A4) are performed at multiple locations along the trajectory to be trained, thereby storing the corresponding dataset for each location, and where In the following mode (MOD1), steps B3)-B5) are performed based on certain stored datasets, where the distance between the corresponding positions of these stored datasets and the current position of the vehicle (100) is less than or equal to a predetermined distance threshold.

5. The method as described in claim 1 or 2, characterized in that, The similarity threshold corresponds to a similarity level lower than the update threshold.

6. The method as described in claim 5, characterized in that, Step B5) is performed only on the following datasets: the first distribution (P(x), P(y), P(p); P1(x)) and the second distribution (P(x), P(y), P(p); P2(x)) of the datasets have the greatest similarity to all datasets of the trajectory.

7. The method as described in claim 4, characterized in that, Based on the corresponding timestamps of the images (IMGs) received in the training mode (MOD0), a first random process is used to determine the first distribution (P(x), P(y), P(p); P1(x)) of the at least one parameter (x, y, P); and a second random process is used to determine the second distribution (P(x), P(y), P(p); P1(x)) of the parameter (x, y, P); based on the corresponding timestamps of the images (IMGs) received in the following mode (MOD1); and step B5 is performed additionally and / or alternatively based on the similarity between the first random process and the second random process.

8. The method as described in claim 1 or 2, characterized in that, The dataset update in step B5) is performed based on the optical features (F1-F8) determined in step B2).

9. The method as described in claim 1 or 2, characterized in that, The dataset update in step B5) includes: replacing the dataset with the current dataset and / or replacing at least one optical feature (F1-F8) contained in the stored dataset and / or updating at least one parameter (x, y, p) of the optical feature (F1-F8) contained in the stored dataset.

10. A computer program product comprising commands that, during program execution by a computer, cause the computer to perform the method as claimed in any one of claims 1 to 9.

11. A parking assistance system (110) for a vehicle (100), configured to capture and store a training trajectory in a training mode (MOD0) and configured to follow the stored trajectory by the vehicle (100) in a following mode (MOD1), wherein the parking assistance system (110) comprises: The receiving unit (111) is configured to receive at least one image (IMG) of the environment (200) of the vehicle (100) as step A2 when the vehicle is traveling along the training trajectory in the training mode (MOD0). The first determining unit (112) is used to determine multiple optical features (F1-F8) in the received image (IMG) as step A3), wherein each optical feature (F1-F8) is characterized by at least one parameter (x, y, p), and Storage unit (113) for storing a dataset including the determined optical features (F1-F8) as part of step A4), The receiving unit (111) is configured to receive at least one current image (IMG) of the environment (200) of the vehicle (100) as step B1) when the vehicle (100) is traveling along the trajectory in the following mode (MOD1), and the first determining unit (112) is configured to determine the optical features (F1-F8) in the received current image (IMG) as step B2). And the parking assistance system (110) further includes: The second determining unit (114) is configured to determine a first distribution (P(x), P(y), P(p); P1(x)) of at least one parameter (x, y, p) based on the stored dataset, and to determine a second distribution (P(x), P(y), P(p); P2(x)) of the parameter (x, y, p) based on the determined optical features (F1-F8) of the current image (IMG) as step B3). The comparison unit (115) is used to determine the similarity between the first distribution (P(x), P(y), P(p); P1(x)) and the second distribution (P(x), P(y), P(p); P2(x)) as step B4), and The update unit (116) is used to update the stored dataset as step B5 when the determined similarity is less than or equal to a predetermined update threshold. In the training mode (MOD0), steps A2)-A4) are performed on multiple locations along the trajectory to be trained, thereby storing a corresponding dataset for each location. In the following mode (MOD1), steps B3) and B4) are performed on all the stored datasets, and step B5) is performed on datasets where the similarity between the first distribution (P(x), P(y), P(p); P1(x)) and the second distribution (P(x), P(y), P(p); P2(x)) of these datasets is higher than a predetermined similarity threshold.

12. A vehicle (100) having at least one camera (120) for capturing and outputting an image (IMG) of the environment (200) of the vehicle (100), and having a parking assistance system (110) as claimed in claim 11.

Images