Method for training an AI for a driver assistance system for a semi-trailer truck

Abstract

The invention relates to a method for training an artificial intelligence for the automated selection and approach of a semi-trailer truck (16) to a loading edge and / or loading ramp using a driver assistance system (10), in which a neural network (18) is provided for analyzing an image dataset of loading edges (12) and / or loading ramps (14), wherein partial obstruction of the loading edge (12) and / or the loading ramp (14) by a trailer (16a) of the semi-trailer truck (16) is emulated, and wherein specific vehicle parameters are assigned to each image of the dataset, by which the neural network (18) is trained. Furthermore, the invention relates to a method for performing an automated selection and approach of a semi-trailer truck (16) to a loading edge and / or loading ramp using a trained artificial intelligence via a driver assistance system (10). The invention further relates to such a driver assistance system (10).

Description

[0001] The invention relates to a method for training an artificial intelligence for the automated selection and approach of a semi-trailer truck to a loading edge and / or loading ramp using a driver assistance system according to claim 1. Furthermore, the invention relates to a method for performing an automated selection and approach of a semi-trailer truck to a loading edge and / or loading ramp using a trained artificial intelligence by means of a driver assistance system. Furthermore, the invention relates to such a driver assistance system.

[0002] DE 10 2022 118 083 A1 discloses a method for determining a distance between a motor vehicle and a loading ramp, as well as an electronic computing device. This method involves calculating the distance between a trailer and a loading edge based on geometric relationships in an image captured by one or more mirror cameras.

[0003] In addition, methods for training neural networks are known.

[0004] The object of the invention is to provide a method by means of which a particularly safe return trajectory is offered as a driving aid for a driver.

[0005] This object is achieved by means of a method having the features of patent claim 1 and by means of a method having the features of patent claim 3 and by means of a driver assistance system having the features of patent claim 6. Advantageous embodiments and further developments of the invention can be found in the dependent claims, wherein all means of the driver assistance system can be used to carry out the respective method steps.

[0006] A first aspect of the invention relates to a method for training an artificial intelligence for the automated loading edge and / or loading ramp selection and approach of a semi-trailer by means of a driver assistance system, in which a neural network is provided for the analysis of an image data set of loading edges and / or loading ramps, wherein a partial occlusion of the loading edge and / or the loading ramp by a trailer of the semi-trailer is emulated and wherein each image of the data set is assigned certain vehicle parameters by which the neural network is trained.

[0007] The invention thus aims to provide a method that enables a particularly safe reversing trajectory as a driving aid for a driver. The first aspect of the invention relates to the training of an artificial intelligence (AI) for the automated selection and approach of loading edges and / or loading ramps of a semi-trailer truck using the driver assistance system. For this purpose, a neural network is used that analyzes an image dataset of loading edges and / or loading ramps.

[0008] The driver assistance system runs on an electronic processing unit and is electronically linked to other required components. These components can either be located on the semi-trailer or designed for this purpose. Partial obscuration of the loading sill and / or loading ramp by the semi-trailer's trailer is emulated. This means that partial obscuration is simulated if an optical detection device such as a camera cannot detect the loading sill or loading ramp.

[0009] For each image in the dataset, specific vehicle parameters are assigned, which are used to train the neural network. The images are captured by optical cameras or other optical capture devices and transmitted to the electronic computing device. There, they are analyzed using AI to train the neural networks. The vehicle parameters are captured by appropriate capture devices such as sensor devices, control devices, optical capture devices, GPS, or others and transmitted to the electronic computing device to train the AI ​​and the neural network.

[0010] The goal is to autonomously select and approach loading edges and / or loading ramps in order to calculate and drive along a particularly safe return trajectory. This is intended to support the driver.

[0011] In an advantageous embodiment of the invention, training is performed based on various vehicle parameters that are assigned to each image in the data set. These vehicle parameters include: Position and / or orientation of the loading sill and / or loading ramp: The exact position and orientation of the loading sill and / or loading ramp are recorded and used as vehicle parameters.

[0012] “Bounding box” of the entire loading edge and / or loading ramp: A bounding box that encloses the entire loading edge and / or loading ramp is defined and used as a vehicle parameter.

[0013] Height of the loading sill and / or loading ramp above the ground: The vertical height of the loading sill and / or loading ramp above the ground is recorded and used as a vehicle parameter.

[0014] Width of the accessible loading edge and / or loading ramp: The width of the area that the semi-trailer can drive through is determined and used as a vehicle parameter.

[0015] Orientation of the loading edge and / or loading ramp in the image: The orientation of the loading edge and / or loading ramp within the image is recorded and used as a vehicle parameter.

[0016] Distance from the point where the image was taken to the loading edge and / or loading ramp: The distance between the point where the image was taken and the loading edge and / or loading ramp is measured and used as a vehicle parameter.

[0017] Free space of a loading bay: The available area in the loading bay that the semi-trailer can drive through is recorded and used as a vehicle parameter.

[0018] By training based on these vehicle parameters, the neural network can learn to select and approach the optimal loading sills and / or loading ramps for the semi-trailer truck in order to calculate a safe reversing trajectory and assist the driver when reversing.

[0019] A second aspect of the invention relates to a method for performing automated loading edge and / or loading ramp selection and approach of a semi-trailer truck using trained artificial intelligence by means of a driver assistance system according to the first aspect. After the neural network has been trained with the image dataset of loading edges and / or loading ramps, it can be used in the driver assistance system to enable automated selection and approach of loading edges and / or loading ramps.

[0020] The procedure may include the following steps: Detection of the current environment: The driver assistance system detects the current environment of the tractor-trailer, possibly using cameras, sensors or other optical detection devices.

[0021] Image analysis: The captured images are transmitted to the electronic computing device, where the trained neural network analyzes the images. The neural network identifies and recognizes the loading edges and / or loading ramps in the images based on the trained or learned patterns and features.

[0022] Selection of the loading edge and / or loading ramp: Based on the image analysis, the driver assistance system uses AI to select the loading edge and / or loading ramp the tractor-trailer should approach. This may take previously trained vehicle parameters into account to ensure a safe and appropriate selection.

[0023] The process thus enables automated selection and approach of loading edges and / or loading ramps by the semi-trailer with the help of the trained driver. This supports the driver when reversing and contributes to increased safety and efficiency of the reversing process.

[0024] In an advantageous embodiment of the invention, the automatic selection of a loading edge and / or loading ramp is followed by an automated driving maneuver for approaching the loading edge and / or loading ramp. This initiates the approach to the selected loading edge and / or loading ramp, with the driver assistance system automatically steering the semi-trailer to the selected loading edge and / or loading ramp along a calculated trajectory. Steering, braking, and other vehicle functions can be controlled accordingly to enable a precise and safe approach.

[0025] In another advantageous embodiment of the invention, the driving maneuver is defined as a docking according to a predetermined trajectory, during which contact is made with the loading edge and / or loading ramp. The driver assistance system steers the semi-trailer along a predefined trajectory, which in particular guides the semi-trailer safely to the loading edge and / or loading ramp. The steering, braking, and other vehicle functions are controlled accordingly to execute the desired movement. Contact with the loading edge and / or loading ramp occurs during the docking process. This can be achieved, for example, by precise reversing or other suitable maneuvers. The driver assistance system monitors the process and, if necessary, adjusts the vehicle movements to establish safe and stable contact.

[0026] The defined docking maneuver, following a predefined trajectory, enables reliable positioning of the semi-trailer truck relative to the loading edge and / or loading ramp. This enables efficient loading and unloading and reduces the risk of collisions or damage.

[0027] In yet another advantageous embodiment of the invention, the area obscured by the trailer is displayed on a display device of at least one mirror-replacement camera using "content-based image filling." With conventional rearview camera systems, the area obscured by the trailer may be invisible to the driver, which can lead to restricted visibility and potential safety risks. To solve this problem, this embodiment of the invention uses "content-based image filling."

[0028] The "content-based image filling" technique makes it possible to reconstruct the hidden area in real time using image processing algorithms. The existing image information from the visible area is used to supplement and display the hidden area. This is done by generating image content based on the surrounding structures and textures.

[0029] The display device, such as a mirror-replacement camera, then shows the driver the combined image, which supplements the obscured area using content-based image filling technology. This gives the driver an expanded view and allows them to better assess the area behind the trailer.

[0030] This embodiment of the invention improves the driver's visibility and situational awareness while reversing, particularly in situations where the trailer obscures parts of the surroundings.

[0031] A third aspect of the invention relates to a driver assistance system for performing a method for performing automated loading edge and / or loading ramp selection and approach of a semitrailer truck using a trained artificial intelligence according to the first aspect. The driver assistance system comprises an electronic computing device equipped with all the necessary circuits, components, and couplings to other components of the semitrailer truck to provide the driver assistance system.

[0032] The electronic processing unit serves as the core of the driver assistance system and enables the processing of image data, analysis by the trained neural network, and control of the vehicle functions. It is connected to optical detection devices, such as cameras, to capture the surroundings and transmit the images to the processing unit for analysis. The driver assistance system can also feature a display device that can show the driver relevant information, including the area obscured by the trailer, using content-based image filling.

[0033] The solution according to the invention has several differences from the identified prior art: Consideration of partial obscuration: The prior art does not take into account that the automated approach of a semi-trailer truck or box truck may be partially obscured by the trailer. The invention, however, solves this problem by offering a solution for selecting and approaching the loading edge despite obscuration.

[0034] Automatic selection by a trained neural network: Unlike the conventional method, in which the driver selects the loading edge, the invention performs the selection automatically by a trained neural network. This reduces the risk of errors and provides a robust, reliable selection.

[0035] Increased sense of security: By using content-based image filling to fill and display the area obscured by the trailer, the driver's sense of security is increased. This provides the driver with a broader view and allows them to better assess the obscured area, leading to increased safety during the start-up process.

[0036] Further advantages, features, and details of the invention will become apparent from the following description of a preferred embodiment and from the drawings. The features and combinations of features mentioned above in the description, as well as the features and combinations of features mentioned below in the description of the figures and / or shown alone in the figure, can be used not only in the respective combinations specified, but also in other combinations or on their own, without departing from the scope of the invention.

[0037] The only Fig. 1 is a picture diagram illustrating a method according to the invention for carrying out an automated loading edge and / or loading ramp selection and approach of a semitrailer using a trained artificial intelligence by means of a driver assistance system.

[0038] Fig. 1 shows an image diagram illustrating the method according to the invention for carrying out an automated loading edge and / or loading ramp selection and approach of a semitrailer truck 16 using a trained artificial intelligence by means of a driver assistance system 10.

[0039] In this method, a neural network 18 is used to analyze an image dataset of loading sills 12 and / or loading ramps 14. Partial occlusion of the loading sill 12 and / or loading ramp 14 by a trailer 16a of the semi-trailer 16 is emulated. This means that the partial occlusion is simulated to capture realistic scenarios.

[0040] Each image in the data set is assigned specific vehicle parameters that are used to train the neural network 18. These vehicle parameters include the position and / or orientation of the loading edge 12 and / or loading ramp 14, the bounding box of the entire loading edge 12 and / or loading ramp 14, the height of the loading edge 12 and / or loading ramp 14 above the ground, the width of the approachable loading edge 12 and / or loading ramp 14, the orientation of the loading edge 12 and / or loading ramp 14 in the image, the distance from the image capture point to the loading edge 12 and / or loading ramp 14, and the free area of ​​a loading bay.

[0041] Another method enables automated selection and approach of the loading sills and / or loading ramps by the semitrailer 16 using trained artificial intelligence. The driver assistance system 10, which runs on an electronic computing device 22, uses the neural network 18 to analyze the images and select the optimal loading sill 12 and / or loading ramp 14. The driver assistance system 10 then steers the semitrailer 16 along a predetermined trajectory T to the selected loading sill 12 and / or loading ramp 14.

[0042] An optical detection device 20 can be coupled to the electronic computing device 22 and detects an optical detection area 20a. The optical detection device 20 can, for example, be a camera or a sensor system capable of detecting visual information from the environment. It is electronically connected to the electronic computing device 22 in order to transmit the detected images or data to it.

[0043] This process enables safe and efficient approach to the loading edge 12 and / or loading ramp 14. The automated selection and approach is based on training the neural network 18 with the specific vehicle parameters, resulting in increased safety and accuracy of the process. List of reference symbols 10 Driver assistance system 12 Loading sill 14 Loading ramp 16 semi-trailer trucks 16a Trailer 18 neural network 20 optical detection device 20a optical detection range 22 electronic computing device T trajectory QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] DE 10 2022 118 083 A1

[0002]

Claims

[1] Method for training an artificial intelligence for the automated loading edge and / or loading ramp selection and approach of a semitrailer truck (16) by means of a driver assistance system (10), in which a neural network (18) is provided for analyzing an image data set of loading edges (12) and / or loading ramps (14), wherein a partial occlusion of the loading edge (12) and / or the loading ramp (14) by a trailer (16a) of the semitrailer truck (16) is emulated and wherein each image of the data set is assigned certain vehicle parameters by means of which the neural network (18) is trained. [2] Method according to claim 1, characterized by , that training based on: - as the position and / or orientation of the loading edge (12) and / or loading ramp (14); and / or - as a “bounding box” of the entire loading edge (12) and / or loading ramp (14); and / or - as the height of the loading edge (12) and / or loading ramp (14) above the ground; and / or - as the width of the approachable loading edge (12) and / or loading ramp (14); and / or - as orientation of the loading edge (12) and / or loading ramp (14) in the image; and / or - as the distance from the point of capture of the image to the loading edge (12) and / or loading ramp (14); and / or - as open space in a loading bay defined vehicle parameters, which are assigned to each image of the data set. [3] Method for carrying out an automated loading edge and / or loading ramp selection and approach of a semitrailer truck (16) using a trained artificial intelligence by means of a driver assistance system (10) according to claims 1 to 2. [4] Method according to claim 3, characterized bythat upon automatic selection of a loading sill (12) and / or loading ramp (14), an automated driving maneuver for approaching the loading sill and / or loading ramp is carried out. [5] Method according to claim 4, characterized by that the driving maneuver is defined as a docking according to a predetermined trajectory, in which contact is made with the loading edge (12) and / or loading ramp (14). [6] Method according to claims 2 to 4, characterized by that the area concealed by the trailer is displayed in a display device of at least one mirror replacement camera by means of “content based image filling”. [7] Driver assistance system for carrying out a method for carrying out an automated loading edge and / or loading ramp selection and approach of a semitrailer truck (16) using a trained artificial intelligence according to claim 3.

Images