Computer-implemented vehicle-side method for driver assistance, computer program, control unit for a vehicle, vehicle, computer-implemented server-side method for determining the driving trajectory, server computer program and server device

Abstract

A computer-implemented method for driver assistance, comprising the following steps: acquisition of sensor data from the vehicle's environment using at least one vehicle sensor; transmission of an output radio signal representing at least a part of the acquired sensor data; reception of a result radio signal representing at least one driving trajectory determined for the vehicle's journey as a function of the output radio signal; and display of the at least one received driving trajectory and / or control of at least one steering motor of the vehicle and / or at least one drive of the vehicle to drive the vehicle along the received driving trajectory.

Description

The present invention relates to a computer-implemented vehicle-side method for driving assistance or for displaying a received driving trajectory and / or for controlling a vehicle along the received driving trajectory. The invention also relates to a computer program comprising instructions which, when the program is executed by a computer, cause the computer to execute the steps of this vehicle-side method. The invention further relates to a control device for a vehicle, comprising a computing unit which is configured to execute the steps of this vehicle-side method. In addition, the invention also relates to a vehicle comprising the control device according to the invention. The invention further relates to a computer-implemented server-side method for determining the travel trajectory of the vehicle by means of a server device. The invention also relates to a server computer program comprising instructions which, when the program is executed by a server device, cause the latter to execute the steps of this method on the server side. The invention further relates to a server device having a server computing unit configured to perform the steps of this server-side method.Prior ArtDocument WO 2019 / 050873 A1 discloses a system consisting of a data processor; and a trajectory planning module configured to generate a trajectory proposal for an autonomous vehicle.Document DE 10 2020 111 938 A1 discloses a method for planning and updating a vehicle trajectory.The document EP 3 616 182 A1 discloses a computing system, having a computing device of a motor vehicle and a cloud comprising a computing device. The application which calculates output data from input data and can be executed partly by the motor vehicle-side computing device and partly on the cloud side.Document DE 10 2022 119 206 A1 discloses a method comprising identifying a scenario for a first vehicle at least based on the analysis of sensor data generated by at least one sensor of the first vehicle in an environment; receiving a waiting element connected to the scenario, wherein the waiting element encodes a first path for the first vehicle for traversing an area in the environment and a second path for a second vehicle for traversing the area; determining, from the first path, a first trajectory in the area for the first vehicle at least based on a first position of the first vehicle at a time; determining, from the second path, a second trajectory in the area for the second vehicle at least based on a second location of the second vehicle at the time; judging whether there is a conflict between the first trajectory and the second trajectory; and transmitting data that causes the first vehicle to operate according to a waiting state based at least on the judgment, the waiting state defining a driving behavior for the first vehicle.According to the document Addendum 78: UN Regulation No. 79, concerning steering systems for vehicles, for parking assist systems which detect an obstacle (e.g. vehicles, pedestrians) in the manoeuvring area, the vehicle is to be immediately brought to a standstill in order to avoid a collision. The latency requirements according to ISO 17386 MALSO (Mannoeuving Aids for Low Speed Operation) and UNDECE R158 relate to the reaction times of driver assistance systems that support at low speeds, such as during parking or maneuvering. These standards and regulations define how fast these systems must react to sensor data and perform appropriate warnings or interventions in order to ensure the safety and effectiveness of the driver assistance systems. These latency requirements ensure that the systems react fast enough to prevent or mitigate potential collisions. Too long a latency could result in warnings getting too late or the system not intervening in time to avoid a collision. The exact latency values may vary depending on the specific application and the details of the respective standard or regulation. Generally, however, the aim is for the lowest possible latency in order to maximize the safety and effectiveness of the driver assistance systems. A latency period for parking assistants could therefore be, for example, 500 milliseconds, this latency period being required only for collision avoidance. In other words, the determination of a parking trajectory after detection of, for example, a parking space situated ahead in the direction of travel may last quite longer, wherein this is unlikely to be noticeable to the user, in particular during a slow travel of the vehicle.In the context of continuous evolution of wireless technologies, the fifth (5G) as the sixth generation (6G) extends the spectrum of communication possibilities. The introduction of 5G into 2019 allows theoretical transmission speeds of up to 10 gigabits per second, which is more than thirty-fold increase compared to 4G LTE technologies. The fifth generation uses frequency bands in the range of 24 GHz to 66 GHz. The sixth generation (6G) market introduction is expected for the year 2030. The sixth generation achieves transmission speeds of 206.25 gigabits per second in test trials. 6G aims at utilizing frequency bands in the spectrum of 30 GHz to 300 GHz. While 5G reaches latencies of about 5 milliseconds, a further reduction to only one millisecond is expected for 6G.Processing times on a server or cloud refer to the time required to process a request and generate a response. This time duration depends on the complexity of the request, the current load on the server, the efficiency of the algorithms and the performance of the hardware. It can be, for example, in the range from milliseconds to several seconds.The document Drews, F. et al. (2022) "DeepFusion: A Robust and Modular 3D Object Detector for Lidars, Cameras and Radars", arXiv:2209.12729v2 [cs.CV] discloses a sensor fusion as so-called deepFusion, which comprises a modular multimodal architecture of learned machine recognition methods or neural networks in order to use different sensor types in different combinations for 3D object recognition and map creation. Features or output values extracted from the neural networks are advantageously segmented and / or assigned to a spatial map in bird's eye view based on distance, which corresponds to a feature space. The features or feature vectors determined from the neural network are thus transformed and assigned in bird's-eye view in a common space, wherein individual cells of the space can be provided, which can comprise a plurality of features or features and, for example, also in each case height information relating to the features. Finally, a superordinate, learned machine recognition method uses the multimodal features from this feature space as a recognition head for recognizing objects and / or for creating a map, the recognition head can comprise, for example, a transformer architecture or similar approaches.The object of the present invention is to reduce the costs of a vehicle which has driving assistance systems with higher degrees of automation and / or to improve the driving assistance, for example with regard to automatic parking or parking-out.Disclosure of the InventionThe above object is achieved according to the invention according to independent claims 1, 9 to 12, 17 and 18.The invention relates to a computer-implemented vehicle-side method for driving assistance or for displaying a received driving trajectory and / or for controlling a vehicle along the received driving trajectory. First, in an optional first step of the method, a registration method can be carried out between the vehicle and a server device, advantageously in a mobile radio communication standard, wherein the registration method is carried out in particular as a function of a start of a drive of the vehicle.The registration method is advantageously configured to provide the vehicle with computing capacities on the server device for determining a travel trajectory for the vehicle, so that, in the case of a corresponding request from the vehicle or during the travel of the vehicle, the travel trajectory can be calculated quickly by means of the server device or a low latency period results with respect to the communication between the vehicle and the server device. The method comprises the acquisition of sensor data of the environment of the vehicle by means of at least one vehicle sensor. The sensor data are preferably captured by a vehicle sensor system which comprises at least two vehicle sensors, for example in each case at least one camera sensor, radar sensor and / or ultrasonic sensor. The vehicle sensor system particularly preferably comprises a radar sensor at each vehicle corner, i.e. in total four radar sensors, a vehicle front camera which captures the environment in front of the vehicle, and a rear camera directed rearward in the direction of the longitudinal axis of the vehicle, and at least two ultrasonic sensors in each case at the front and rear or alternatively in each case at least one ultrasonic sensor array at the front and rear. The sensor data are advantageously transmitted from the respective sensors by means of at least one bus system of the vehicle to a control device of the vehicle, in particular a central computing device of the vehicle, and are preferably stored at least for a predefined period of time. In a further step of the method, an output radio signal is transmitted, which represents at least a part of the acquired sensor data. It can be provided that the output radio signal is transmitted in the direction of travel of the vehicle as a function of a parking space detected based on the detected sensor data and / or as a function of a bottleneck detected based on the detected sensor data and based on navigation data of the vehicle, in particular the position of the vehicle. In other words, the output radio signal is advantageously transmitted automatically as a function of a detected driving situation of the vehicle, wherein the driving situation is detected in particular on the basis of the detected sensor data and / or a detected input of the user and / or detected operating parameters of the vehicle. The output radio signal represents the acquired sensor data, for example, as raw data (digital or analog), wherein preprocessing or processing of the raw data and / or compression of the raw data can be carried out. The output radio signal can alternatively or additionally represent the acquired sensor data as envelopes, as distance data determined from the acquired sensor data and / or further operating data. The output radio signal corresponds in particular at least to the 5G or 6G mobile radio standard and is advantageously transmitted to the corresponding mobile radio towers of the 5G or 6G infrastructure. The output radio signal preferably represents only the sensor data, which are optionally preprocessed, and in particular the feature vector determined based on the sensor data. The output radio signal, however, advantageously still does not represent detected objects and / or segmentations, such as a parking space. A result radio signal is then received, which in particular likewise corresponds at least to the 5G or 6G mobile radio standard. Between the transmission of the output radio signal and the reception of the result radio signal, a time period or latency period of less than or equal to 3 seconds, less than or equal to 2 seconds, less than or equal to 1 second or particularly preferably less than or equal to 500 milliseconds results. The result radio signal represents at least one travel trajectory ascertained for the travel of the vehicle as a function of the output radio signal. The driving trajectory is determined in particular for parking the vehicle in or out of the vehicle and / or for maneuver assistance of the vehicle at low speeds. Subsequently, in a further step of the method, the at least one received travel trajectory is displayed. As a result, the driver is assisted when driving the vehicle, in particular when parking, for example he can advantageously follow the displayed travel trajectory. Alternatively or additionally, at least one steering motor of the vehicle and / or at least one drive of the vehicle is actuated for partially automatic or autonomous travel of the vehicle along the received travel trajectory. As a result, the travel of the vehicle is very comfortable for the user, in particular with regard to the parking or unparking of the vehicle. In any case, the method advantageously saves computing resources in a control device of the vehicle or in the central computing device of the vehicle, as a result of which hardware costs for providing automatic driving assistance functions are reduced. In addition, the method allows the ongoing improvement in the trajectory determination, the automatic adaptation of the trajectory to further road users and / or the automatic adaptation of the trajectory to other server data, such as map data and / or published free parking spaces, and the increasing provision of additional driving assistance functions for the same control unit or central computing device of the vehicle or the same hardware in the vehicle. This also provides for improved long term discovery of new business models.In a particularly preferred embodiment of the invention, at least one feature vector or feature vector is determined by a computing unit of the vehicle as a function of at least a part of the captured sensor data. It can optionally be provided that the components or features of the feature vector are segmented and / or assigned to a two- or three-dimensional space on a distance basis. The feature vector can thus be represented as a two- or three-dimensional space in which the components are assigned on a distance basis. The feature vector is determined in particular by means of a learned machine recognition method, preferably by at least one neural network per sensor type in each case. Advantageously, at least one feature vector per sensor type is determined in each case, preferably by at least one neural network per sensor type in each case. The neural network for determining a feature vector comprises in particular a pyramidal architecture (feature pyramid network, FPN). Each specific feature vector is advantageously multidimensional, in particular non-human interpretable, and represents an understanding of the learned machine recognition method of the captured sensor data, which has been trained to the learned machine recognition method advantageously by training with labeled and / or unlabeled training data, advantageously by joint training with sensor data of a plurality of sensor types and / or by joint training via the next higher-height human interpretable layer, comprising a second learned machine recognition method as a recognition head for recognizing static and / or movable objects and / or creating a map of the environment. The training data advantageously comprise sensor data of a multiplicity of journeys of different vehicles acquired in the past, but in particular of the same vehicle type. The transmitted output radio signal comprises at least the determined feature vector as representation of the captured sensor data, in particular a feature vector per sensor type. This configuration compresses the sensor data or reduces the data volume of the output radio signal, as a result of which the duration of the transmission of the output radio signal is reduced, which in turn advantageously enables a rapid trajectory determination corresponding to the safety-critical requirements, for example for changing lanes and / or turning on a journey at relatively high speeds. Compared to the above basic embodiment, the reaction time of the vehicle is advantageously increased, since the travel trajectory is received more quickly.In an advantageous alternative embodiment of the invention, static and / or dynamic objects and / or a current map of the environment are determined on the basis of the at least one determined feature vector by means of a second computing unit of the vehicle, in particular by means of a second learned machine recognition method, preferably by means of a second neural network, which can comprise transformer architectures or network architectures related thereto, for example. In this embodiment, the transmitted output radio signal represents at least the static and / or dynamic objects and / or the ascertained map of the environment as a representation of the acquired sensor data. In other words, in this alternative embodiment, the transmitted output radio signal does not necessarily represent the feature vector. In this configuration, the necessary computing resources in the vehicle are increased, but the sensor data is compressed to a greater extent than in the previous embodiment or the data volume of the output radio signal is reduced to a great extent, as a result of which the duration of the transmission of the output radio signal can be reduced further. This increases the reaction time of the vehicle since the travel trajectory is received very quickly.In an advantageous embodiment of the invention, the vehicle-side method comprises, after the transmission of the output radio signal, a reduction in the speed of the vehicle until the reception of the result radio signal. The reduction of the speed takes place in particular on the basis of a predefined speed value. The speed is alternatively reduced on the basis of an expected calculation time assigned to the current driving situation, wherein the current driving situation is detected as a function of the acquired sensor data. Alternatively or additionally, the speed is reduced as a function of the current server load, which is advantageously received by the server device by radio signal at regular intervals. For calculations of a parking trajectory, therefore, for example, the speed can be reduced relatively more strongly than for the calculation of a trajectory for passing through a narrow point in the direction of travel. This embodiment advantageously ensures that the starting point of the trajectory is not reached or passed at an early stage when the result radio signal is received and the vehicle does not have to be brought to a standstill in order to wait for the result radio signal to be received.In a preferred further development of the vehicle-side method, it is provided that an emergency braking situation is determined as a function of the captured sensor data by an emergency braking algorithm, wherein the determination of the emergency braking situation is carried out by means of a computing device of the vehicle. Subsequently, at least one braking device of the vehicle is actuated as a function of the determined emergency braking situation. In other words, a vehicle-based emergency braking assistant is activated, which is independent of the communication with the server device, so that collisions can be reliably prevented even in the case of a high latency period or waiting time for the result signal. Optionally, automatic return of the vehicle along the already traveled travel trajectory in the opposite direction is then carried out as a function of the determined emergency braking situation by actuating the steering motor and / or the drive. This continuation increases the safety of the vehicle. In other words, the emergency braking assistant remains completely carried out in the vehicle, only the trajectories for driving assistance functions which are not relevant to collisions are carried out on a server basis. The emergency braking assistant is advantageously prioritized before other driving assistance functions are carried out, that is to say the emergency braking assistant can intervene, for example, during the parking process or during a lane change, each with a received trajectory if an emergency braking situation is determined on the basis of the captured sensor data.In another continuation, a current speed of the vehicle is detected. Alternatively or additionally, a first input of the driver for carrying out a desired autonomous driving function of the vehicle is detected by means of an input device of the vehicle. Alternatively or additionally, a position of the vehicle is detected by means of a position sensor. In this further development, the acquisition of the sensor data, the transmission of the output radio signal, the display of the travel trajectory and / or the actuation of the steering motor and / or the actuation of the drive is carried out additionally in each case based on the acquired speed, in particular if the magnitude of the acquired speed is less than or equal to a speed threshold value, based on the first input of the driver for carrying out a desired autonomous travel function and / or depending on the acquired position of the vehicle, in particular if the position is within a predefined surrounding range. This results in the advantage that, for example, the computing resources of the server are reduced, since the method is only completely carried out if this is potentially required. Alternatively, the advantage results that the user is supported only when this is potentially required. The distraction of the user is thus reduced or the comfort of the user is increased.In a further refinement, a second input by the driver is carried out in order to confirm the performance of the desired autonomous driving function of the vehicle by means of the first input device and / or a second input device of the vehicle. In this embodiment, the steering motor and / or the drive are additionally controlled as a function of the detected second input. The first input device can be designed, for example, as a touchscreen of a display device or by a button, in particular on the steering wheel. The second input device can be designed, for example, as a turn signal lever or by another button, in particular on the steering wheel. As a result, the driver's intention can be taken into account more strongly. For example, after the display of the received trajectory, the second input is detected, which represents a confirmation or activation of the actuation of the steering motor of the vehicle and / or of the drive of the vehicle for the travel of the vehicle along the received travel trajectory. In this embodiment, an undesired control of the vehicle or a control of the vehicle along a received but undesired trajectory is consequently avoided.The position of the vehicle is preferably detected by means of a position sensor, for example a sensor for a satellite-assisted navigation system (GPS, Galileo, etc.). In this embodiment, the transmitted output radio signal additionally represents the position of the vehicle. In this embodiment, the trajectory is advantageously additionally determined by the server device based on the position of the vehicle and, for example, map data and / or the volume of traffic in the vicinity of this position, as a result of which the trajectory can be better adapted to the current driving situation and / or automatically validated based on the vehicle position, as a result of which the satisfaction or the acceptance level increases with the received trajectory.The invention also relates to a computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the steps of the computer-implemented vehicle-side method.The invention also relates to the control unit or the central computing device for a vehicle, comprising at least one first signal input for providing a first signal, which represents sensor data acquired by means of at least one vehicle sensor. The control unit or the central computing device advantageously has a plurality of first signal inputs for a bus system of the respective sensor type of a sensor system of the vehicle. The control device furthermore comprises a transmitting unit for transmitting an output radio signal to a server device, wherein the output radio signal represents at least a part of the sensor data provided at the first signal input, in particular the output radio signal corresponds to at least the 5G or 6G standard. The control device also comprises a receiving unit for receiving a result radio signal which represents at least one travel trajectory calculated for the travel of the vehicle by means of the server device, in particular for parking or unparking and / or for maneuver assistance at low speeds of the vehicle. The result radio signal corresponds in particular to at least the 5G or 6G standard. The control device further has at least one signal output for outputting a control signal based on the received result radio signal, wherein the control signal is configured to actuate a steering motor of the vehicle and / or at least one drive of the vehicle for driving the vehicle along the received driving trajectory. In addition, the control unit or the central computing device comprises a computing unit, in particular a processor, which is configured to execute the steps of the vehicle-side method.The invention further relates to a vehicle comprising the control device according to the invention.The invention also relates to a computer-implemented server-side method for determining the travel trajectory of a vehicle by means of a server device. The server-side method comprises the reception of the output radio signal by means of a receiving unit of the server device. At least one travel trajectory for the vehicle is then calculated on the basis of the received output radio signal by means of the server device, in particular by means of at least one arithmetic unit of the server device. Subsequently, a result radio signal is transmitted to the vehicle by means of a transmission unit of the server device, said signal representing at least one travel trajectory calculated for the travel of the vehicle.In one embodiment, the server-side method preferably comprises a determination of at least one static and / or dynamic object in the environment of the vehicle, the determination of a movement of the dynamic object and / or the determination of a map of the environment of the vehicle in each case based on the received output radio signal. The calculation of the travel trajectory for the vehicle is then additionally carried out on the basis of the at least one specific static and / or dynamic object, the specific movement of the dynamic object and / or on the basis of the specific map. Optionally, in this embodiment, the transmitted result radio signal can additionally represent the specific static and / or dynamic object in the environment of the vehicle and / or the specific map of the environment of the vehicle.In a further refinement of the method on the server side, at least two travel trajectories for the vehicle are calculated by at least one algorithm and / or a neural network. In this configuration, the transmitted result radio signal represents the at least two travel trajectories or one of the at least two travel trajectories is selected in accordance with a safety criterion. The safety criterion can be predefined or can be detected by the user by input. For example, a predicted collision time with another static and / or movable object is determined as a safety criterion and that travel trajectory which has the maximum collision time is selected. Alternatively or additionally, a maximum steering angle deflection with respect to the straight-ahead travel and / or the number of travel trains required for carrying out the travel assistance along each calculated travel trajectory is determined as a safety criterion, for example for parking the vehicle in a parking space identified on the basis of the sensor data. Subsequently, the travel trajectory is selected which has the minimum steering angle deflection and / or the smallest number of travel trains. Alternatively, the safety criterion can be determined for each calculated travel trajectory based on a position of the vehicle and / or at least one specific static and / or dynamic object, the specific movement of the dynamic object and / or based on the specific map. As a safety criterion, for example, the time duration for travel along each travel trajectory can be determined, wherein, in particular, waiting times corresponding to the traffic are taken into account. Subsequently, the driving trajectory for which the shortest time duration results is selected.In one embodiment of the method on the server side, the calculated travel trajectory comprises a command chain, wherein the command chain has a sequence of steering angles and drive parameters. This reduces the required computing resources on the control unit or the central computing device of the vehicle.The calculation of the travel trajectory in the server-side method can additionally be carried out on the basis of the position of the vehicle, current operating data of other vehicles, static evaluations of the traffic in the environment of the vehicle, parking space occupancy maps and / or map data.The invention also relates to a server computer program comprising instructions which, when the program is executed by a server device, cause the latter to execute the steps of the method on the server side.The present invention further relates to the server apparatus including a server receiving unit for receiving the output radio signal from a vehicle. The server device also has a server transmission unit for transmitting a result radio signal to the vehicle. In addition, the server device includes a server computing unit configured to execute the steps of the server-side method.Further advantages result from the following description of exemplary embodiments with reference to the figures. FIG. 1 : Flowchart of the vehicle-side method as a block diagram FIG. 2 : Flow diagram of the method on the server side as a block diagramExemplary EmbodimentsFIG. 1 shows a flow diagram of the computer-implemented vehicle-side method as a block diagram. First, in an optional step 110, a registration method can be carried out between the vehicle and the server device for providing or maintaining computing capacities of the server device for the calculation of travel trajectories for the vehicle, as a result of which a later calculation of the travel trajectories is advantageously accelerated, that is to say it is not necessary to wait for other calculations and a later rapid reception of result radio signals with a low latency period is ensured. The registration method can be carried out in known mobile radio communication standards between the vehicle and the server device and, for example, at the start of a drive of the vehicle or at the start of travel. In the further optional step 111, a current speed of the vehicle is detected. Additionally or alternatively, a first input of the driver for carrying out a desired autonomous driving function of the vehicle can likewise optionally be detected by means of an input device of the vehicle. In a further optional step 113, a position of the vehicle is alternatively or additionally detected by means of a position sensor. Subsequently, in step 120 according to the invention, sensor data of the environment of the vehicle are acquired by means of at least one vehicle sensor. In particular, camera images are captured as sensor data by means of at least one vehicle camera as a vehicle sensor, in particular by means of a front camera, at least one wide-angle camera and / or a rear camera. Alternatively or additionally, ultrasonic sensor data is acquired as sensor data by means of at least one ultrasonic sensor as a vehicle sensor, in particular by means of an ultrasonic sensor array. Alternatively or additionally, radar data is acquired as sensor data by means of at least one radar sensor of the vehicle as a vehicle sensor, in particular by means of at least two corner radar sensors which, by varying adapted frequency modulations, are configured to acquire distances of objects to the vehicle both in the near range and in the middle range and / or in the far range. The acquisition 120 of the sensor data can additionally be carried out as a function of the acquired speed and / or as a function of the sensor type, in particular if the amount of the acquired speed is less than or equal to a speed threshold value. For example, the acquisition of ultrasonic sensor data does not take place at a vehicle speed above 30 km / h. Alternatively or additionally, the acquisition 120 of the sensor data can also take place based on the acquired first input, for example, if activation of the parking assistance is acquired as first input by the driver or user by actuating a button or another input device. The acquisition 120 of the sensor data can furthermore alternatively or additionally optionally take place as a function of the acquired position of the vehicle, in particular if the position of the vehicle is within a predefined surrounding area. In other words, the method can be started based on the detected speed of the vehicle and / or based on the detected first input of the driver or user and / or based on the detected position of the vehicle, that is to say for example automatically based on predefined and / or detected environmental and / or operating parameters and / or the first input. As a result, for example, the power consumption for capturing the sensor data and the resource outlay and power consumption on the server device are reduced, since the method is carried out for a specific required driving function and therefore not permanently, but rather only if the specific driving function is desired or is to be expected with it. For example, the method for a parking or unparking maneuver as driving assistance would therefore not be carried out in the case of a vehicle position on a freeway and / or in the case of a detected vehicle speed of greater than or equal to 30 km / h, and the method for a lane change as driving assistance would not be carried out on a single-lane road or a parking space. In an optional step of the method, not shown, it can also be provided that the captured sensor data are still preprocessed in an optional step 120 a, not shown, for example camera images captured by means of a wide-angle optical unit can be equalized by the preprocessing and / or converted into a gray-scale value world. In another example, acquired radar or ultrasound data as sensor data may be filtered and / or smoothed and / or averaged by the non-illustrated preprocessing 120 a. In a particularly preferred embodiment of the invention, at least one feature vector or feature tensor or feature space is determined in optional step 121 as a function of at least a part of the acquired sensor data by means of a computing unit of the vehicle. It can be provided, for example, that edge regions of the detection regions in the respective sensor data are ignored in the determination 121 of the feature vector. The determination 121 of the at least one feature vector takes place in particular separately on the basis of the sensor data of each sensor type, for example a first feature vector for the camera data, a second feature vector for the radar data and / or a third feature vector for the ultrasonic data. A plurality of feature vectors may be provided for the sensor data of each sensor type. The feature vector is advantageously determined by a learned machine recognition method, preferably by a neural network. The feature vector is advantageously determined by means of a first arithmetic unit. The neural network is advantageously trained with labeled and / or unlabeled training data and optionally in conjunction with the recognition of objects and / or the mapping, wherein, during training with unlabeled training data (selfsupervised), for example, a masking of the training data and the prediction of the masked areas of the original data can take place. Each specific feature vector is in particular multidimensional and advantageously represents a non-human interpretable understanding of the captured sensor data of a sensor type, wherein the feature vector does not represent objects and / or a map or a mapping of the environment of the vehicle. However, it can be provided that the components of the feature vector can be assigned to a two- or three-dimensional space on a distance-based basis. In other words, the feature vector is an intermediate stage in the sensor data analysis before and for the purpose of later recognition of objects and / or map creation with respect to the current environment of the vehicle. Based on the at least one determined feature vector, static and / or dynamic objects and / or a current map of the environment can be determined in a further step 122. Step 122 is advantageously carried out by means of a second computing unit of the vehicle, in particular by means of a second learned machine recognition method or the recognition head, preferably a second neural network. Subsequently, according to the invention, in step 130, an output radio signal is transmitted which represents at least a part of the captured sensor data and / or the at least one feature vector as representation of the captured sensor data or the feature space and / or optionally the at least one determined static and / or dynamic object and / or optionally the determined map of the environment as representation of the captured sensor data and / or the position of the vehicle. The output radio signal is advantageously transmitted in the 5G or 6G standard. The output radio signal is advantageously transmitted as a function of a detected driving situation of the vehicle, wherein the driving situation is detected in particular on the basis of the detected sensor data and / or a detected first input of the user and / or detected operating parameters of the vehicle, for example the detected speed of the vehicle and / or the position of the vehicle. For example, the transmission 130 takes place, in particular when the amount of the detected speed is less than or equal to a speed threshold value, based on the first input of the driver for carrying out a desired autonomous driving function and / or depending on the detected position of the vehicle, in particular when the position is within a predefined surrounding area. In optional step 140, a self-calculated travel trajectory for future travel of the vehicle in the current environment of the vehicle detected by means of the vehicle sensors is calculated by means of a computing unit of the vehicle, wherein the computing unit or the self-calculated travel trajectory is used, for example, for lane keeping, in particular for predefined travel situations or driving assistance functions which are comparatively easy to control. In optional step 145, the speed of the vehicle is reduced until the result radio signal is received, in particular as a function of the current server load, which is received by the server device by radio signal at regular intervals, and / or based on an expected calculation time assigned to the current driving situation, wherein the current driving situation is recognized as a function of the captured sensor data and / or the determined feature vector and / or the determined objects and / or the determined map and / or the current vehicle position. Subsequently, in step 150, a result radio signal is received which represents at least one travel trajectory determined for the travel of the vehicle as a function of the output radio signal, in particular for parking the vehicle in or out of the vehicle and / or for maneuver assistance of the vehicle at low speeds. In an optional step 155, not shown, the self-calculated travel trajectory is validated and / or replaced by the travel trajectory received from the server device, wherein, in particular in the case of a deviation between the self-calculated travel trajectory and the travel trajectory received from the server device, the self-calculated travel trajectory is replaced by the travel trajectory received from the server device or the received travel trajectory is prioritized. The optional replacement 155 of the self-calculated travel trajectory can also be carried out on the basis of the detected first input and / or on the basis of the detected speed of the vehicle and / or on the basis of the detected position of the vehicle. In step 160, the at least one received travel trajectory is displayed. In an optional step 170, a second input of the driver is detected for confirming the performance of the desired autonomous driving function of the vehicle by means of the first input device and / or a second input device of the vehicle. Subsequently, in addition or as an alternative to the display 160 of the received travel trajectory, the actuation 180 of at least one steering motor of the vehicle and / or of at least one drive of the vehicle for travel of the vehicle along the received travel trajectory takes place. The steering motor and / or the drive are optionally controlled as a function of the detected second input, in particular only when the second input represents a confirmation for travel along the displayed received travel trajectory by the user. The display 160 of the travel trajectory and / or the control 180 of the steering motor and / or the control of the drive can optionally take place additionally in each case on the basis of the detected speed, in particular if the magnitude of the detected speed is less than or equal to a speed threshold value, on the basis of the first input of the driver for carrying out a desired autonomous driving function and / or on the basis of the detected position of the vehicle, in particular if the position is within a predefined surrounding region. In a preferred embodiment, an emergency braking situation is determined in parallel and independently of the transmission 130 of the output radio signal and the reception 150 of the result radio signal and independently of the optional actuation 180 of the steering motor and / or the actuation of the drive in optional step 190 by an emergency braking algorithm, which is carried out by means of another computing device of the vehicle. Subsequently, in optional step 191, at least one braking device of the vehicle is advantageously actuated as a function of the determined emergency braking situation. Optionally, in step 192, the vehicle can be returned automatically along the already traveled travel trajectory in the opposite direction as a function of the determined emergency braking situation by actuating the steering motor and / or the drive.FIG. 2 shows a flow diagram of the computer-implemented server-side method as a block diagram, which is carried out by means of a server device. The server-side method begins with the reception 210 of the output radio signal by means of a reception unit of the server device. The output radio signal represents at least a portion of sensor data acquired by at least one vehicle sensor. The output radio signal advantageously corresponds at least to the 5G or 6G standard. Preferably, in optional step 220, at least one static and / or dynamic object in the environment of the vehicle, a movement of the dynamic object and / or a map of the environment of the vehicle is determined in each case on the basis of the received output radio signal by means of the server device. Subsequently, in step 230, at least one travel trajectory for the vehicle is calculated by means of the server device on the basis of the received output radio signal. The calculation 230 of the travel trajectory for the vehicle is additionally carried out on the basis of the at least one specific static and / or dynamic object, the specific movement of the dynamic object and / or on the basis of the specific map. In this case, the calculation 230 of the travel trajectory can be carried out by at least one algorithm and / or a neural network. In step 230, at least two travel trajectories or else a multiplicity of trajectories for the vehicle are furthermore advantageously calculated. In step 230, the travel trajectory to be transmitted can then be selected from among the at least two or the plurality of calculated trajectories, in particular on the basis of a safety criterion which comprises, for example, a collision time as long as possible with respect to adjacent moving objects and / or a steering and / or acceleration movement as small as possible with respect to the respective trajectory. The comparison or the safety criterion can be predefined or can be detected by the user by input. Alternatively or additionally, the safety criterion can be determined for each calculated travel trajectory based on a position of the vehicle and / or at least one specific static and / or dynamic object, the specific movement of the dynamic object, a specific own movement of the vehicle and / or based on the specific map. Thus, the travel trajectory is selected which appears advantageous after a comparison with respect to the safety criterion. For example, the calculated trajectory of the two or the plurality of calculated trajectories with the longest collision time to adjacent moving objects and / or the lowest steering and / or acceleration movement is selected.The travel trajectory calculated in step 230 comprises, for example, a command chain, the command chain having a sequence of steering angles and drive parameters. In the subsequent step 240, the result radio signal is transmitted to the vehicle by means of a transmission unit of the server device, which transmission unit represents at least one travel trajectory calculated for the travel of the vehicle. The result radio signal corresponds in particular to at least the 5G or 6G standard. Optionally, the transmitted result radio signal additionally represents the specific static and / or dynamic object in the environment of the vehicle and / or the specific map of the environment of the vehicle. It can optionally be provided that the result radio signal transmitted in step 240 represents at least two calculated travel trajectories, wherein the vehicle is then configured to select one of the calculated travel trajectories.References included in the specificationThis list of documents cited by the applicant has been produced in an automated manner and is only included for the better information of the reader. The list is not part of the German patent application or utility model application. The DPMA does not take any adhesion for any faults or omissions.Patent Literature citedWO 2019 / 050873 A1

[0002] DE 10 2020 111 938 A1

[0003] EP 3 616 182 A1

[0004] DE 10 2022 119 206 A1

[0005] Cited Non-Patent LiteratureDrews, F. et al. (2022) "DeepFusion: A Robust and Modular 3D Object Detector for Lidars, Cameras and Radars", arXiv:2209.12729v2

[0009]

Claims

Computer-implemented vehicle-side method for driving assistance, comprising the following steps • Optional implementation (110) of a registration method between the vehicle and a server device, • Detection (120) of sensor data of the environment of the vehicle by means of at least one vehicle sensor, • Transmission (130) of an output radio signal to the server device, which signal represents at least a part of the detected sensor data, • Reception (150) of a result radio signal from the server device, which signal represents at least one travel trajectory determined for the travel of the vehicle as a function of the output radio signal, and • Display (160) of the at least one received travel trajectory, and / or • Activation (180) of at least one steering motor of the vehicle and / or of at least one drive of the vehicle for the travel of the vehicle along the received travel trajectory.Vehicle-side method according to Claim 1, characterized in that the following step is carried out • determination (121) of at least one feature vector as a function of at least part of the captured sensor data by a computing unit of the vehicle, in particular by means of a learned machine recognition method, preferably a neural network, and • the transmitted output radio signal comprising at least the determined feature vector as representation of the captured sensor data.Vehicle-side method according to Claim 1, wherein the following steps are carried out • determination (121) of at least one feature vector as a function of at least part of the captured sensor data by means of a first computing unit of the vehicle, in particular by means of a first learned machine recognition method, preferably a first neural network, and • determination (122) of static and / or dynamic objects and / or a current map of the environment on the basis of the at least one determined feature vector by means of a second computing unit of the vehicle, in particular by means of a second learned machine recognition method, preferably a second neural network, and • the emitted output radio signal comprises at least the static and / or dynamic objects and / or the determined map of the environment as representation of the captured sensor data.Vehicle-side method according to one of the preceding claims, wherein the following step is carried out • reduction (145) of the speed of the vehicle until the result radio signal is received, in particular as a function of the current server load, which is received by the server device by radio signal at regular intervals, and / or on the basis of an expected calculation time assigned to the current driving situation, wherein the current driving situation is detected as a function of the captured sensor data.Vehicle-side method according to one of the preceding claims, wherein the following steps are carried out • determination (190) of an emergency braking situation as a function of the captured sensor data by an emergency braking algorithm which is carried out by means of a computing device of the vehicle, and • actuation (191) of at least one braking device of the vehicle as a function of the determined emergency braking situation, wherein • optionally automatic return (193) of the vehicle along the already traveled travel trajectory in the opposite direction as a function of the determined emergency braking situation takes place by actuation of the steering motor and / or of the drive.Vehicle-side method according to one of the preceding claims, wherein the following steps are carried out • detection (111) of a current speed of the vehicle, and / or • detection (112) of a first input of the driver for carrying out a desired autonomous driving function of the vehicle by means of an input device of the vehicle, and / or • detection (113) of a position of the vehicle by means of a position sensor, and wherein • the detection (120) of the sensor data, the transmission (130) of the output radio signal, the display (160) of the driving trajectory and / or the actuation (180) of the steering motor and / or of the drive are each additionally carried out on the basis of the detected speed, on the basis of the first input of the driver for carrying out a desired autonomous driving function and / or on the basis of the detected position of the vehicle.Vehicle-mounted method according to Claim 6, wherein the following step is carried out • detection (170) of a second input by the driver in order to confirm the performance of the desired autonomous driving function of the vehicle by means of the first input device and / or a second input device of the vehicle, and wherein • the actuation (180) of the steering motor and / or of the drive additionally takes place as a function of the detected second input.Vehicle-mounted method according to one of the preceding claims, wherein the following steps are carried out • detection (113) of the position of the vehicle by means of the position sensor, and wherein • the transmitted output radio signal additionally represents the position of the vehicle.A computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the steps of the vehicle-mounted method according to any one of the preceding claims.Control device for a vehicle, comprising at least the following components • a first signal input for providing a first signal which represents sensor data detected by means of at least one vehicle sensor, • a transmission unit for transmitting an output radio signal to a server device, wherein the output radio signal represents at least part of the sensor data provided at the first signal input, • a reception unit for receiving a result radio signal which represents at least one travel trajectory calculated for the travel of the vehicle by means of the server device, and • a signal output for outputting a control signal based on the received result radio signal, wherein the control signal is configured to control a steering motor of the vehicle and / or at least one drive of the vehicle for the travel of the vehicle along the received travel trajectory, and • a computing unit, in particular a processor, which is configured in such a way that, In that it carries out the steps of the vehicle-mounted method according to one of Claims 1 to 8.Vehicle comprising a control device according to Claim 10.Computer-implemented server-side method for determining the travel trajectory of a vehicle by means of a server device, comprising the following steps • receiving (210), by means of a receiving unit of the server device, an output radio signal which represents at least a part of sensor data acquired by means of at least one vehicle sensor, in particular the output radio signal corresponds at least to the 5G or 6G standard, • calculating (230), by means of the server device, at least one travel trajectory for the vehicle on the basis of the received output radio signal, and • transmitting (240), by means of a transmitting unit of the server device, a result radio signal which represents at least one travel trajectory calculated for the travel of the vehicle, in particular the result radio signal corresponds at least to the 5G or 6G standard.The server-side method according to claim 12, wherein the following step is carried out • determining (220) at least one static and / or dynamic object in the environment of the vehicle, a movement of the dynamic object and / or a map of the environment of the vehicle, in each case based on the received output radio signal, by means of the server device, and wherein • the calculation (230) of the travel trajectory for the vehicle is additionally carried out based on the at least one determined static and / or dynamic object, the determined movement of the dynamic object and / or based on the determined map, and wherein • optionally the transmitted result radio signal additionally represents the determined static and / or dynamic object in the environment of the vehicle and / or the determined map of the environment of the vehicle.The server-side method according to one of claims 12 or 13, wherein at least two travel trajectories for the vehicle are calculated by at least one algorithm and / or a neural network, wherein the transmitted result radio signal represents the at least two travel trajectories or a selection of the travel trajectory to be transmitted takes place according to a safety criterion.The server-side method according to any one of claims 12 to 14, wherein the calculated travel trajectory comprises a command chain, wherein the command chain comprises a sequence of steering angles and drive parameters.The server-side method according to one of claims 12 to 15, wherein the calculation (230) of the travel trajectory is additionally carried out on the basis of the position of the vehicle, current operating data of other vehicles, static evaluations of the traffic in the environment of the vehicle, parking space occupancy maps and / or map data.A server computer program comprising instructions which, when the program is executed by a server device, cause the server device to carry out the steps of the method according to any one of claims 12 to 16.Server device, comprising at least the following components • a server receiving unit for receiving an output radio signal from a vehicle, which represents at least a part of sensor data acquired by means of at least one vehicle sensor, in particular the output radio signal corresponds at least to the 5G or 6G standard, • a server transmitting unit for transmitting to the vehicle a result radio signal, which represents at least one travel trajectory calculated for the travel of the vehicle, in particular the result radio signal corresponds at least to the 5G or 6G standard, and • a server computing unit, which is configured such that it carries out the steps of the server-side method according to one of Claims 12 to 16.

Images