Using artificial neural networks to detect trailers
By using artificial neural networks to process radar data in a motor vehicle radar system, the problems of accuracy and detailed representation of trailer detection were solved, achieving higher redundancy trailer identification and representation, and simplifying the training process.
Patent Information
- Application Number
- CN202580026575.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2024-04-08
- Filing Date
- 2025-03-05
- Publication Date
- 2026-11-10
AI Technical Summary
In the existing technology, the radar system of a motor vehicle has problems such as false warnings and difficulty in function adaptation when detecting trailers, especially when driving with a trailer, it is difficult to accurately identify the presence of the trailer and characterize the trailer data in detail.
Artificial neural networks are used to process radar system data. An end-to-end learning method is used to directly extract the presence and detailed characterization of trailers from radar data, and combined with the motion data of the tractor unit for higher redundancy detection and characterization.
It achieves robust detection and detailed characterization of trailers, simplifies neural network training, improves the adaptability and recognition accuracy of radar systems, and provides additional verification methods, especially in the case of electrical connection failures or omissions.
Smart Images

Figure CN123076808A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method for detecting trailers towed by a tractor unit using a radar system of the tractor unit. Background Technology
[0002] Motor vehicles are often equipped with radar systems that provide driver assistance systems with location data about the traffic environment, such as for speed and distance adjustment, blind spot monitoring, lane change assist, emergency braking assist, or, especially in the case of freight vehicles, trailer shunting assist. When a vehicle is towing a trailer, this requires some adaptation in the functions of the driver assistance system. For example, speed limits must be observed when traveling with a trailer. In blind spot monitoring, false warnings may occur if the radar system (e.g., when turning) interprets the trailer as another vehicle in the blind spot. In lane change assist, the warning area must be adapted to the greater length of the vehicle train (Gespanns) consisting of the tractor and trailer.
[0003] US2020 / 02829A describes a method for training an image-based system for recognizing trailer angles, in which images of the trailer drawbar at different trailer angles are used. In one example, the system includes a neural network. Summary of the Invention
[0004] The objective of this invention is to describe a novel method for detecting the presence of trailers and / or for characterizing trailer data in more detail.
[0005] According to the present invention, this task is solved by a method for detecting a trailer towed by a tractor using a radar system of a tractor, the method comprising the following steps: Obtain radar data detected by the radar system from the radar system; and Radar data is input into an artificial neural network, which receives and processes the input radar data to generate at least one output signal that indicates the detection of the presence of a trailer towed by the tractor and / or trailer data characterizing the trailer towed by the tractor.
[0006] Therefore, neural networks can be used to directly extract the presence of trailers or provide more detailed radar data characterizing trailers from radar data. This particularly simplifies the training of neural networks. This allows for training using end-to-end learning methods. Thus, the effort required to adapt the method to new sensors, such as radar systems, can be reduced to creating new training datasets.
[0007] Robust detection or characterization of trailers can also be achieved by directly accessing radar data.
[0008] The radar system can be, in particular, an environmental radar system for the tractor unit. In the step of acquiring radar data, the current radar data detected by the radar system is obtained, especially from the radar system itself. Specifically, radar data detected by the radar system can be obtained from the radar system while the tractor unit is in motion, and / or the radar data can be input into an artificial neural network during travel.
[0009] The method may include receiving at least one output signal from an artificial neural network.
[0010] Typically, trailers have lighting equipment that must be connected to the vehicle's electrical system via a connector. When this electrical connection is established, a detection signal indicating the trailer's presence can be generated simultaneously. If at least one output signal indicating the detection of the trailer's presence is generated via an artificial neural network, then the trailer's presence can therefore be identified in a different way than by recognizing the electrical connection, thus enabling a higher degree of redundancy. This is useful, for example, in cases of electrical connector failure or forgetting to connect the lighting equipment.
[0011] If an artificial neural network generates at least one output signal that describes trailer data characterizing a tractor-trailer towed by the tractor, this can be used to further verify the existence of the trailer.
[0012] Other advantageous configurations of the invention are described below and in the dependent claims.
[0013] In this implementation, the radar data input to the artificial neural network includes point probes of reflective points. Point probes may include radial range and / or radial velocity, respectively. Point probes may include at least one of the following parameters: radial range, radial velocity, azimuth, elevation, and radar cross section (RCS). Point probes may be transmitted as a point cloud or input to the artificial neural network. The input vector of the neural network can be padded to the maximum number of points (point probes).
[0014] In an implementation, the radar data input to the artificial neural network includes at least one spectrum, which includes a domain corresponding to a range range. This spectrum may also include a domain corresponding to a Doppler range or a velocity range. The values of the spectrum may, in particular, include the amplitude, intensity, and / or phase of the received radar signal. For example, the radar data input to the artificial neural network may include at least one range-Doppler spectrum. For example, multiple spectra from various channels of the radar system may be input. This spectrum may also include a domain corresponding to an angular range.
[0015] Radar data includes detection of the tractor's environment. Radar data can be provided through prior signal processing. For example, radar signals received by the radar system can be sampled and converted into digital radar data.
[0016] In an implementation, the artificial neural network includes: a feature extraction network into which radar data is input; at least one target detection head connected to the feature extraction network for outputting target data characterizing targets detected in the environment of the tractor based on the radar data; and a trailer detection head for outputting at least one output signal, i.e., at least one output signal indicating the detection of the presence of a trailer towed by the tractor and / or indicating trailer data characterizing the trailer towed by the tractor. Specifically, the target data can be generated by the artificial neural network from the processing of the input radar data. In other words, the artificial neural network receives and processes the input radar data to thereby also generate target data. The artificial neural network thus generates at least one output signal and target data. The trailer detection head can be another target detection head. The artificial neural network can therefore include multiple target detection heads connected to the feature extraction network for outputting target data characterizing targets detected in the environment of the tractor based on the radar data, wherein one of the target detection heads is a trailer detection head for outputting at least one output signal.
[0017] Therefore, it is possible to use or "reuse" the feature extraction network (particularly the encoder and backbone of the neural network) together with the artificial neural network used for target detection when using the artificial neural network for target detection. This is another advantage of the end-to-end learning approach when training artificial neural networks. By using the feature extraction network together for at least one target detection head and one trailer detection head, trailer characterization can be performed, for example, as an additional task in a separate head in addition to target detection.
[0018] In implementations, trailer data includes at least one of the following: folding angle, number of axles, wheelbase, trailer type classification, length, width, height, and coupling position (location of the coupling point). Trailer data can therefore include dynamic, i.e., variable trailer data during operation, such as the trailer's folding angle and, in the presence of a lift axle, the number of axles and / or wheelbase, and optionally, the trailer type classification. Trailer data can also include static trailer data, i.e., data that remains constant during operation, such as the number of axles, wheelbase, trailer type classification, trailer length, trailer width, trailer height, and coupling position, particularly the coupling position on the tractor unit. The output signal indicating the detection of the trailer's presence represents static or dynamic trailer data in this sense. The output signal, i.e., the output representing the trailer, as a static or dynamic parameter, can therefore include information such as whether the trailer is currently hooked to the tractor unit. One or more folding angles (trailer folding angles) can be output. As mentioned, in the presence of a lift axle, the number of axles and therefore the wheelbase can vary during operation. The coupling point can vary drastically between models in commercial vehicles and, less commonly, in passenger cars. Typically, this coupling point is needed, among other parameters, to more accurately describe the trailer's position relative to the tractor. Trailer type can be understood, for example, as a characterization based on the number of axles and / or the number of folding points per trailer. Trailer type can be classified, for example, by trailer type. Trailer types can include: center-axle trailers, swivel-beam trailers, and optionally, trolley trailers (Rollwagen or Dolly). A trolley trailer represents a very short trailer with a saddle-type coupling.
[0019] In this implementation, the method further includes: obtaining motion data (particularly vehicle motion data) detected by the tractor's own sensor system; and also inputting the motion data into an artificial neural network, wherein the artificial neural network receives and processes the input radar data and the input motion data to generate at least one output signal. Since the trailer is connected to the tractor via a coupling, and therefore the motion of the tractor directly affects the motion of the trailer, the current motion data also contains information about the trailer's motion. The motion data may in particular include one or more of the following: current speed, yaw rate, and steering angle. Specifically, the steering angle may include the steering angle on the front axle and the steering angle on the rear axle of the tractor.
[0020] This task is also addressed by a system for a tractor unit that detects trailers towed by the tractor unit. This system includes a radar system and an artificial neural network. The system includes an evaluation and control unit designed to input radar data detected by the radar system into the artificial neural network. The artificial neural network is designed to receive and process the input radar data to generate at least one output signal indicating the detection of the presence of the trailer towed by the tractor unit and / or indicating trailer data characterizing the trailer towed by the tractor unit. The evaluation and control unit can be specifically designed to perform the method described.
[0021] The artificial neural network of the method or system described below can be trained.
[0022] According to another aspect of the present invention, a method for training an artificial neural network is provided, the artificial neural network being used in the method described above or as an artificial neural network for the system described above. The method for training the artificial neural network includes the following steps: During at least one training run in the case where the tractor has a trailer towed by the tractor, radar data is detected by means of the tractor's radar system (and optionally, vehicle motion data of the tractor is also detected). Reference data is allocated, wherein the reference data is allocated to radar data (and optionally vehicle motion data), wherein the reference data indicates the presence of a trailer towed by the tractor and / or indicates reference trailer data characterizing the trailer towed by the tractor; and Artificial neural networks are trained based at least on radar data and reference data assigned to the radar data.
[0023] When training an artificial neural network, radar data can be used as both input and output, with the output signal expected or obtained. This output signal can be compared to reference data. The artificial neural network can then be adapted based on this comparison. Therefore, the artificial neural network can be trained using end-to-end learning methods.
[0024] Methods for training artificial neural networks may include one or more features or steps described below.
[0025] Specifically, the method may further include: detecting vehicle motion data of the tractor during at least one training run, wherein (in the step of allocating reference data) reference data is allocated to both radar data and vehicle motion data. The vehicle motion data may, in particular, include the vehicle's own motion data.
[0026] The method (training method) may further include: detecting additional radar data (and optionally additional vehicle motion data of the tractor) using the radar system of the tractor during at least one training run in the absence of a trailer towed by the tractor; and allocating reference data to the additional radar data (and optionally additional vehicle motion data), wherein the reference data indicates the absence of a trailer towed by the tractor. The radar data and reference data used to train the neural network can therefore include cases with and without trailers.
[0027] The method for detecting trailers towed by a tractor unit may in particular include methods for training artificial neural networks.
[0028] This method may include: inputting or detecting invariant parameters of the trailer in question. Invariant parameters may in particular be a portion of reference data.
[0029] This method may include recording radar data (and optionally vehicle motion data) during the corresponding training drive. In particular, the actual training of the artificial neural network may be performed after one or more training drives have been completed.
[0030] This method may include preprocessing the radar data. Preprocessing may include filtering the radar data by distance, speed, and / or spatial angle. This filtering simplifies the process because these parameters are easily constrained for the respective trailers used.
[0031] The method may include: altering the detected radar data (and optionally related vehicle motion data), particularly by enhancement. Enhancement or alteration may include, for example, removing and / or adding individual detectors (e.g., point detectors), rotating all or individual detectors (e.g., point detectors), and / or mirroring radar data or detectors.
[0032] Reference data may include, for example, reference angles (reference folding angles) and / or reference number of axles. Reference data may also include, for example, the aforementioned trailer data, which characterizes the trailer towed by the tractor. Trailer data may include, for example, at least one of the following: trailer folding angle, trailer type, trailer width, trailer length, trailer height, trailer wheelbase, number and / or position of trailer axles, and trailer coupling point, at which the trailer coupler couples with the tractor.
[0033] This method may include: obtaining reference data by evaluating radar data. Evaluating the radar data may include, for example, Principal Component Analysis (PCA) to obtain, for example, the orientation of a point cloud (a set of point probes) as the trailer orientation. Furthermore, evaluating the radar data may include: evaluating the radar data to detect micro-Doppler detections in areas of the trailer's axles or wheels to determine the position and / or number of the trailer's rotation axes. Evaluating the radar data may also include: evaluating the distribution of detected velocities on parts of the trailer (particularly rigid parts of the trailer, such as the trailer body) to determine, for example, the trailer's mass, dimensions, and / or other parameters.
[0034] The method may include obtaining reference data from a folding angle sensor, such as a folding angle sensor of a tractor or trailer.
[0035] This method may include obtaining reference data via the tractor's bus system. For example, the reference data can be supplemented or improved by evaluating the bus data, particularly trailer bus data. In the commercial vehicle sector, vehicle buses through which bus data, such as trailer bus data, can be obtained are frequently used as standardized protocols.
[0036] This method may include checking whether the detected radar data and their respective assigned reference data are sufficiently complete, with respect to the trailer data values to be trained and / or the vehicle conditions to be trained. For example, the completeness of the existing dataset (radar data and reference data) in terms of dynamic parameters can be checked.
[0037] To achieve sufficient system performance for the detection method, it is desirable that both the validation and training datasets contain a minimum number of measurement-label pairs (pairs of radar data with associated reference data). These should cover a wide range of possible states.
[0038] The method may include: if the result of the sufficient integrity check is positive, then start training the artificial neural network.
[0039] This method may include generating additional radar data and related reference data, particularly through enhancement, especially as described above, if the inspection results do not yet provide sufficient completeness. In this way, if the expected data is not yet included in the training set, this data can be additionally generated through variation / enhancement methods and by assigning corresponding reference data.
[0040] The method may include: obtaining motion data detected by the tractor's own sensor system, wherein the training of the artificial neural network is based at least on radar data, reference data assigned to the radar data, and associated motion data. The motion data may be used herein as input to the artificial neural network and / or as part of the reference data.
[0041] The method may include: detecting vehicle motion data of the tractor unit during at least one training run, wherein reference data is allocated to radar data and vehicle motion data. Detecting vehicle motion data may include: obtaining vehicle motion data detected by the tractor unit's own sensor system.
[0042] The detection method can also be used to link (especially fuse) the evaluation (at least one output signal and optionally target data) obtained by the artificial neural network with detection data from other environmental sensor systems of the tractor, such as environmental sensor systems that include at least one of additional radar sensors, lidar sensors and video-based sensors. Attached Figure Description
[0043] The embodiments are explained in more detail below with the aid of the accompanying drawings. The drawings show: Figure 1 A block diagram of a system for detecting trailers according to the present invention; Figure 2 A block diagram of the artificial neural network of this system; Figure 3 A top-view sketch of the tractor and trailer; Figure 4 Examples of radar data presented in point cloud or range-Doppler spectrum format; and Figure 5 A flowchart of a method for training artificial neural networks. Detailed Implementation
[0044] Figure 1 The system shown for detecting a trailer towed by a tractor unit includes a radar system with two radar sensors 10 and 12, and an electronic evaluation and control unit 14, which includes a trailer characterization module 16. The radar sensors 10 and 12 are mounted at the left and right rear corners of the tractor unit (motor vehicle) and monitor the space behind and to the left or right of the vehicle, respectively. Optionally, the positioning areas of the two radar sensors can overlap, thus also detecting the space directly behind the vehicle. The system may include additional radar sensors (not shown), particularly front sensors, as well as other environmental sensor systems, particularly lidar sensors and camera-based sensors.
[0045] The trailer representation module 16 includes an artificial neural network 18, which is a feedforward neural network. The neural network 18 therefore does not have feedback connections. Alternatively, the neural network 18 could be a recurrent neural network or a convolutional neural network.
[0046] The evaluation and control unit 14 obtains detected radar data 20 from the radar sensors 10, 12 of the radar system. The radar data 20 contains the detection of radar reflections in the vehicle's environment. The radar data can be transmitted in the form of a point cloud, whose points are characterized, for example, by estimates of radial distance d, radial velocity v, azimuth and elevation angles, and radar cross section. Alternatively, the radar data can also be transmitted, for example, in the form of a range-Doppler spectrum.
[0047] In addition, the evaluation and control unit 14 obtains motion data 22 of the tractor from the tractor's own sensor system 24.
[0048] The evaluation and control unit 14 inputs the acquired radar data 20 and motion data 22 as input vectors into the artificial neural network 18. This artificial neural network processes the described data and thereby generates an output signal 26. The output signal 26 is smoothed temporally (in the time domain) by a filter 27 to balance fluctuations. The filtered output signal 26' is then output by the trailer characterization module 16 as a result of trailer detection and trailer characterization. The output signal 26 or 26' specifically includes a trailer presence signal and more detailed trailer data characterizing the detected trailer towed by the tractor. Trailer data includes, for example, the trailer's folding angle, trailer type, trailer width, trailer length, trailer height, trailer wheelbase, number and position of trailer axles, and trailer coupling point where the trailer's coupling arm couples with the tractor. The trailer folding angle is the current angle of the trailer drawbar relative to the coupling point.
[0049] The estimated trailer data and trailer presence signal of the output signal 26' can be provided, for example, to the driver assistance function 15 implemented in the evaluation and control unit 14.
[0050] Figure 2The schematic diagram illustrates the construction of an embodiment of a neural network 28 implemented in an evaluation and control unit, which includes a neural network 18 of a trailer characterization module 16. The neural network 28 is used for target detection and includes a feature extraction network 30 (also referred to as a backbone network) and multiple target detection heads connected to the output of the feature extraction network 30, including a trailer detection head 32 and other target detection heads 34 for detecting other targets. The feature extraction network 30 receives radar data 20 and motion data 22 as input. The trailer detection head 32 outputs an output signal 26, which includes a trailer presence signal and trailer data. The other target detection heads 34 output target data 36 for detected targets. The target data characterizes targets detected in the environment of the search vehicle based on the radar data 20.
[0051] Figure 3 The tractor unit 38 and trailer 40 are schematically shown in the top view. Radar sensors 10 and 12 are schematically shown at the rear corner of the tractor unit 38. The reflection point 42 at the corner of the trailer 40's structure, detected as a radar reflection by radar sensor 12, is schematically shown. Other reflection points 44 may be generated and detected, for example, at the wheel arches 46 of the trailer 40. In addition, reflection points 48 exhibiting a micro-Doppler effect due to the rotating wheels can be detected. The self-sensor system 24 can detect the folding angle between the longitudinal axis f of the tractor unit 38 and the longitudinal axis a of the trailer 40, in addition to the speed and yaw rate (or yaw speed) of the tractor unit 38. The location of the coupling point K is also schematically shown.
[0052] Figure 4 The left-hand portion exemplarily and schematically illustrates radar data 20 in the form of a point cloud of detected radar reflections, wherein each radar reflection of the point cloud may be associated with other characteristics characterizing the radar reflection, such as azimuth and / or elevation angles and radar cross section, in addition to radial distance d and radial velocity v. The right-hand portion serves as an alternative example of radar data 20. Figure 4 The range-velocity spectrum or range-Doppler spectrum is schematically shown, wherein the domain of definition in the first dimension includes the range range d0 to d1, and the domain of definition in the second dimension includes the velocity range v0 to v1. The values of the spectrum can be, in particular, the amplitude or intensity of the radar signal received and sampled by the corresponding channel of the radar sensor 10, 12 involved.
[0053] The methods for training the corresponding components of neural network 18 or neural network 28 are described below using... Figure 5 explain.
[0054] In the first branch 50 of building the training dataset, static training data (invariant parameters) are detected for the corresponding trailer in step 52.
[0055] In branch 54, which runs parallel to the construction of the training dataset, dynamic data (data about dynamic parameters) is summarized.
[0056] In step 56, radar data 20 and its own motion data 22 are recorded for at least one test drive of the tractor 38 with the trailer 40. In step 58, the radar data 20 is preprocessed, which may include, for example, filtering the radar data 20 according to radar data that may involve the trailer 40. In step 60, the radar data or detections during the test drive may be modified, or supplemented (enhanced) with artificially generated additional radar data.
[0057] In step 64, reference data is assigned to the radar data, such as a reference angle and reference axle number for the trailer's folding angle. Here, the reference data is summarized for the trailer data estimated later by the neural network and correlated with the radar data. Data that could be obtained in step 62, for example, via the vehicle bus, from the vehicle's own sensor system 24, and / or from at least one external sensor, can be accessed here.
[0058] In step 66, the training dataset is checked for predefined features (e.g., to see if the radar dataset corresponds to the desired radar dataset and thus meets the desired integrity criteria), and in step 68, it is determined whether the desired integrity criteria for the training dataset are met. If not, additional training data is generated in step 70, for example, using the method described in step 60. If the integrity criteria are met, the neural network is trained in step 72 using the training dataset, which includes both static and dynamic data.
Claims
1. A method for detecting a trailer (40) towed by a tractor (38) using a radar system (10, 12) of the tractor (38), comprising the following steps: Obtain radar data (20) detected by the radar system (10, 12); and The radar data (20) is input into the artificial neural network (18), wherein, The artificial neural network (18) receives and processes the input radar data (20) to generate at least one output signal (26) that describes the detection of the presence of the trailer (40) towed by the tractor (38) and / or describes trailer data characterizing the trailer (40) towed by the tractor (38).
2. The method according to claim 1, wherein, The radar data (20) input into the artificial neural network (18) includes point detection of reflection points.
3. The method according to claim 1, wherein, The radar data (20) input to the artificial neural network (18) includes at least one spectrum, which includes a domain corresponding to the range.
4. The method according to any one of the preceding claims, wherein, The artificial neural network (18) includes: Feature extraction network (30) inputs the radar data into the feature extraction network; At least one target detection head (34) connected to the feature extraction network (30), the target detection head being used to output target data (36), the target data representing targets detected in the environment of the tractor (38) based on the radar data (20); and Trailer detector (32), the trailer detector is used to output the at least one output signal (26).
5. The method according to any one of the preceding claims, wherein, The trailer data includes at least one of the following: folding angle, number of axles, wheelbase, trailer type classification, length, width, height, and connection position (K).
6. The method according to any one of the preceding claims, wherein, The method further includes: The motion data (22) detected by the tractor (38)'s own sensor system (24) is obtained; and The motion data (22) is also input into the artificial neural network (18), wherein the artificial neural network (18) receives and processes the input radar data (20) and the input motion data (22) to generate the at least one output signal (26).
7. The method according to any one of the preceding claims, wherein, The method further includes: Based on the output signal (26) of the artificial neural network (18), a trailer presence signal or trailer data is output to at least one driver assistance function (15) implemented in the evaluation and control unit (14) of the tractor (38).
8. A system for a tractor unit, the system being used to detect a trailer (40) towed by the tractor unit (38), wherein, The system includes a radar system (10, 12) and an artificial neural network (18), wherein the system includes an evaluation and control unit (14) designed to input radar data (20) detected by the radar system (10, 12) into the artificial neural network (18), wherein the artificial neural network (18) is designed to receive and process the input radar data (20) to generate at least one output signal (26) that describes the detection of the presence of the trailer (40) towed by the tractor (38) and / or describes trailer data characterizing the trailer (40) towed by the tractor (38).
9. A method for training an artificial neural network, said artificial neural network for use in any one of claims 1 to 7 or as an artificial neural network for use in a system according to claim 8, comprising the following steps: During at least one training drive with the tractor (38) having a trailer (40) towed by the tractor (38), radar data (20) is detected (56) by means of the radar system (10, 12) of the tractor (38). Assign (64) reference data, where, The reference data is assigned to the radar data (20), wherein the reference data describes the presence of the trailer (40) towed by the tractor (38) and / or describes reference trailer data characterizing the trailer (40) towed by the tractor (38); and The artificial neural network (18) is trained (72) based at least on the radar data (20) and the reference data assigned to the radar data.
10. The method according to claim 9, wherein, The method further includes detecting (62) vehicle motion data (22) of the tractor (38) during the at least one training drive, wherein the reference data is allocated to the radar data (20) and the vehicle motion data (22).
