Method and device for detecting the load of a vehicle

Abstract

Method for load detection of a vehicle which has an external camera (K) for capturing the vehicle environment in front of the vehicle, wherein - first image data (UD1) is captured (11) and stored (12) by the outdoor camera (K) and second image data (UD2) is captured (13) by the outdoor camera (K) at a later time; - in the image data, an image of a static environment object (U1, U2) is detected by object recognition; - by comparing the first and second image data (UD1, UD2) a geometric shift of the representation of a static environment object (U1, U2) that is contained in both the first and second image data (UD1, UD2) is determined (14); and - a change in the vehicle load is determined from the geometric shift of the representation of the static environment object (U1, U2) (15).

Description

[0001] The present invention relates to a method for detecting the load of a vehicle, which can be used in particular to detect unfavorable loads that may affect the vehicle's driving behavior. The present invention further relates to a corresponding device.

[0002] Especially with road vehicles, the payload can have a significant impact on vehicle handling and behavior. It can make a considerable difference whether the vehicle is unloaded, partially loaded, fully loaded, or even overloaded.

[0003] Overloading can endanger vehicle occupants and other road users, for example, through longer braking distances or poorer handling. Similarly, a heavily rear-biased or asymmetrical load can provoke unintended oversteer and therefore dangerous handling, or cause oncoming traffic to be blinded when driving in the dark. Furthermore, driver assistance systems are typically designed to assume a maximum load regardless of the actual cargo, which can counteract the goal of maximizing occupant comfort and safety.

[0004] For trucks, especially those used in the transport industry, checking the payload is regularly observed, not least because of random checks by police authorities to ensure compliance with the permissible total weight and the threat of penalties for overloading. In contrast, this aspect is often neglected when using passenger cars.

[0005] Mobile wheel load scales or integrated axle load weighing systems are used to measure axle loads and total weight, particularly in vehicles such as trucks and trailers. Furthermore, automatic load detection can also be achieved, for example, by analyzing the contact patch of the vehicle tires, as this increases with the weight bearing down on them.

[0006] DE 10 2015 006 571 A1 discloses a device for determining the twisting of a vehicle's body or chassis while the vehicle is in motion, which can be used to detect damage to a vehicle component or the vehicle's load condition. The twisting is determined by jointly evaluating differences between directions of movement and angles of effect in the image data from at least two cameras arranged at different positions on the vehicle.

[0007] DE 10 2018 114 494 A1 relates to a method for detecting the load state of a vehicle using at least one ultrasonic sensor, wherein the method comprises at least the following steps: a) detection of the distance of a vehicle part in a stationary state to a stationary object using the at least one ultrasonic sensor; b) detection of the current distance R of the ultrasonic sensor to the stationary object during a loading / unloading process of the vehicle; c) calculation of the change in distance ΔR from the difference between the distance R and the distance R; d) calculation of the load state B of the vehicle based on the change in distance ΔR and optionally a vehicle-dependent variable. Furthermore, the present invention comprises a driving assistance system suitable for carrying out the method according to the invention and a vehicle with such a driving assistance system.

[0008] DE 10 2014 001 031 A1 provides a method for the assisted loading of a motor vehicle that is loaded with at least one transport load, which is detected by a sensor device and output as loading information by an output device of the motor vehicle. The method comprises detecting a change in the sprung vehicle mass, which is correlated with the at least one transport load, by means of a plurality of sensor devices arranged in the motor vehicle and generating signals corresponding to the change in the sprung vehicle mass and transmitting the signals to a central control unit of the motor vehicle. Furthermore, the method comprises the determination and evaluation of the at least one transport load by the central control unit from the signals, taking into account the unsprung vehicle mass, which is stored in the central control unit.The central control unit then generates at least one load information relating to the at least one transport load and transmits this information to the output device. The output device then displays this load information. Furthermore, the invention discloses a motor vehicle with a device for carrying out the method.

[0009] WO 2015 / 053 434 A1 discloses a device for the decomposition of non-degradable organic substances, capable of decomposing non-degradable organic substances in treated water by electrochemical generation of at least one of the following substances, hydroxyl radicals, ozone or hydrogen peroxide, by means of an electrode, and a water treatment system that uses this device.

[0010] It is an object of the invention to provide an improved method for detecting the load of a vehicle and a corresponding device.

[0011] This problem is solved by a method having the features of claim 1 and by a device according to claim 7. Preferred embodiments of the invention are the subject of the dependent claims.

[0012] In an inventive method for load detection of a vehicle, which has an external camera for capturing the vehicle's surroundings in front of the vehicle, in which first image data are captured and stored by the external camera and second image data are captured by the external camera at a later time, in which an image of a static surrounding object is detected in the image data by object recognition, a geometric shift of the image of a static surrounding object, which is contained in both the first and second image data, is determined by comparing the first and second image data; and a change in the vehicle load is determined from the geometric shift of the image of the static surrounding object.

[0013] Changes in static surrounding objects can thus be used to easily detect the vehicle's payload and any changes in that payload. No additional environmental sensors are required, as modern production vehicles already have a multitude of different sensors installed to monitor the vehicle's surroundings, which can be used for the load detection method according to the invention. This allows for a cost-neutral estimation of the vehicle's load without any additional hardware.

[0014] Advantageously, the first image data is captured when the vehicle's engine is switched off, the parking brake is applied and / or a parking gear is engaged, and subsequently the vehicle's locking system is activated to lock the doors, and the second image data is captured when the vehicle's locking system is then activated to open the doors, and subsequently the engine is started, the parking brake is released, and / or a driving gear is engaged.

[0015] According to one embodiment of the invention, a characteristic map is stored in the vehicle in which different values ​​of the geometric displacement of a statistical environment object are each assigned a corresponding payload of the vehicle, wherein the current payload of the vehicle is determined from the current geometric displacement using the characteristic map.

[0016] Preferably, distance information to the static surrounding object is also captured by sensors.

[0017] According to a further embodiment of the invention, the control behavior of a vehicle dynamics control system is adapted based on the determined payload.

[0018] According to a further embodiment of the invention, a warning is issued to the driver of the vehicle if the vehicle's load is poorly distributed or too large.

[0019] The device according to the invention for load detection of a vehicle, comprising at least one external camera for capturing the vehicle environment in front of the vehicle, a storage unit and an evaluation and control unit, wherein - with at least one outdoor camera, initial image data and, at a later time, second image data are captured; - at least the first image data is stored in the storage unit; - the evaluation and control unit detects an image of a static environment object in the image data through object recognition, determines a geometric shift of the image of the static environment object, which is contained in both the first and second image data, by comparing the first and second image data, and determines a change in the vehicle load from the geometric shift of the image of the static environment object.

[0020] According to one embodiment of the invention, the device - at least one radar, ultrasound and / or lidar sensor is provided for generating distance information to the surrounding object.

[0021] The invention also relates to a vehicle in which the inventive method or device is used.

[0022] Further features of the present invention will become apparent from the following description and the claims in conjunction with the figures. Fig. Figure 1 shows a flowchart of a method according to the invention; Fig. Figure 2 schematically shows an example of a camera image of the vehicle's surroundings with two surrounding objects, which was captured by an external camera of the vehicle when the vehicle was parked and after subsequent loading of the vehicle; and Fig. Figure 3 schematically shows a motor vehicle with a device according to the invention.

[0023] To better understand the principles of the present invention, embodiments of the invention are explained in more detail below with reference to the figures. It is understood that the invention is not limited to these embodiments and that the described features can also be combined or modified without departing from the scope of protection of the invention as defined in the claims.

[0024] Fig. Figure 1 shows a flowchart of a method according to the invention. The method according to the invention can, for example, be implemented by a central control unit for an assistance system in the vehicle in combination with the vehicle's environmental sensors.

[0025] In process step 11, initial image data from the vehicle is generated by sensors. This can occur, in particular, when the vehicle is parked by a user after use on the road. This can be detected, for example, by evaluating control signals from the vehicle's relevant control units, indicating that the engine has just been switched off, the parking brake applied, and / or the transmission has been put into park. Additionally, it can be detected that the vehicle's locking system has subsequently been activated to lock the doors. This ensures that the initial image data is generated when potential loading or unloading occurs, namely between the time the vehicle comes to a standstill and the subsequent time when the user exits the vehicle.

[0026] The image data can be acquired using various environmental sensors on the vehicle. Specifically, the vehicle's surroundings can be captured with an image sensor integrated into a camera, or with multiple image sensors or cameras. These can map the vehicle's surroundings in the visible spectrum or be equipped with infrared cameras to enable reliable capture even in darkness. The image data is assigned to an image sensor or camera coordinate system based on the orientation of the respective image sensor or camera. Such acquisition of the vehicle's surroundings using integrated external cameras is particularly advantageous because camera data with high spatial resolution can be generated, allowing for very precise capture of surrounding objects and their changes.If necessary, the vehicle's surroundings can also be detected additionally or instead using other sensors, such as ultrasonic, radar or laser-based sensors.

[0027] To generate image data from the sensor data produced by the environmental sensors, the sensor data may first undergo processing, depending on the type of environmental sensor. For example, image data captured with a camera requires image processing and object recognition to detect environmental objects within the image data that can be used for load detection.

[0028] For example, static landmarks such as buildings or traffic signs can be detected in front of, to the side of, or behind the parked vehicle and, if necessary, identified and classified. Plants, especially trees and larger bushes, in the vehicle's vicinity can also be recorded. In principle, other parked vehicles in the vicinity can also serve as environmental objects; however, it must be ensured that the data is not distorted by changes in the load of these other vehicles. This can be achieved, for example, by recording the contact area of ​​the tires with the road surface, as this is independent of the other vehicle's load.

[0029] As part of object recognition, the coordinates of each detected object within the captured image are also determined. Specifically, the vertical coordinates of the edges and corners of these objects are calculated. Additionally, the horizontal coordinates of these edges and corners are also determined.

[0030] Furthermore, depth measurements of surrounding objects can also be performed to use the generated distance information for distance plausibility checks. When using a 3D camera, this distance information can be determined directly from the 3D images, or it can be obtained using other sensors such as ultrasonic, radar, or lidar sensors. Based on this distance information, it is also possible to select surrounding objects located as close as possible to the vehicle for the procedure, thus minimizing measurement errors.

[0031] The processing of sensor data can take place directly in the vehicle itself. Alternatively, the vehicle can transmit the raw sensor data to a central server, which then performs the necessary processing and sends the result back to the vehicle. This can be particularly advantageous when complex algorithms are required for unambiguous identification.

[0032] The image data is then stored in process step 2. Here, the coordinates of the key features of the surrounding objects or the complete raw data captured can be stored. Additionally, this data can also be time-stamped.

[0033] In process step 13, a second set of image data is generated at a later time using the same environmental sensors of the vehicle. This data can be used to detect when, after the vehicle has been parked, a user next approaches the vehicle and activates the vehicle's locking system to open the door locks, since loading the vehicle interior is only possible after the trunk or a door has been opened.

[0034] The vehicle environment can then be detected until the engine is started, the parking brake is released and / or a gear is engaged, as it can be assumed that any potential change in the vehicle load is complete at this point.

[0035] As with the first image data, the sensor data may also undergo further processing in the second image data. In particular, environmental objects are detected for load recognition, and the coordinates of each detected environmental object are determined.

[0036] The initially stored image data and the subsequently acquired second image data are then compared in process step 14. For this purpose, it can first be checked whether environmental objects that were included in the original environmental model based on the first image data are still detected in the environmental model according to the second image data. This allows, for example, misinterpretations to be ruled out if a vehicle in front of or behind the vehicle has moved away after the ego vehicle has parked and the corresponding parking space is now empty or occupied by another vehicle. For environmental objects that are included in both the first and second image data, it is then checked whether there is a change, in particular a substantially vertical shift, in the representation of these environmental objects in the acquired sensor data.

[0037] If a change in the image of a static object in the environment is detected, a change in the vehicle load is then determined from this. If several objects in the environment are detected in the acquired sensor data, changes in the image of each of these objects can be determined to increase accuracy, and these changes can then be evaluated together.

[0038] The information thus obtained regarding a change in vehicle load can then be used in process step 16. For example, the control behavior of a vehicle dynamics control system can be adjusted to maintain the vehicle's driving stability. Using the load state estimated according to the invention, the vehicle's steering system, braking system, and / or drive system can be proactively adapted to the current vehicle state after loading or unloading. Likewise, the driver can be warned if the load is poorly distributed or excessive. This warning can, for example, be displayed by an assistance system on a screen in the vehicle. Additionally or instead, an acoustic warning, such as a beep or voice output, can also be provided.

[0039] Fig. Figure 2 schematically shows an example in the form of image data UD1 and UD2, which were captured as camera images of the vehicle's surroundings using an external camera on the vehicle. The camera image includes two static landmarks: a house U1 and a tree U2. When the vehicle is parked at time t1, the landmarks U1 and U2 are initially positioned within the camera image. After the parked vehicle has been loaded, the same surroundings were captured by the external camera at time t2. Analysis using suitable image processing algorithms reveals a change or transformation of the landmarks contained in the camera image; in this case, the surrounding objects in the camera image have been shifted by an amount Δy. The roof peak of house U1 serves as an example.The transformation parameters can be described, in particular, by Cartesian coordinates, which specify the position of the environmental objects in the camera image on the x- and y-axes of a coordinate system. The resulting displacement of the landmarks in the camera image can then be used to infer the amount of payload.

[0040] Provided that a change in load results in a uniform perpendicular shift of the vehicle's chassis relative to the road surface and does not cause rotational movements of the vehicle around its axes, the displacements of the static landmarks Δy in the camera images correspond to the change in vehicle geometry ΔL. The payload can then be determined by calculating the change in spring length within the vehicle based on the geometric displacement in the camera images.

[0041] According to Hooke's law, the spring force F of a spring is proportional to the change in length ΔL of the spring, with the spring constant D as the proportionality factor D. This also applies, to a first approximation, especially as long as nonlinear behavior does not occur due to excessive overall weight, to the spring stiffness of the vehicle suspension. The spring constant D, or spring stiffness of the suspension, can be determined once for each vehicle type by means of a suspension measurement. By equating the spring force with the weight force, which results from the product of the mass m of the payload and the acceleration due to gravity g, m⋅g=F=D⋅ΔL The payload can then be calculated: m=D⋅ΔLg

[0042] If a customized adjustment for a specific vehicle is required, the payload can be determined using a calibrated map. For this purpose, the map is programmed with a calibration chart and known load information and then stored in the vehicle, for example in a control unit.

[0043] Calibrating a front camera can be performed as follows, for example. First, the vehicle is parked in front of a calibration target at a defined position, and the positions of the vehicle's camera and at least one mark on the calibration target are determined, specifically the height of the camera and the at least one mark above the ground. Then, if the rear of the vehicle is loaded, for example, by placing an additional load of known weight in the trunk, the vertical displacement of the mark on the calibration target can be recorded. Alternatively, pressure can be applied to the rear of the vehicle, and the resulting vehicle tilt can be determined.

[0044] Calibration must be performed separately for each vehicle type, as the vehicle sensors, and especially the external cameras, can be installed in different areas of the vehicle depending on the model. Furthermore, calibration can be performed for each vehicle type specifically for the chassis designed for that type, since the extent of changes when the vehicle is loaded also depends on whether, for example, a sporty or a more comfort-oriented suspension is fitted. Alternatively, individual calibration can be performed for each vehicle, for example, during the vehicle assembly process by the manufacturer as part of the standard calibration of the vehicle's sensors.

[0045] Furthermore, additional parameters, such as the effects of temperature on the chassis, can also be taken into account. The chassis includes components such as shock absorbers or vibration dampers, which dampen or reduce spring oscillations and are filled with oil. During this process, kinetic energy is converted into heat energy through fluid friction, which can lead to a significant warming of the oil. Depending on the oil temperature, which depends on the ambient temperature and the vehicle's operating conditions, the oil can exhibit different viscosity and therefore vary in its degree of resistance. These temperature effects can be considered as additional parameters in the characteristic map.

[0046] Furthermore, a vehicle user can program a "baseline level." This can be done, for example, each time the tires are changed between summer and winter tires, similar to resetting a tire pressure monitoring system. This programmed baseline level can then be saved as completely "empty," i.e., without any load, or optionally as "empty + driver." This can be linked, for example, to a seat detection system that provides information about whether the driver's seat is occupied. In this way, it can be distinguished whether the driver is in the vehicle, in which case the camera refers to the programmed baseline level with the driver present, or whether the baseline level without a driver is used as a reference. Based on this baseline level, the system then uses the methods described above to determine how the load has changed.

[0047] In addition to vertical displacement of the captured objects in the image data, it can also be determined if they are obliquely shifted or distorted relative to the originally captured objects. By determining such pitch, yaw, or roll angles compared to the state before loading, an estimation of the payload distribution can be made. Optionally, a warning can then be issued if the load is asymmetrically distributed.

[0048] In Fig.Figure 3 schematically shows a motor vehicle F with a load detection device according to the invention. The motor vehicle has at least one camera K for capturing the area in front of the vehicle. The camera can be installed, for example, in the front grille, but can also be located in other positions that allow for unobstructed detection of the vehicle's surroundings. Additional cameras, not shown in the figure, such as a reversing camera or cameras in the side mirrors, can also be provided. In the illustrated example, the vehicle also has two further, different types of sensors S1 and S2, such as radar sensors, which generate distance information for the objects in the surroundings detected by the camera. This distance information can then be used to further increase accuracy.

[0049] As described above, the camera generates image data of the vehicle's surroundings when the vehicle is parked and later after loading or unloading, preferably high-resolution RGB images with a resolution in the megapixel range. The camera data is fed to an evaluation and control unit A, which can, for example, be part of the vehicle's control unit and may include a processor capable of performing the steps of the method according to the invention. The processor can comprise one or more processing units, such as microprocessors, digital signal processors, or combinations thereof. The evaluation and control unit A detects one or more surrounding objects in the image data using suitable image processing algorithms.

[0050] A storage unit S is provided to store the image data for comparison with later acquired image data. This unit stores the image data or environmental objects recognized in the image data along with their position data. The evaluation and control unit A then determines any changes in the vehicle load by comparing the image data, as described above.

[0051] Additionally, an output unit (not shown in the figure) can be provided, which can issue warnings to the driver if the vehicle load is unfavorable to its handling. For example, the warning can be displayed on a screen in the vehicle or broadcast via the vehicle's audio system speakers. Reference symbol list 11-16 procedural steps U1, U2 surrounding objects UD1, UD2 image data F vehicle K Camera A Evaluation and control unit S1, S2 additional environmental sensors S storage unit

Claims

Method for load detection of a vehicle which has an external camera (K) for capturing the vehicle environment in front of the vehicle, wherein: - first image data (UD1) are captured (11) and stored (12) by the external camera (K) and at a later time second image data (UD2) are captured (13) by the external camera (K); - an image of a static environment object (U1, U2) is detected in the image data by object recognition; - a geometric displacement of the image of a static environment object (U1, U2) which is contained in both the first and second image data (UD1, UD2) is determined (14) by comparing the first and second image data (UD1, UD2); and - a change in the vehicle load is determined (15) from the geometric displacement of the image of the static environment object (U1, U2). Method according to claim 1, wherein the first image data (UD1) are captured (11) when the engine of the vehicle (F) is switched off, the parking brake is applied and / or a parking gear of the transmission is engaged, and subsequently the vehicle's locking system is activated to lock the door locks; and the second image data (UD2) are captured (13) when the vehicle's locking system is subsequently activated to open the door locks, and subsequently the engine is started, the parking brake is released, and / or a driving gear of the transmission is engaged. Method according to claim 1 or 2, wherein a characteristic map is stored in the vehicle (F) in which different values ​​of the geometric displacement of a statistical environment object are each assigned a corresponding payload of the vehicle and the current payload of the vehicle is determined from the current geometric displacement using the characteristic map. Method according to one of the preceding claims, wherein distance information to the static surrounding object is additionally sensorially detected (13). Method according to one of claims 3 or 4, wherein the control behavior of a vehicle dynamics control system of the vehicle is adapted (16) based on the determined payload. Method according to one of claims 3 to 5, wherein a warning is issued to the driver of the vehicle in the event of poorly distributed or large loads on the vehicle (16). Device for vehicle load detection, comprising at least one external camera (K) for capturing the vehicle's surroundings in front of the vehicle, a storage unit (S), and an evaluation and control unit (A), wherein: - the at least one external camera (K) captures initial image data (UD1) and, at a later time, second image data (UD2); - at least the initial image data (UD1) is stored in the storage unit (S); - the evaluation and control unit (A) detects an image of a static environmental object (U1, U2) in the image data by means of object recognition, determines a geometric displacement of the image of a static environmental object (U1, U2) that is contained in both the initial and second image data (UD1, UD2) by comparing the initial and second image data (UD1, UD2), and determines a change in the vehicle load from the geometric displacement of the image of the static environmental object (U1, U2). Device according to claim 7, wherein at least one radar, ultrasonic and / or lidar sensor (S1, S2) is provided for generating distance information to the surrounding object (U1, U2). Vehicle equipped to perform a method according to any one of claims 1 to 6 or comprising a device according to claim 7 or 8.

Images