Method and device for detecting damage to a vehicle

Abstract

The invention relates to a method for detecting damage to a vehicle. According to the invention, it is provided that... - a potential collision of the vehicle with an object in the environment is determined if a predetermined minimum distance between the object in the environment and the vehicle's outer skin is not met, whereby - External images of the vehicle, captured by the vehicle's environmental sensors after a potential collision, are compared with reference images captured and stored before the potential collision in order to determine damage to the vehicle. The invention further relates to a device for carrying out this method.

Description

[0001] The invention relates to a method for detecting damage to a vehicle. The invention further relates to a device for carrying out the method.

[0002] German patent application DE 10 2019 125 320 A1 discloses a method for handling collision damage to a vehicle. The method involves a sensor in the vehicle continuously recording a value of a collision-related physical quantity while the vehicle is in motion. The sensor transmits the recorded value to a control unit in the vehicle, which uses the transmitted values ​​to detect a collision. Upon detection of a collision, a component of the vehicle damaged by the collision is predicted based on the transmitted values, and a camera on a mobile device captures an image of the damaged component.

[0003] EP 3 696 537 A1 describes a device and a method for detecting damage to a moving vehicle. The method comprises: illuminating the moving vehicle with monochromatic or polarized light and capturing image data of the moving vehicle while illuminated with the monochromatic or polarized light; and illuminating the moving vehicle with a first light and capturing first image data of the moving vehicle in an image area while illuminated with the first light, and capturing second image data of the moving vehicle in the image area under a second illumination, wherein the second illumination differs from the illumination with the first light; detecting vehicle damage based on the captured image data or the captured first and second image data.

[0004] Furthermore, a method for damage inspection of motor vehicles is known from DE 10 2017 220 027 A1. In this method, images of the motor vehicle are captured using a mobile smart device containing a camera. From these images, a current three-dimensional form of the motor vehicle is derived, which is then compared with a stored reference form of the motor vehicle to detect damage. The motor vehicle is a vehicle temporarily rented by a user, and application software is installed on the mobile device. This software guides the user on how to walk around the motor vehicle and take pictures from specific positions, showing the motor vehicle from all sides.

[0005] German patent DE 10 2013 102 951 A1 describes a method for monitoring a vehicle, whereby image data from a specific area of ​​the vehicle's surroundings is captured using a camera. The captured image data is recorded when a relevant event is detected that indicates damage, a break-in, or theft. The relevant event is identified by processing the captured image data.

[0006] German patent DE 10 2007 021 405 A1 describes a method for monitoring a parked vehicle using a parking system. The parking system detects an object approaching the vehicle within its range. The vehicle emits a warning signal when the distance between the object and the vehicle falls below a first threshold value.

[0007] DE 10 2014 222 761 A1 describes a method for monitoring a vehicle, comprising the following steps: - Capturing an original state of at least a partial area of ​​the vehicle with a camera attached to the vehicle in a target image, - Capturing a current image of at least a part of the vehicle using the camera attached to the object, and - Identifying differences between the target representation and the actual representation.

[0008] WO 2017 / 176 304 A1 describes a procedure for the automatic estimation of repair costs of a vehicle with the following steps: - Receiving one or more images of a damaged vehicle from a client computer to a server via an electronic network; - Performing computer-aided image processing on each of the images to detect damage to specific vehicle parts; and - Calculating estimated repair costs for the vehicle based on the identified damage using a parts database that contains repair costs.

[0009] The invention is based on the objective of providing a method and a device for detecting damage to a vehicle.

[0010] The problem is solved according to the invention by a method which has the features specified in claim 1 and by a device which has the features specified in claim 9.

[0011] Advantageous embodiments of the invention are the subject of the dependent claims.

[0012] In the inventive method for detecting damage to a vehicle, it is provided that - a potential collision of the vehicle with an object in the environment is determined if a predetermined minimum distance between the object in the environment and the vehicle's outer skin is not met, whereby - After the potential collision, external images of the vehicle captured by the vehicle's environmental sensors are compared with reference images captured and stored before the potential collision in order to determine damage to the vehicle.

[0013] By applying this method, it is possible to detect a potential collision of the vehicle with an environmental object, such as a road user or hail, even while the vehicle is in operation, in order to subsequently determine, based on the reference images and the recorded external images of the vehicle, whether the vehicle is damaged as a result of the potential collision.

[0014] The system automatically triggers an external camera scan of the vehicle the next time it comes to a stop, provided a potential collision is detected. This means that the occupant does not need to manually trigger the recording, even if they were unaware of the potential collision.

[0015] In one version, damage identified through a comparison between reference images and external images is classified, categorized as a dent, a scratch, or more extensive damage. This allows for differentiation between the recorded types of damage, enabling the selection of appropriate measures based on the specific type of damage.

[0016] In one implementation, classification is performed using a model trained with machine learning and / or artificial intelligence. This allows for a precise assignment of the respective vehicle damage without requiring any action from the vehicle's occupant, particularly the driver. The occupant may not yet be aware of the potential collision with an object in the environment, such as another road user and / or hail. Specifically, the model is trained to distinguish between dirt and damage, thus potentially ruling out damage to the vehicle.

[0017] In one possible implementation, after a potential collision of the vehicle is detected, information regarding the potential collision is displayed to an occupant via a user interface, particularly automatically. That is, if a potential collision is detected due to the vehicle falling below the specified minimum distance, especially its outer skin, the vehicle occupant is informed without requiring any action from the occupant.

[0018] Another possible implementation of the procedure involves identifying vehicle damage by comparing reference images with external images. In such cases, information about the identified damage is displayed via an application program installed on the occupant's mobile device. The occupant thus receives information about the detected damage on their mobile device. It is assumed that even if a potential collision is detected and the occupant is informed, but no damage is actually detected, the occupant will not receive this information.

[0019] In a further development of the process, an external photograph of the vehicle showing the damage is transmitted to a repair shop and / or insurance company. For example, it is possible to estimate repair costs based on the identified and classified damage. The application can also automatically schedule a repair appointment. Based on the transmitted external photograph showing the damage, the repair shop can then prepare a cost estimate without having to present the damaged vehicle. For instance, if the damage is classified as significant, the repair appointment can be automatically adjusted to inform the occupants of the estimated repair duration. This also improves the planning reliability of repair shops.

[0020] In one implementation, after a potential collision between the surrounding object and the vehicle has been detected, the exterior images are captured as a three-dimensional point cloud using a lidar-based and / or an ultrasound-based detection unit. Such a three-dimensional point cloud makes it possible to detect dents in the vehicle's exterior. In particular, irregularities in the vehicle's exterior are identified using the point cloud.

[0021] In one version, the captured exterior images are stored on the vehicle itself and / or in a central computer unit that is linked to the vehicle's data processing system. This means that the exterior images are stored on the vehicle and / or can be retrieved by the central computer unit, particularly automatically, when a potential collision involving the vehicle is detected.

[0022] Furthermore, the invention relates to a device for carrying out the method for detecting and classifying damage to a vehicle. According to the invention, the device comprises - an environmental sensor system designed to detect signals by a control unit when the specified minimum distance between an environmental object and a vehicle exterior is breached, and - the control unit which is designed, in the event of a potential collision due to the failure to maintain the specified minimum distance, to activate the environmental sensors to take external images of the vehicle, to compare stored reference images with the recorded external images to determine damage to the vehicle, and to automatically trigger the recording of external images of the vehicle when the vehicle next comes to a standstill.

[0023] The device makes it possible to detect a potential collision between a vehicle and an object in its surroundings, to compare reference images with captured external images, and to determine any damage to the vehicle without requiring any action from the vehicle's occupant. Essentially, the process can be automated using the device without any manual intervention from the occupant.

[0024] Exemplary embodiments of the invention are explained in more detail below with reference to a drawing.

[0025] This shows: Fig. 1 schematically illustrates the process for detecting and classifying damage to a vehicle.

[0026] The single figure shows the sequence of a procedure for detecting and classifying damage to a vehicle.

[0027] To detect damage to a vehicle, reference images of the vehicle are created in a first process step S1, particularly after the vehicle's completion or before a buyer purchases it. This is done, for example, using signals from the vehicle's environmental sensors. Specifically, the reference images are captured using a camera and / or a lidar-based environmental sensor unit. These reference images are stored in the vehicle's control unit and / or in a central computer unit connected to the vehicle.

[0028] In particular, the reference images are stored in the control unit and / or central computing unit in such a way that the reference images can be retrieved in the long term, i.e., are available in the long term.

[0029] In one configuration, the camera is designed as a 360° camera, primarily capturing reference images from the sides of the vehicle. Using this camera and / or an additional camera, reference images of the vehicle roof, hood, and, if present, the trunk lid are captured, depending on the specific viewing angle, i.e., the detection area. By integrating additional cameras, all areas of the vehicle can be included within a given detection area, allowing for the creation of reference images.

[0030] At least one additional lidar-based detection unit can also be arranged on the vehicle roof, whereby a three-dimensional point cloud is generated as a reference image using the lidar-based detection unit and / or an ultrasound-based detection unit.

[0031] In a second process step S2, a potential collision of the vehicle with an object in the environment is detected.

[0032] To detect a potential collision when the vehicle is parked, it is equipped with a collision detection system. Upon detection of a collision, the vehicle's environmental sensors are activated to capture images of the vehicle's exterior. If a potential collision is detected while the vehicle is parked, these images are compared with previously acquired reference images, particularly to determine if the vehicle has sustained any damage.

[0033] During vehicle operation, the distance between the vehicle and an object in the environment, such as a road user or an obstacle, can be determined based on signals received from the environmental sensors, in particular signals from a radar-based, ultrasound-based, and / or lidar-based detection unit. If a predefined distance between the vehicle's exterior and the object falls below a certain threshold, a potential collision between the vehicle and the object is detected.

[0034] The detection of a potential collision of the vehicle is a trigger criterion for capturing external images using the vehicle's environmental sensors when the vehicle next comes to a standstill.

[0035] Furthermore, a device used to carry out the procedure can transmit detected signals to a collision sensor system, so that an indication of the potential collision can be used for further analysis. If occupant protection devices, such as airbags, are triggered due to the detected potential collision, the collision sensor system also has information to classify the severity of the collision.

[0036] Regardless of the determined and classified severity of the collision, two-dimensional and / or three-dimensional external images of the vehicle are recorded the next time the vehicle comes to a standstill, provided the environmental sensors are functional.

[0037] If a potential collision of relatively high severity is detected, the vehicle already has the necessary information about the potential collision, and may automatically place an emergency call. The airbags have most likely deployed, and the vehicle is generally no longer drivable.

[0038] A relatively minor potential collision of the vehicle, such as so-called parking bumps or a potential collision at a relatively low speed, primarily results in sheet metal damage, especially in the form of small dents and / or paint scratches.

[0039] In a third procedural step S3, it is determined whether the vehicle was damaged as a result of the potential collision.

[0040] A dent in a vehicle's exterior is characterized primarily by a change in the lighting conditions on a surface, namely the vehicle's exterior. The lighting conditions known from the reference images are altered, and shadows appear that are caused by the dent. Additionally, brighter areas are visible, which are more illuminated due to the dent.

[0041] Furthermore, a dent alters the distance, for example, between the lidar-based detection unit and the surface. An undamaged surface of the vehicle's exterior no longer has a smooth contour due to the dent, but rather irregularities. A dent, such as hail damage, increases the distance to the lidar-based detection unit, with larger dents potentially having an outwardly raised edge, thus reducing the distance to the lidar-based detection unit. It is also possible to identify a dent by analyzing the captured exterior images of the vehicle using artificial intelligence.

[0042] Paint scratches cannot typically be detected using signals from a lidar-based detection unit. A color change resulting from paint abrasion caused by a collision differs from the rest of the vehicle's paint color. Therefore, a paint scratch can be identified based on a brightness analysis of reference images in relation to the exterior images of the vehicle taken after the potential collision, in conjunction with a color analysis of the exterior images.

[0043] By comparing the recorded exterior images with the stored reference images, scratches and dents in the vehicle's outer skin can be identified as damage.

[0044] Using machine learning and / or artificial intelligence, a model can be trained to recognize what damage to a vehicle's exterior might look like. This model can then be used to detect changes to a vehicle's body, particularly its exterior. Specifically, a machine learning and / or artificial intelligence-trained model can be used in conjunction with sensor fusion to definitively identify damage to the vehicle. The model can also be trained to detect contamination, thus potentially ruling out damage.

[0045] In one possible implementation of the procedure, a vehicle occupant, particularly the driver, can be instructed to take an exterior photograph of the vehicle, especially using their mobile device. Using a vehicle-specific application stored on the mobile device, the occupant can be given instructions on how to hold the device and how to take the photograph in order to determine the extent of the detected damage. For example, the occupant is advised that at least one reference point, such as a side mirror and / or a door handle, must be visible in the photograph. Distances are generally known, so a given distance can be compared to a scratch and / or dent to determine the extent of the damage. This information can be used from the reference images and / or exterior photographs generated by the vehicle.

[0046] Distortions, for example caused by external images of the vehicle taken using a so-called fish-eye camera, must be taken into account when designing the device for carrying out the procedure and must be factored out accordingly.

[0047] In one possible implementation, the recorded external images are transmitted via the vehicle-specific application program to the central computer unit, which can then be used to analyze the damage to the vehicle.

[0048] If the analysis is to be performed using artificial intelligence, the information is analyzed either via the vehicle's control unit or via the central processing unit. Depending on the size of the artificial intelligence system, an algorithm may be available in the central processing unit, allowing the vehicle's external images to be transmitted to it. These images are then analyzed, and the resulting analysis data is made available to the application program and / or a workshop.

[0049] Furthermore, the procedure stipulates that any detected damage, particularly to the vehicle's exterior, is classified, for example using machine learning and / or artificial intelligence. Specifically, a distinction is made between a dent, a scratch in the paint, and more extensive damage.

[0050] Furthermore, the system uses position signals from a satellite-based positioning unit and / or signals from the vehicle's light sensor to determine the time of day and identify whether a current time is suitable for capturing exterior images of the vehicle. If it is determined that it is night, the recording time is postponed to the next day. Precipitation can be determined using signals from a rain sensor, taking into account, for the evaluation and analysis of the exterior images in relation to the reference images, that the vehicle is wet, i.e., that there are droplets on the vehicle's exterior. Light is important for determining the optimal recording time in order to identify differences in brightness and / or color.

[0051] In a third process step S3, if a potential collision of the vehicle is detected, an occupant, in particular the driver, is informed. For example, the occupant is notified of the potential collision by means of a message displayed in a screen area of ​​the vehicle's display unit. Alternatively or additionally, the occupant receives a corresponding message via the vehicle-specific application program and / or another suitable user interface. The vehicle's external images are made available if a potential collision, in particular resulting damage to the vehicle, has occurred.

[0052] For relatively minor damage, the device and / or the vehicle-specific application program can, in a fourth process step (S4), provide an initial rough cost estimate for repairing the damage. A user / occupant can also request and perform a short test. With the customer's consent, the result of this short test is saved and / or output at regular intervals and is, for example, at least partially accessible for after-sales purposes. If a collision is detected, a short test can be accessed to better classify the damage. Data from the fault environment that has been stored also provides information about the severity of an accident. For example, updating and calibrating the environmental sensors can also lead to costs.

[0053] Furthermore, in the fourth process step, S4, a workshop appointment is scheduled using the application program. Both the reference images and the captured exterior images are transmitted directly to the workshop, allowing for the creation of a cost estimate. If the damage is extensive, the workshop appointment can be automatically adjusted so that the vehicle's occupant, especially the driver, gets an initial idea of ​​the expected repair duration. This improves the workshop's planning reliability.

[0054] In a fifth process step (S5), if motor vehicle insurance exists and an insurance claim has been filed, the reference images and / or the exterior images of the vehicle taken after the potential collision, and any other relevant vehicle-related information, are forwarded to the vehicle's insurance company. For example, an insurance application program may be stored within or linked to the vehicle-specific application program. The insurance application program may also be integrated into a vehicle-specific app store.

Claims

[1] Method for detecting damage to a vehicle, wherein - a potential collision of the vehicle with an object in the environment is determined if a predetermined minimum distance between the object in the environment and the vehicle's outer skin is not met, - External images of the vehicle captured by the vehicle's environmental sensors after the potential collision are compared with reference images captured and stored before the potential collision in order to determine damage to the vehicle and - a recording of the vehicle's exterior images is automatically triggered the next time the vehicle comes to a stop. [2] Method according to claim 1, characterized by, that damage identified by comparison between reference images and external images is classified, classifying the damage as being a dent and / or a scratch and / or more extensive damage. [3] Method according to claim 2, characterized by that the classification is carried out using a model trained using machine learning and / or artificial intelligence. [4] Method according to any one of the preceding claims, characterized by , that after a potential collision of the vehicle has been detected, information regarding the potential collision is displayed on a user interface for an occupant. [5] Method according to any one of the preceding claims, characterized by, that after damage to the vehicle has been identified by comparing the reference images with the external images, information about the identified damage is output via an application program stored on a mobile device of the occupant. [6] Method according to claim 5, characterized by that a photograph showing the damage will be sent to a repair shop and / or an insurance company. [7] Method according to any one of the preceding claims, characterized by that, after determining a potential collision between the surrounding object and the vehicle, the external images are recorded at least as a three-dimensional point cloud using a lidar-based detection unit and / or an ultrasound-based detection unit. [8] Method according to any one of the preceding claims, characterized bythat the external images recorded are stored on the vehicle side and / or in a central computer unit that is linked to the vehicle in terms of data technology. [9] Apparatus for carrying out a method according to any of the preceding claims, characterized by - an environmental sensor system designed to detect signals by a control unit when the specified minimum distance between an environmental object and a vehicle exterior is breached, and - the control unit which is designed to activate the environmental sensors to record external images of the vehicle in the event of a potential collision due to the failure to maintain the specified minimum distance, to perform a comparison between stored reference images and the recorded external images to determine damage to the vehicle, and to automatically trigger the recording of external images of the vehicle when the vehicle next comes to a standstill.

Images