Method and computer program for analyzing external vehicle condition

Abstract

The invention relates to a method (10) for analyzing an external vehicle condition of a vehicle (F1). The method comprises the following steps: positioning (11) the vehicle in front of the reflective surface (21) such that the vehicle reflected in the reflective surface (21) is located in the detection region of the camera (21a) or the identification vehicle (F1) is located in front of the reflective surface in such a way; recording (12) an image of the reflected vehicle by means of the camera (21a); -analysing (13) the image with respect to a vehicle condition; and-outputting (14) information comprising the vehicle condition.

Description

Technical Field

[0001] The exemplary embodiment relates to a method for analyzing an external vehicle condition of a vehicle. In particular, a camera can be used for optical analysis, so that dirt or damage to the vehicle can be detected in an image of the vehicle. In addition, a corresponding computer program product for executing the method is proposed. Background Art

[0002] Motor vehicles are usually designed to have an attractive exterior appearance. Motor vehicle users want to use clean, undamaged and perfectly presented vehicles. Therefore, when renting or leasing motor vehicles or providing transport services (such as taxi services), it is necessary that the vehicles are always well maintained, i.e., clean and free of dents, for example. Technical Field

[0003] When a rented or leased vehicle is returned, it is usually inspected, during which it is also determined whether the vehicle is excessively dirty or has dents. However, it can be difficult to determine the extent of dirtiness or possible damage to a rental vehicle in a sufficiently quantitative manner, which can lead to disputes between the vehicle lessee and the lessor. Moreover, in particular in the case of large fleets, regular inspections of each vehicle for dirtiness or damage would result in considerable labor expenditure.

[0004] The automatic detection of dirt and damage on vehicles in a fleet is therefore an important problem. This applies in particular to fleets of autonomous vehicles, since there is no human driver here who can perform an optical inspection for dirt or damage during operation.

[0005] WO 2021 055 988 A1 discloses a system and method for vehicle self-inspection. The self-inspection can be implemented by an application executed on a mobile device, which is configured so that it determines excessive wear and use. The mobile device can obtain images of multiple predetermined views of the vehicle, wherein damage inputs are determined for the multiple predetermined views. Then, a cost estimate for repair or replacement caused by damage in each of the multiple predetermined views and a total cost for repair or replacement can be output, and the cost can be dynamically updated.

[0006] US 2019266715 A1 discloses methods, systems, and computer program products for inspecting an autonomous vehicle using an unmanned aerial vehicle (UAV). The autonomous vehicle carries a UAV (or drone) in a protected area, such as a glove box, trunk, etc. The UAV can be used to inspect the autonomous vehicle between trips. Images from the UAV can be sent to other components for image analysis. After the inspection is completed, the UAV can return to the protected area. The UAV can inspect both the interior and exterior of the autonomous vehicle. If the inspection passes, the autonomous vehicle can start a new trip. If the inspection fails, the autonomous vehicle can report for repair or call a tow truck.

[0007] A disadvantage of the known solutions may be that additional equipment is required, such as a mobile device (e.g. a smartphone) or an additional drone that must be placed in the vehicle in order to be able to determine the vehicle's condition. This may be impractical and lead to high costs for vehicle inspections. Summary of the invention

[0008] The object of the present disclosure is to provide an improved approach for carrying out a check on an external vehicle condition of a vehicle.

[0009] This object is achieved according to the subject matter of the independent claim. Further advantageous embodiments are described in the dependent claims, in the following description and in conjunction with the drawings.

[0010] Therefore, a method for analyzing an external vehicle condition of a vehicle is proposed. In particular, an optical or visual analysis or inspection of the external vehicle condition is performed. The method includes positioning the vehicle in front of a reflective surface so that the vehicle reflected in the reflective surface is located in the detection area of ​​a camera. Alternatively, according to the method provisions, it is recognized that the vehicle is positioned in front of the reflective surface in this way (for example, if a suitable reflective surface already exists in the vehicle environment, so that the vehicle does not have to move there; for example, if the vehicle driver has manually positioned the vehicle in front of the reflective surface or is driving past the reflective surface). For example, the vehicle can be parked in front of the reflective surface or alternatively can also drive past the reflective surface, so that the vehicle is reflected in the surface. The vehicle can be detected in front of the reflective surface by means of vehicle software and vehicle sensor devices, and then further method steps can be prompted. The further specified step is to capture an image of the reflected vehicle by means of a camera, analyze the image with respect to the vehicle condition and output information including the vehicle condition.

[0011] Advantageously, the method is carried out automatically. As a result, the vehicle itself can detect whether its vehicle exterior is normal (e.g. clean) or, for example, has dirt or damage such as dents or scratches, as soon as the vehicle is in front of the reflective surface. By outputting the information, suitable measures can be taken, for example, to restore the vehicle exterior to a normal state. By avoiding necessary manual steps, an efficient self-check of the vehicle can be carried out. Advantageously, an external camera, such as a camera of a mobile phone or a drone, can be omitted.

[0012] As already mentioned, according to one aspect it is provided that the vehicle condition relates to dirt or damage on the vehicle exterior. The use of a camera in combination with a reflective surface can be particularly simple, since the proposed method focuses on checking the vehicle exterior and does not require other complex measures to check the vehicle interior. Analyzing whether dirt or dents are present on the vehicle is particularly important, since this can occur frequently. Since such checks must be performed frequently, an efficient method as proposed herein can be particularly advantageous for this.

[0013] According to one aspect, it is provided that a plurality of images of the reflected vehicle are recorded from different recording angles or a video sequence of the reflected vehicle is recorded. For example, the vehicle can be maneuvered in front of the reflective surface in order to be able to analyze multiple sides of the vehicle. For example, the vehicle is moved in front of the reflective surface during the recording of the video sequence, for example, the vehicle is autonomously maneuvered in front of the reflective surface. In particular, the angle of the vehicle relative to the reflective surface can be changed in order to be able to better identify potholes from the resulting reflections, for example. Alternatively, it can be provided that, for example, if reflective surfaces are present on multiple sides of the vehicle, a pivotable camera (for example of the vehicle itself) is used and pivoted. In this way, a larger area of ​​the outer surface of the vehicle can be examined.

[0014] According to one aspect, a camera on board the vehicle itself is used to record the image. Since many vehicles are already equipped with a camera (e.g. a reversing camera or a front camera for driver assistance functions), the method can be used particularly efficiently for these vehicles. Alternatively, for example, a smartphone camera can also be used, which is coupled to the vehicle, for example, in order to record the image of the reflected vehicle. For example, a camera on board another (e.g. autonomously driven) vehicle can also be used to record the image.

[0015] For example, the method can also include adjusting the route guidance of the vehicle in order to guide the vehicle to a suitable reflective surface. In particular, the route guidance can be adjusted before the above steps. Route guidance along glass facades or the like can be particularly advantageous, since in this way images can be recorded while driving. For example, a plurality of reflective surfaces can be driven past successively, which can be located on different sides of the vehicle. Thus, for example, the entire outer surface of the vehicle can be checked.

[0016] As already mentioned, it is provided according to one aspect that the vehicle is an autonomous vehicle. The autonomous vehicle can, for example, provide passenger transport. By means of the method, the autonomous vehicle can, for example, automatically check whether its exterior appearance is suitable before each trip. For example, the permissible limits in terms of dirtiness can be passenger-dependent, so that some passengers may be able to accept a certain dirtiness of the vehicle exterior, while others may not. In this way, fleet operation can be run more smoothly, for example for users who have lower requirements in terms of exterior appearance. A user of the transport service can, for example, indicate in his personal profile whether he would prefer to use a dirty vehicle if necessary or whether he still wants to wait for a clean vehicle.

[0017] For example, the method may also include navigating the vehicle to a workshop or a car wash line according to the vehicle condition. Navigation to the workshop or car wash line can be performed in particular after the above steps. For example, navigation information can be output to the vehicle driver or the vehicle can drive to the workshop or car wash line autonomously. In particular, it is provided that the vehicle can only be navigated to the workshop or car wash line when the vehicle condition is worse than a predetermined (for example, maximum) dirt or damage tolerance value. This tolerance value can, for example, describe the maximum acceptable dirt or damage. For example, the vehicle can only be navigated to the workshop or car wash line when too heavy dirt or dirt over a large area or too large pits or scratches (which make it necessary to repair in the workshop or wash the vehicle) have been identified. Therefore, it can be advantageously quantitatively determined whether the vehicle can be used directly again or whether the external appearance must be restored first.

[0018] According to one aspect, the method is used to analyze external vehicle conditions of a large number of vehicles of a fleet, in particular an autonomous fleet. In particular, in the case of a large number of autonomous vehicles, it can be advantageous if these vehicles can perform a self-check, since no driver is able to perform the check and non-automatic checking would result in high manual effort.

[0019] Another aspect relates to a method for analyzing an external vehicle situation of a first vehicle. The method comprises the following steps: positioning the first vehicle in front of another autonomous vehicle with an onboard camera, so that the first vehicle is located in the detection area of ​​the onboard camera of the other vehicle. The positioning can be performed from the perspective of both vehicles, for example the first vehicle can move towards the other vehicle and / or the other vehicle can move towards the first vehicle. Alternatively, it can be recognized that the vehicle is so positioned in front of the other vehicle (for example if the other vehicle with a camera is already in the vehicle environment of the first vehicle, so that the first vehicle does not have to move there again; for example if the driver of the first vehicle has manually positioned the first vehicle in the environment of the other vehicle or is passing by the other vehicle; for example if the other vehicle is, for example, autonomously passing by the first vehicle). Recognition that the first vehicle is in front of the other vehicle can be achieved with the aid of vehicle software and vehicle sensor devices, and further method steps can then be prompted. The further specified step is to capture an image of the first vehicle with the aid of the onboard camera of the other vehicle, analyze the image with respect to the vehicle situation of the first vehicle and output information including the vehicle situation of the first vehicle. The other vehicle can be of the same vehicle type as the first vehicle, for example both vehicles can be sedans.

[0020] The advantage of the method over the prior art may be that the inspection can be carried out using an external camera without having to carry an additional drone or the like in the vehicle for this purpose. Instead, the proposed method may use a camera of another road vehicle, for example if this road vehicle happens to be passing by the first vehicle randomly or in a planned manner or is in its environment. The method may be advantageous in particular if at least some of the vehicles in the fleet are already equipped with cameras, since these cameras can be used efficiently and simultaneously to carry out the method.

[0021] Another aspect relates to a computer program having a program code for carrying out the above-mentioned or the following method when the computer program is executed on a processor, a computer or programmable hardware. Such a computer program can advantageously be installed on an onboard computer of a vehicle (e.g. an autonomous vehicle) or provided on a server (e.g. to carry out the method in a cloud-based manner). BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The following is a detailed description of the embodiments with reference to the accompanying drawings.

[0023] Figure 1 A flow chart illustrating a method for analyzing external vehicle conditions of a vehicle;

[0024] Figure 2shows a schematic example of a vehicle positioned in front of a reflective surface so as to capture a reflected image of the vehicle with the aid of an onboard camera; and

[0025] Figure 3 A schematic example of a vehicle is shown which is maneuvered into the detection range of a camera of another vehicle. DETAILED DESCRIPTION

[0026] Various embodiments will now be described in more detail with reference to the accompanying drawings, in which some embodiments are shown. In the accompanying drawings, the thickness dimensions of lines, layers and / or regions may be exaggerated for clarity. In the following description of the drawings showing only a few exemplary embodiments, the same reference numerals may represent the same or equivalent components.

[0027] An element that is said to be "connected" or "coupled" to another element may be directly connected or coupled to the other element or there may be elements located therebetween. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as given to them by ordinary professionals in the field to which the embodiments belong.

[0028] Today, dirt and damage on vehicles are detected manually by vehicle users. Or the individual degree of dirtiness / damage is determined by humans, for example in a fleet of vehicles. These processes are resource-intensive and require humans. Inventions are known that determine the degree of dirtiness / damage based on on-site data (e.g. mileage, environmental conditions). However, these methods are not as precise as optical inspections using image information.

[0029] Figure 1 A flow chart of a method 10 for analyzing an external vehicle condition of a vehicle is shown. The method 10 comprises the steps of positioning 11 the vehicle, capturing 12 images, analyzing 13 the images and outputting 14 information including a vehicle condition of the vehicle.

[0030] According to a first embodiment of the method 10, a vehicle is positioned 11 in front of a reflective surface so that the vehicle reflected in the reflective surface is located in the detection area of ​​a camera. Alternatively, it can also be recognized (for example by means of a vehicle system) that the vehicle is thus positioned in front of the reflective surface. In this case, an image of the reflected vehicle can be recorded 12 by means of a camera. The image can be analyzed 13 with respect to the vehicle's vehicle condition.

[0031] In this embodiment, it can be advantageous that the vehicle self-check can be performed without the need for other external electronic devices. For example, a smartphone or a drone capable of taking images can be omitted for the vehicle exterior analysis. Images can be taken by making advantageous use of reflective surfaces present in the environment and, for example, existing onboard cameras of the vehicle itself. This allows a very efficient vehicle self-check.

[0032] According to a second embodiment of the method 10, a vehicle, in particular a first vehicle, is positioned in front of another autonomous vehicle having an onboard camera 11, so that the first vehicle is located in the detection range of the onboard camera of the other vehicle. Alternatively, it can be recognized that the first vehicle is thus positioned in front of the other vehicle. An image of the first vehicle is captured 12 by means of the onboard camera of the other vehicle, and the image is subsequently analyzed 13 with respect to the vehicle status of the first vehicle. Finally, information including the vehicle status of the first vehicle is output 14.

[0033] In the second embodiment, it can be advantageous that no reflective surfaces are required to carry out the analysis of the exterior appearance of the first vehicle. Rather, an existing onboard camera of another vehicle can be utilized in an efficient manner, which can be useful in particular when analyzing a convoy of multiple vehicles, since the vehicles can check each other. In the proposed second embodiment, a smartphone or a drone capable of taking images can also be omitted for analyzing the vehicle exterior.

[0034] The proposed method solves the problem of complex inspections of vehicles by the vehicle itself or by the interaction between different vehicles. In this case, the vehicle recognizes, for example, through its installed optical sensor device (such as a camera) that it is, for example, passing by a reflective object. This can be achieved, for example, by a corresponding KI (artificial intelligence) model. Subsequently, the reflective surfaces are photographed and evaluated, for example, with the help of another KI model that recognizes dirt and damage. The vehicle can thus recognize its own dirt and damage without the need for external devices with cameras. The method can also be expanded by interaction with other autonomous vehicles. In this case, the camera systems of other vehicles can be used to determine images of the outer surfaces of the vehicle in order to optically resolve all areas of the vehicle. The same KI model as described above can be used here.

[0035] An illustrative example can further describe the proposed method: A (e.g. autonomous driving) vehicle is equipped, for example, with software for detecting dirt and damage. The vehicle detects, via a sensor device (in particular a camera) installed for the autonomous driving function, that it will be passing a reflective surface (e.g. a glass front of a building) in a short time. While passing by the reflective surface, the vehicle uses sensors to capture an image of the glass front and thus an image of itself (reflected in the glass front). These images are evaluated by an installed (or cloud-hosted) KI model with regard to damage and dirt. If damage or dirt is detected, this information is transmitted, for example, to a control center (e.g. a fleet management center of a fleet of vehicles). Suitable measures can now be taken, for example, to repair or clean the vehicle.

[0036] Figure 2A schematic example 20 of a vehicle F1 is shown, which is positioned in front of a reflective surface 21 in order to capture a reflected image of the vehicle F1 by means of an onboard camera 21a. The reflective surface may be, for example, a glass facade or a reflective surface of another vehicle or object, such as a traffic reflector.

[0037] The vehicle F1 may also have a front camera 21a and a rear camera 21b. Depending on the position of the reflective surface 21, it may be advantageous to use the front camera 21a or the rear camera 21b to photograph the reflected vehicle F1. Figure 2 In the example of FIG. 2 , the front side of the vehicle F1 faces the reflective surface 21, so that the front camera 21a can capture the reflection of the vehicle in the reflective surface 21. In this way, an image can be provided that enables the front side of the vehicle F1 to be analyzed with respect to the external vehicle condition. For example, dirt or pits on the front side of the vehicle F1 can be determined on the image.

[0038] If necessary, vehicle F1 can be maneuvered in front of reflective surface 21, for example, to make a U-turn, so that rear camera 21b can photograph vehicle F1 reflected in reflective surface 21. In this way, the rear side of vehicle F1 can also be analyzed with respect to external vehicle conditions.

[0039] The vehicle F1 can, for example, drive in front of the reflective surface 21 in order to be able to record an image. In this case, the vehicle F1 can, for example, stop in front of the reflective surface 21. Alternatively, the vehicle sensor system can also detect when passing by the reflective surface 21 that an image of the vehicle suitable for analysis can be recorded when passing by, so that the vehicle F1 does not have to stop for this purpose, but can continue its travel without interruption. The vehicle control device can be correspondingly designed to detect reflective objects (for example by means of the vehicle sensor system; for example by means of the onboard cameras 21a, 21b) and to evaluate the reflected vehicle surface (for example based on the image).

[0040] For example, one or more of the following steps can be implemented: filming by the vehicle (e.g. a camera on the side, e.g. located in a side mirror of vehicle F1); if a vehicle is recognized (e.g. in the reflecting surface 21, e.g. by means of object recognition, e.g. based on KI), checking whether the vehicle recognized in the image remains stationary relative to vehicle F1 (e.g. in order to check whether the vehicle actually recognized itself); classifying the reflected vehicle (e.g. by comparison with vehicle features) to ensure that it is the vehicle itself (e.g. to avoid analyzing other vehicles); creating image parts and segmentations (e.g. of images); and / or object recognition and classification to detect dirt or damage.

[0041] Furthermore, another vehicle F2 (schematically shown) can record an image of the vehicle F1 using a camera of the other vehicle F2. This can also provide an image of the vehicle exterior that is suitable for analyzing the vehicle F1. The image can be transmitted directly from the other vehicle F2 or via a cloud (not shown) to the vehicle F1. The image can be analyzed, for example, in the recorded vehicle F1, in the other vehicle F2 or also in the cloud.

[0042] Further details and aspects are mentioned in conjunction with the embodiments described above or below. Figure 2 The embodiment shown in may include one or more optional additional features, which correspond to the combination of the proposed solution or the above (such as Figure 1 ) or below (such as Figure 3 ) One or more aspects mentioned in one or more embodiments described in the present invention.

[0043] Figure 3 A schematic example 30 of a first vehicle F3 is shown, which is maneuvered into the detection area of ​​a camera 21a of another vehicle F1. Depending on the relative relationship of the two vehicles F3, F1, the first vehicle F3 and / or the other vehicle F1 can be moved for this purpose. For example, the other vehicle F1 can be an autonomous vehicle, which can be positioned in front of the first vehicle F3 (forward and backward travel is schematically indicated by arrows) so that the first vehicle F3 is positioned in the detection area of ​​the onboard camera 21a. An image of the first vehicle F1 can be recorded.

[0044] Thus, for example, an image of the first vehicle F3 can be captured by another autonomously moving vehicle F1. When the other vehicle F1 recognizes the vehicle structure, for example, an image can be captured automatically. The image can be sent to a backend 32 or a cloud 32, wherein the vehicle can be connected to the cloud 32 via corresponding communication links 31a, 31b.

[0045] The vehicle surface of the first vehicle F3 may be evaluated based on the image (for example in the further vehicle F1 or in the cloud 32 ) and information about the evaluation may be sent to the first vehicle F3 (or to the cloud 32 ).

[0046] Further details and aspects are mentioned in conjunction with the embodiments described above or below. Figure 3 The embodiment shown in may include one or more optional additional features, which correspond to the combination of the proposed solution or the above (such as Figure 1-2 ) or one or more aspects mentioned in one or more embodiments described below.

[0047] The example concerns autonomous dirt detection using indirect methods and artificial intelligence. It is provided that a vehicle control unit or a vehicle system detects a reflective object, captures an image of the vehicle reflected therein, and also evaluates the vehicle surface reflected in the reflective object. If damage or dirt on the vehicle surface is detected, this information can be transmitted and, for example, repair or cleaning measures can be taken.

Claims

1. A method (10) for analyzing external vehicle conditions of a vehicle (F1), the method comprising: - positioning the vehicle (11) in front of the reflective surface (21) so that the vehicle reflected in the reflective surface (21) is located in the detection area of ​​the camera (21a) or the vehicle (F1) is detected as being positioned in front of the reflective surface; - recording (12) an image of the reflected vehicle by means of the camera (21a); - analyzing (13) the image with respect to the vehicle condition; and - The output (14) includes information about the vehicle's condition.

2. The method (10) according to claim 1, wherein: The vehicle condition relates to soiling or damage of the vehicle exterior of the vehicle ( F1 ).

3. The method (10) according to claim 1 or 2, wherein: A plurality of images of the reflected vehicle are recorded (12) from different recording angles or a video sequence of the reflected vehicle is recorded (12).

4. The method (10) according to any one of the preceding claims, wherein: In order to capture (12) the image, the onboard camera (21a, 21b) of the vehicle (F1) itself is used.

5. The method (10) according to any one of the preceding claims, further comprising adjusting the route guidance of the vehicle (F1) so as to navigate the vehicle (F1) to a suitable reflective surface (21).

6. The method (10) according to any one of the preceding claims, wherein: The vehicle (F1) is an autonomously driven vehicle (F1).

7. The method (10) according to any of the preceding claims, further comprising navigating the vehicle (F1 ) to a workshop or a car wash line depending on the vehicle condition.

8. The method (10) according to claim 7, wherein: If the vehicle condition is worse than a predetermined minimum allowable value of dirtiness or damage, the vehicle (F1) is navigated to a workshop or a car wash.

9. The method (10) according to any one of the preceding claims, wherein: The method (10) is used to analyze external vehicle conditions of a plurality of vehicles (F1, F2, F3) of a fleet.

10. A method (10) for analyzing an external vehicle condition of a first vehicle (F3), the method comprising: Positioning (11) a first vehicle (F3) in front of another vehicle (F1) that is automatically driven and has an onboard camera (21a), so that the first vehicle (F3) is located in a detection area of ​​the onboard camera (21a) of the other vehicle (F1) or the first vehicle (F3) is recognized to be positioned in front of the other vehicle (F1); Taking (12) an image of the first vehicle (F3) by means of a camera (21a) on board the other vehicle (F1); analyzing (13) the image with respect to a vehicle condition of the first vehicle (F3); and The output (14) includes information about the vehicle status of the first vehicle (F3). 11 . Computer program having a program code in order to carry out the method ( 10 ) according to claim 1 , when the computer program is executed on a processor, a computer or programmable hardware.

Images