Method and apparatus for capturing image outside motor vehicle and motor vehicle

Abstract

To provide a method and an apparatus for capturing an image outside a motor vehicle and the motor vehicle.SOLUTION: A method (100) and an apparatus (1000) for capturing an image outside a motor vehicle (10) and the motor vehicle (10) having the apparatus (1000) are disclosed. The method (100) includes the steps of: capturing an image outside the motor vehicle (10) using a camera device (40), the captured image being distorted due to the nature of the camera device (40); capturing a property of the camera device (40); equalizing the image on the basis of the property of the camera device (40); checking the equalized image on the basis of the property of the camera device; and providing the equalized image to a user of the motor vehicle (10).SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a method and apparatus for capturing images of the exterior of a motor vehicle, and to a motor vehicle having such an apparatus. [Background technology]

[0002] Motor vehicles with distracting accessories that protrude into the road and are not or are difficult to see from the driver's seat require electronic monitoring using camera devices, but such monitoring is particularly computationally intensive, especially when compliance with one or more safety standards is intended or must be achieved. Summary of the Invention [Problem to be solved by the invention]

[0003] SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide an improved method and an improved apparatus for capturing images of the exterior of a motor vehicle.

[0004] This object is achieved by the subject matter of the independent claims. Advantageous developments are specified in the dependent claims. [Means for solving the problem]

[0005] According to a first aspect, a method for capturing an image of the exterior of a motor vehicle is provided.

[0006] Here, the method according to the present invention includes the steps of capturing an image of the exterior of the vehicle using a camera device, the captured image being distorted due to the properties of the camera device, capturing the characteristics of the camera device, equalizing the image based on the properties of the camera device, checking the equalized image based on the properties of the camera device, and providing the equalized image to a user of the vehicle.

[0007] The method according to the invention is used in particular to provide a simple, cost-effective and possible way of checking the equalization of camera images for correctness and functionality.

[0008] In particular, embodiments of the method according to the invention make it possible to recognize or verify the mounting position of a camera device relative to its environment, in particular its orientation and / or position or coordinate system, thereby identifying and correcting incorrect mountings, in particular tilts, displacements, twists and / or rotations of the camera and / or mount of the camera device.

[0009] Motor vehicles are in particular vehicles with oversized exterior dimensions, such as transport and / or agricultural vehicles, and in particular also equipped with attachments and / or equipment, such as trailers, buckets, forks, troughs and / or mowing devices, in particular rotary mowers.

[0010] For this purpose, at least one image of the exterior of the motor vehicle is captured by means of a camera device, which is a device designed to record one or more, in particular successive, images of the exterior of the motor vehicle, in particular the camera device is designed to record the images of the exterior continuously and / or periodically, for example in a video or video stream.

[0011] For this purpose, a camera device, which may also be referred to as a camera or image capture device, may be arranged or fixed, in particular on the exterior of or in the vicinity of the motor vehicle, where the camera device may be both permanently or fixedly arranged on the motor vehicle and designed to be releasable or removable.

[0012] For example, the camera may be positionable or disposable on the outside of the vehicle, particularly on or near an attachment to the vehicle, and therefore naturally or optionally in one or more fixed devices such as mounts or receptacles.

[0013] The camera device is in particular a camera with a special lens or special objective that generates a non-proportional image, such as a so-called wide-angle and / or fish-eye lens, which may have an aperture or image angle of at least 180°, 200°, 220°, 240°, 260°, 280° or more, meaning that the image captured by the camera device may be distorted and / or misrepresented, in particular misrepresented as at least partially curved and / or barrel-shaped and / or fish-eye shaped.

[0014] In a further step, at least one characteristic of the camera device is captured.

[0015] The characteristics may also be referred to as the context of the camera device, and as described below, characterize or define one or more parameters of the camera device, such as the type or nature of the camera or lens used, the position and / or the image section or angle that can be captured, etc. In particular, the characteristics include an indication or assignment of the type and / or severity of distortion of the recorded image and / or the orientation, tilt, displacement, twist, and / or rotation of the camera device.

[0016] The characteristics here can be captured, inter alia, by communication between the camera device and a control device, which can, inter alia, perform the steps of the methods described here and below and can also cause or trigger the capture of one or more images by the camera device.

[0017] For example, the camera may provide its own parameters, in particular the parameters of the installed lens, to the control device. In addition, suitable sensors, such as short-range sensors, in particular NFC and / or RFID sensors, may detect the position of the camera, which may be installed on or in the mount and communicate with the control device.

[0018] Alternatively or additionally, a code located in, on, or near the mount, such as a barcode or QR code, can also be read by the camera device, which allows for an indication of the location of the camera device, in which case the camera device and control device can communicate with each other both wired and wirelessly and provide characteristics of the camera device.

[0019] Likewise, characteristics may also be captured by the camera device or control device itself, for example so that images or captured image sections are analyzed and conclusions are drawn accordingly regarding the position and / or mounting location of the camera device.

[0020] In this case, the step of capturing the characteristics can be performed before, during or after capturing the image, in particular, this step is performed initially, i.e. before the first image is captured by the camera device.

[0021] Optionally, an image section can be selected from the captured image, which image section comprises only a partial region or a partial amount of the captured image, for example with a certain opening angle, in particular a certain horizontal and / or vertical opening angle out of all possible opening angles. For example, only a horizontal opening angle of 30° and a vertical opening angle of 20° are selected. This can be done, for example, by cutting and / or cropping. The image section can also be selected here based in particular on the characteristics of the camera device.

[0022] The image or image section is equalized based on the captured characteristics of the camera device.

[0023] In this case, captured images that are distorted due to camera characteristics are equalized, in particular so that previously distorted or misrepresented images, in particular strongly barrel-distorted and / or fish-eye-distorted images, as captured by a camera device, are imaged into undistorted or undistorted images, in particular gnomonic, rectilinear, orthographic and / or angularly correct images. To do this, image processing algorithms are used to edit, arrange and organize the picture elements or pixels so that an equalized image is created.

[0024] How exactly the distorted image is equalized can be determined by the control device accessing a database containing one or more instructions for equalization based on the characteristics of each camera, for example, the database may contain predetermined equalization routines or instructions for a number of camera types, lens types, and / or attachment positions, based on which the image is equalized.

[0025] In this case, the database may be located or stored both on or within the camera device and on or within the control device. Alternatively or additionally, the database may be stored on a remote computing unit from which it can be wirelessly retrieved as required.

[0026] Optionally, after the characteristics of the camera have been identified, they can be provided to a user of the vehicle, in particular the driver, in particular for verification of the captured characteristics. This can for example be provided on an output device, such as a display device, e.g. a monitor or display, that is particularly located inside the vehicle and that is visible from the driver's seat while the vehicle is being driven. Along with the characteristics of the camera, characteristics of any attachments that at least partially appear in the captured image can also be provided.

[0027] The equalized image is now checked based on the characteristics of the camera device.

[0028] Here, a check can be carried out using a suitable image evaluation algorithm, such as an image comparison algorithm, which checks whether individual or multiple elements in the image correspond to predetermined elements, as will be explained below, and in particular whether the equalization has been performed properly or correctly and that no errors have occurred in the equalization. This can be done on the basis of criteria defined in the characteristics of the camera device.

[0029] The equalized image is provided to a user of the vehicle.

[0030] The equalization image is then output to the user. In particular, the equalization image is provided to the user only if the check is positive or if the equalization is identified as correct. Alternatively, an image can also be provided if the check is negative or if the equalization is identified as erroneous or incorrect, in which case an indication of the negative check or error, in particular its extent, can optionally be provided. The output can here be provided, for example, on an output device, as described above.

[0031] The solution according to the invention makes it possible to provide a simple and fast possible way of verifying or checking an equalization process or algorithm.

[0032] According to one development, the method further comprises a step of enabling a function of the vehicle in response to the check.

[0033] If the check shows that the equalization has been performed properly or accurately, the vehicle functions can be enabled. On the other hand, if the check shows that the equalization has not been performed properly or accurately, for example because deviations or changes in the equalization image have been detected with respect to characteristics, the vehicle functions cannot be enabled or can be blocked or limited. In particular, this can also be done together with the information provided.

[0034] The function may in particular be a function of the vehicle that is performed by an element that is at least partially positioned in the field of view or image section that reproduces the image of the camera device.

[0035] In particular, the functions relate to attachments and / or equipment such as trailers, buckets, forks, troughs and / or mowing equipment, in particular elements that are positioned in the vehicle so that they cannot be seen or confirmed, or at least cannot be seen or confirmed completely, from the driver's seat.

[0036] For example, buckets, forks, troughs, or mowing devices that project far ahead of the vehicle and into intersecting roadways, particularly at narrow and / or poorly visible intersections.

[0037] Furthermore, for example, the trailer is one whose rear end the driver cannot overlook, particularly when reversing, to the extent that the driver can safely start or continue driving.

[0038] This function can then be blocked if the user or driver has used their user input to falsify the provided image, i.e. not enabled it, for example, activation of the attachment can be limited or prevented, or forward and / or reverse travel, i.e. the initiation of travel, can be prevented, or only allowed at limited speeds, for example walking speed, in particular below 30 km / h, 10 km / h, 5 km / h or 3 km / h.

[0039] This development makes it possible to make the operation of motor vehicles, in particular those with attachments, particularly safe.

[0040] According to one development, the checking comprises checking image characteristics.

[0041] For example, image features here include elements captured or represented in an image. Individual or all image features can be extracted here from an image by, for example, suitable image recognition algorithms, in particular from the field of pattern recognition.

[0042] These features can then be compared to features stored or defined in particular camera characteristics from a database, for example, one or more features in the image should have a certain shape, size, and / or position in the image, and then tested by checking for accuracy.

[0043] In particular, it is possible to check for one or more characteristic contours in the image, in particular lines and / or edges with clear lines of components of the motor vehicle, such as body parts, in particular fenders, a hood, a rear cover, pillars, in particular A-pillars, B-pillars and / or C-pillars, and / or a roof.

[0044] One or more of these characteristic contours are now checked for their correct position in the image, in particular whether lines and / or surfaces match predetermined lines and / or surfaces in the overlay image as described above, in particular the checks are designed to check one or more of these characteristic contours with a tolerance of less than 5 pixels, 3 pixels, 2 pixels, 1 pixel and / or with pixel accuracy or precision.

[0045] This development allows for a particularly simple and quick check.

[0046] According to one development, the checking is performed in the functional safety domain.

[0047] A functional safety domain here relates to hardware and / or software used in a motor vehicle and that has been verified as particularly functionally reliable, for example according to standards and / or certifications. For example, a functional safety domain is an area that is certified according to ASIL standards, in particular according to ASIL-A and / or ASIL-B.

[0048] In particular, the checks are performed exclusively in the functional safety domain, whereas the further steps of the method according to the invention, in particular the equalization, are not processed or performed in the functional safety domain, i.e. are processed or performed by conventional hardware or software.

[0049] Moreover, in particular, only image feature checking is performed in the functional safety domain: in particular, feature detection or extraction from the captured image is not performed in the functional safety domain.

[0050] This development allows a particularly safe implementation of the method.

[0051] According to one development, the checking is carried out by means of artificial intelligence.

[0052] Here, the artificial intelligence is a computer-aided function approximation that performs or at least supports the checks. In particular, this may be an artificial neural network (ANN).

[0053] In particular, artificial intelligence is specially trained for the check and feature comparison. This can be performed in particular by so-called overlay images, in which individual elements or features are highlighted, in particular in color, using actual real images corresponding to the image to be checked and / or using CAD data. Here, these overlay images can be placed on the equalized image by the camera device, or alternatively, they can first be pre-distorted and placed on the distorted image of the camera device, and then equalized together. Alternatively or additionally, the artificial intelligence can also be performed using real images, in particular the actual placement of the camera device and / or attachment on the vehicle.

[0054] This development allows for a particularly reliable check.

[0055] According to one development, the camera properties comprise a model of the camera device.

[0056] The model of the camera device comprises, among other things, the characteristics, type, and / or properties of one or more components of the camera device, especially the sensor and / or the objective or lens, such as the manufacturer, type, production model, and / or age.

[0057] In particular, the model of the camera device comprises the focal length and / or opening angle of the objective or lens.

[0058] The model of the camera device may also comprise data or information relating to the housing and / or fixing device of the camera device, in particular data derived from simulation and / or manufacturing, for example so-called CAD data, i.e. computer-aided design, and which substantially correspond to real values, in particular dimensions, of the camera device and / or fixing device.

[0059] In particular, the model also comprises the sensor resolution, format, aspect ratio and / or possible image sections.

[0060] This development makes it possible to carry out equalization and checking particularly accurately.

[0061] According to one development, the camera properties comprise a captured images section.

[0062] In particular, the camera characteristics comprise the size, aspect ratio, number of pixels and / or resolution and / or color space of the captured image. For example, the camera characteristics include information about the image section used to equalize and check the image.

[0063] This development makes it possible to carry out equalization and checking particularly accurately.

[0064] According to one development, the camera properties comprise the elements that are captured in the image.

[0065] In this case, the element may in particular be an element of a motor vehicle and / or an attachment of a motor vehicle. For example, the element may be a body part such as one or more fenders, one or more pillars, one or more wheels, and / or one or more trim parts. Similarly, the element may be, for example, a hydraulic cylinder or another element or component of an attachment.

[0066] This development makes it possible to carry out equalization and checking particularly accurately.

[0067] To perform the described steps, a processor circuit may be provided having programming or software with program instructions that, when executed, cause the processor circuit to perform the method embodiments. The processor circuit may include at least one microprocessor and / or microcontroller for this purpose. The program instructions may be stored in a data memory of the processor circuit.

[0068] According to a further aspect, an apparatus is specified that is designed to carry out one of the above-described embodiments of the method according to the invention.

[0069] Here, the device may comprise and / or have access to one or more camera devices, and may also comprise a control device and processor circuitry as already described above.

[0070] According to a further aspect, a vehicle having such a device is specified.

[0071] The invention also comprises developments of the device according to the invention and of the motor vehicle according to the invention, which have the features already explained in connection with the development of the method according to the invention, so that the corresponding developments of the device according to the invention and of the motor vehicle according to the invention will not be described again here.

[0072] For use cases or application situations that may arise in the method and that are not explicitly described herein, error messages and / or requests for user feedback may be output in accordance with the method, and / or default settings and / or predetermined initial states may be established.

[0073] The present invention also includes combinations of features of the described embodiments.

[0074] Exemplary embodiments of the present invention are described below. [Brief explanation of the drawings]

[0075] [Figure 1] 1 shows a schematic plan view of one embodiment of a vehicle and an apparatus for capturing images of the exterior of the vehicle. [Figure 2] A detailed view from FIG. [Figure 3] 1 shows a simplified flow diagram of one embodiment of an apparatus and method for capturing images of the exterior of a vehicle. [Figure 4] 1 shows a flow diagram of an alternative embodiment of an apparatus and method for capturing images of the exterior of a motor vehicle. DETAILED DESCRIPTION OF THE INVENTION

[0076] The exemplary embodiments described below are preferred embodiments of the present invention. In the exemplary embodiments, each of the described components of the embodiments should be considered independently of one another, each independently develops the present invention, and therefore represents individual features of the present invention that can be considered part of the present invention individually or in combinations other than those shown. Furthermore, the described embodiments can also be supplemented by further features of the present invention already described.

[0077] In the figures, elements having the same function are each provided with the same reference numerals.

[0078] FIG. 1 shows a schematic plan view of one embodiment of a vehicle 10 and an apparatus for capturing images of the exterior of the vehicle.

[0079] The vehicle 10 moves along a direction of travel indicated by an arrow and is provided with an attachment 20 or attachment device, shown here as an example as a mowing device, which is here as an example as a front attachment or front attachment device.

[0080] The vehicle 10 also includes a camera device, not shown in this Figure 1 for clarity, which captures a capture area 30 that is generally spherical and has both horizontal and vertical capture angles of 220°.

[0081] In this illustrated embodiment, the sections designated by reference numeral "31" are of particular relevance here as they capture significant portions of the attachment 20, where these sections are also spherically distorted due to the capture area or angle of the camera device.

[0082] FIG. 2 shows a detailed view from FIG.

[0083] In this FIG. 2, camera device 40 is shown, here as an example, with a fisheye lens capturing the right part of the attachment in section 31 .

[0084] FIG. 3 shows a simplified flow diagram of one embodiment of an apparatus 1000 and method 100 for capturing images of the exterior of a vehicle.

[0085] Here, the device 1000 performs the method or steps thereof described below.

[0086] The method 100 according to the present invention begins at step 105 where one or more characteristics or contexts of the camera device 40 are identified.

[0087] In a further step 110, the camera device 40 is configured based on the identified characteristics by accessing a memory or database 115 in which information about the camera device 40 is stored.

[0088] A video comprising one or more images or multiple sequential images of the exterior of the vehicle is then captured using the camera device 40 in step 120, where the images are distorted due to the characteristics of the camera device.

[0089] The captured image is then equalized in step 125 using an appropriate image equalization algorithm and based on the camera characteristics, resulting in an equalized image in step 130 .

[0090] Step 135 detects features in the equalized image based on features 121 provided from a database based on camera information.

[0091] In step 145, the identified features are then checked.

[0092] Here, in this embodiment, both feature recognition and checking are performed in the functional safety domain 160 .

[0093] If the check is then positive, i.e., the equalization is determined to be correct, the equalized image is displayed on a display device to the vehicle user in step 140. Optionally, in this step 140, vehicle functions can also be enabled in response to the check as well.

[0094] FIG. 4 shows a flow diagram of an alternative embodiment of an apparatus 1000 and method 100 for capturing images of the exterior of a vehicle.

[0095] This embodiment differs from the embodiment shown in FIG. 3 in particular in that only a partial range of functions from FIG. 3 are executed in the functional safety domain.

[0096] The method 100 according to the present invention begins at step 105 where one or more characteristics or contexts of the camera device 40 are identified.

[0097] In a further step 110, the camera device 40 is configured based on the identified characteristics by accessing a memory or database 115 in which information about the camera device 40 is stored.

[0098] A video comprising one image or multiple consecutive images of the exterior of the vehicle is then captured using the camera device 40 in step 120, where the image is distorted due to the characteristics of the camera device.

[0099] The captured image is then equalized in step 125 using an appropriate image equalization algorithm and based on the camera characteristics, resulting in an equalized image in step 130 .

[0100] Step 135 detects features in the equalized image based on features 121 provided from a database based on camera information.

[0101] These detected features are then extracted or separated in step 145 .

[0102] In step 150, these separated features are then compared and checked based on the extracted features 121.

[0103] Here, only the checking step 150 is performed in the functional safety domain 160 .

[0104] If the check is then positive, i.e., the equalization is determined to be correct, the equalized image is displayed on a display device to the vehicle user in step 140. Optionally, in this step 140, vehicle functions can also be enabled in response to the check as well.

[0105] Overall, the examples show how a method can be provided for cost-effective implementation of functional safety operation of camera devices on motor vehicles. [Explanation of symbols]

[0106] 10. Automobiles 20 Attachments 30 Capture Area 31 Section 40 Camera Devices 100 ways 105 Method Steps 110 Method Steps 115 databases 120 method steps 121 Method Steps 125 method steps 130 method steps 135 method steps 140 method steps 145 method steps 150 method steps 155 method steps 160 Functional Safety Area 1000 devices

Claims

1. A method (100) for capturing an image of the exterior of a motor vehicle (10), comprising: - capturing an image of the exterior of the motor vehicle (10) using a camera device (40), said captured image being distorted due to the nature of said camera device (40); - capturing the characteristics of said camera device (40); - equalizing the image based on the characteristics of the camera device (40); - checking the equalized image based on the characteristics of the camera device; - Providing said equalized image to a user of said vehicle (10). A method comprising:

2. - further comprising enabling a function of the vehicle in response to said check.

2. The method of claim 1 (100).

3. said checking including checking image characteristics; 3. The method (100) of claim 1 or 2.

4. The checking is performed in a functional safety domain (160). The method (100) according to any one of claims 1 to 3.

5. The checking is performed using artificial intelligence. The method (100) according to any one of claims 1 to 4.

6. The camera characteristics comprise a model of the camera device (40). The method (100) according to any one of claims 1 to 5.

7. The camera characteristics include a captured image section. The method (100) according to any one of the preceding claims.

8. The camera characteristics comprise elements captured in the image. The method (100) according to any one of the preceding claims.

9. A device (1000) for capturing an image of the exterior of a motor vehicle (10), the device (1000) being designed to carry out the method (100) according to any one of the preceding claims.

10. A motor vehicle (10) comprising a device (1000) according to claim 9.

Images