Method for capturing a digital image of the surroundings of a motor vehicle and motor vehicle with an image capture device

Abstract

Method for capturing a digital image (27) of the environment of a motor vehicle (22), comprising the following steps a) Capturing and evaluating a first gesture (21) performed freely in a motor vehicle interior (20) by an operator (40), wherein the first gesture (21) defines a first image section (24), b) Detection of at least one position attributable to the first gesture (21) and at least one position attributable to the operator (40) in the motor vehicle (22), c) Determining a field of view (α) depending on the positions captured in step b), and d) Controlling an image acquisition device (35) depending on the specified image angle (α) for capturing at least one digital image (27), characterized by the steps g) Detecting and evaluating the gaze direction (31) of the operator (40) in addition to the first gesture (21), whereby a focus field (33) for the controlled image acquisition device (35) is determined depending on the gaze direction (31), and h) Determining the dimensions of the focus field (33) depending on the fixation duration of the gaze direction (31) of the operator (40).

Description

[0001] The invention relates to a method for capturing a digital image of the environment of a motor vehicle, comprising capturing and evaluating a first gesture freely performed by an operator within the vehicle interior, wherein the first gesture defines a first image section. Furthermore, the invention also relates to a motor vehicle with an image capture device for capturing a digital image of the environment of the motor vehicle, comprising a gesture recognition device designed to recognize and evaluate a first gesture performed by an operator within the vehicle interior, wherein the first gesture defines a first image section.

[0002] Gesture control enables the contactless, i.e., non-tactile, operation of various control devices using freely executed gestures by an operator. These gestures can include, for example, signs, i.e., movements with the fingers, hand, and / or arm of an operator, or they can be tracked using eye tracking.

[0003] As shown in DE 20 2012 005 255 U1, gesture monitoring enables the contactless operation of various control devices, such as vehicles or household appliances. This utility model specification describes a control device, particularly for a vehicle or household appliance. The control device comprises a gesture monitoring unit designed to detect non-tactile gestures from a user. Furthermore, an actuation unit is provided, designed to actuate an associated device. This actuation unit can be either in a locked state, in which it is not activated, or in an operating state, in which the control device can activate the actuation unit upon detection of a non-tactile actuation gesture to actuate the associated device.The operating device is put into operating mode when the gesture monitoring unit detects a non-tactile activation gesture.

[0004] Furthermore, US Publication 2017 / 0 174 227 A1 describes a dynamic sensor area in advanced driver assistance systems, US Publication 2017 / 0 220 106 A1 describes an image display device, and US Publication 2017 / 0 313 248 A1 describes a device for displaying a view of a rear-seat passenger on a display device in a vehicle.

[0005] It is also possible to take digital pictures using gesture control. DJI provides the DJI Spark multicopter for this purpose. This is a small drone, or quadcopter, that can be used as a "flying selfie stick," meaning a camera that allows the operator to take a digital picture of themselves. The drone has a camera and a gesture control unit. When the drone is in flight, the operator can frame the drone's camera with four fingers, whereupon the drone starts taking a selfie. This description, however, only describes how a selfie, i.e., a digital picture of the operator, can be captured with a quadcopter using a non-tactile gesture.

[0006] The described state of the art has the disadvantage that it is not possible to capture a digital image from inside a vehicle.

[0007] The invention is based on the objective of enabling the capture of a digital image from a motor vehicle, in particular a moving motor vehicle, in the simplest possible way.

[0008] The problem is solved by the subject matter of the independent patent claims. Advantageous embodiments of the invention are disclosed by the dependent patent claims, the following description, and the figures.

[0009] The invention is based on the understanding that capturing digital images, i.e., taking photographs from inside a vehicle, presents several challenges. Firstly, current technology requires an additional image capture device, such as a digital camera or a smartphone (mobile device), to record digital images, i.e., photos or videos, of the vehicle's surroundings from inside the vehicle. Furthermore, unwanted reflections from the vehicle's windshield are common in these recordings. Lowering the windshield to avoid these reflections is often impossible, especially at high speeds. Additionally, spontaneous shots with a separate camera or smartphone while driving are limited. For one thing, the camera or smartphone is usually not readily accessible.On the other hand, the camera or smartphone must first be activated before taking a picture. Both searching for and activating the respective image capture device takes time, making spontaneous photography impossible.

[0010] The invention described below now enables the direct recording of photos or videos from inside a motor vehicle without the use of additional image capture devices, such as digital cameras or smartphones.

[0011] The invention provides a method for capturing a digital image of the environment of a motor vehicle. The method comprises, in step a), capturing and evaluating a first gesture freely performed by an operator within the vehicle's interior, wherein the first gesture defines a first image section. Furthermore, in step b), the method comprises capturing at least one position attributable to the first gesture and at least one position attributable to the operator within the vehicle. In step c), an image angle is then defined based on the positions captured in step b). Finally, in step d), an image capture device is controlled based on the defined image angle to capture at least one digital image.

[0012] In other words, a method is presented here for taking at least one photograph, particularly a digital photograph, of a landscape surrounding a vehicle. Preferably, a sequence of digital photographs can also be taken, forming a video sequence. The method comprises several steps. In step a), the movement of at least part of the body of an occupant of the vehicle can be recorded and analyzed. Here and in the following, "occupant" refers to the operator. The movement or gesture can be, in particular, a finger, hand, or arm movement. The gesture can be performed without contact, i.e., non-tactilely, in the interior of the vehicle, preferably in the passenger compartment. In particular, the occupant can define a desired initial image section with the movement.Preferably, the first image section can represent a frame by which a part of the environment of the motor vehicle visible to the occupant can be optically delimited.

[0013] In step b), at least one position in the vehicle that correlates with the first gesture can be recorded. This means that at least one location or point in the vehicle interior can be determined that corresponds to the first image section. Furthermore, at least one position in the vehicle that correlates with the occupant can also be recorded. This means that at least one location or point in the vehicle interior can be determined that corresponds to the occupant. For clarity, the position correlated with the first gesture, or the position attributable to the first gesture, will be referred to as the gesture position. Similarly, the position correlated with the occupant, or the position attributable to the operator, will be referred to as the occupant position.

[0014] In step c), a first angle can be determined based on the at least one gesture position and the at least one occupant position. Preferably, the occupant position can be the vertex of the angle. If preferably two gesture positions are present, two lines can be drawn from the vertex to the respective gesture positions. These two lines can then form the sides of the angle and enclose the first angle. If, for example, the first gesture represents a framing gesture, the first angle can be determined, in particular, as a diagonal angle through opposite corner points of the first image section.

[0015] In step d), the first angle can be transmitted to a control unit. Depending on the first angle, the control unit can then adjust an image acquisition device such that the image acquisition device can capture a digital image of the vehicle's surroundings. Preferably, the image acquisition device can be located on the outside of the vehicle. In particular, the image acquisition device can also have a defined focal length. Furthermore, the control unit can preferably adjust the camera viewing angle of the image acquisition device such that the camera viewing angle coincides with the first angle. The camera viewing angle is understood to be the angle that determines the dimensions of the camera image area to be captured.In other words, the camera's viewing angle can be used to determine the "zoom", that is, the enlargement / reduction of an object in the vicinity of the vehicle that is to be depicted in the digital image.

[0016] This method is particularly well-suited for capturing digital images or videos of the vehicle's surroundings in the direction of travel. That is, the occupant preferably performs the first gesture in the direction of the vehicle's windshield.

[0017] The invention offers the advantage that a digital image of an external environment can be captured from inside the vehicle by an operator, such as an occupant like the driver or passenger, without requiring an additional image capture device, such as an external camera or a smartphone. This effectively avoids reflections on the photographs caused by glare from the vehicle's windshield. Furthermore, spontaneous captures, particularly while driving, are possible, as the occupant does not need to first locate and activate an external image capture device.

[0018] The first gesture can be captured and evaluated by a gesture recognition system. This system can comprise various gesture capture and evaluation units. For example, the gesture capture unit can be implemented using one or more cameras and / or one or more spatial sensors, allowing monitoring of the entire vehicle interior. Another gesture recognition option involves the operator wearing sensor-equipped gloves or a sensor-equipped body suit. The cameras and / or sensors can operate in the visible light spectrum or the infrared range. The sensors can be, for example, ultrasonic sensors, radio wave sensors, or sensors that respond to changes in an electric field.The evaluation unit of the gesture recognition device can be implemented as a processing unit. Here, the operator's movement pattern, recorded by the gesture capture unit, can be analyzed. The resulting movement data can be used to control a desired operating device, for example, an image capture device mounted externally on the vehicle.

[0019] The gesture recognition unit can detect not only finger movements but also the movement of the hand and forearm, as well as their interaction with each other.

[0020] As a first gesture, for example, a frame can be formed with four fingers, especially with the thumbs and index fingers. An alternative is to draw a frame in the air with the outstretched index finger of one hand. Another possibility is to mimic the shutter release of a camera with one finger.

[0021] To determine at least one gesture position and at least one occupant position, the vehicle can be equipped with an additional camera array and processing unit. This camera array could record and monitor the vehicle interior, including one or more occupants. When an occupant performs the first gesture, the processing unit can analyze the camera footage to determine the gesture position and the occupant's position. The camera array's recording area could, for example, be limited to specific areas of the vehicle interior. Alternatively, the gesture recognition system itself could be designed to determine these positions.

[0022] If the operator's gesture is, for example, a frame gesture, the corner points of the drawn frame, particularly the diagonally opposite corner points, can be determined by the operator's fingertip position to define the gesture position. The occupant position can be determined in a similar way. Preferably, the point on the bridge of the nose between the operator's eyes is defined as the occupant position. Alternatively, the center of the operator's head can also be defined as the occupant position.

[0023] The viewing angle from the gesture position and the occupant position can then be determined, for example, by a computer. The calculated viewing angle can then be transferred by the responsible computer to a control unit, which controls an image acquisition device. The image acquisition device can be, for example, a camera, in particular a camera with digital or analog zoom. The camera can be a digital camera, a compact camera, a camera with a fixed or variable focal length, or a camera with digital zoom or a zoom lens or varifocal lens. The camera can also be a video camera. Preferably, the camera is located on the outside of the vehicle. That is, the camera is preferably arranged on the outside of the vehicle.

[0024] Preferably, the image capture device is in automatic mode. This means that aperture, shutter speed, white balance, ISO value, and, if applicable, the flash, etc., are set by the camera itself. These settings are adjusted depending on various environmental factors, such as lighting conditions and whether the image is to be captured while driving or stationary. Focusing, i.e., adjusting the focus on an object in the vicinity of the vehicle that is within the frame, can also be handled by the camera itself. Preferably, only the camera's field of view can be adjusted by the control unit, depending on the desired field of view. Alternatively, the digital image can also be adjusted to the desired frame only after the image capture device has taken the picture.

[0025] If the image capture device is, for example, a wide-angle camera with a 75-degree field of view, the camera can first capture an image across the entire field of view. Subsequently, at least a portion of the captured image can be adjusted in a processing unit inside the camera or in the vehicle to the image area specified by the operator, as is known, for example, from cameras with digital zoom.

[0026] If the image capture device is, for example, a camera with a zoom lens, the camera's zoom lens can be adjusted by the control device so that the camera's viewing angle corresponds to the specified viewing angle.

[0027] Furthermore, it is planned that in step g) in addition to the first gesture, the operator's gaze direction will also be recorded and evaluated, whereby a focus field for the controlled image acquisition device will be determined depending on the gaze direction.

[0028] This means that, in addition to capturing the initial gesture, the operator's eye movements can also be recorded and analyzed. Specifically, at least one fixation on an object within the field of view defined by the initial gesture can be detected and evaluated. Based on this single fixation, at least one focus area can then be determined for the image capture device. This focus area can represent a defined region within the camera's field of view, upon which the device can focus. If an occupant is specifically looking at several objects within the initial field of view, meaning multiple fixations are detected, the image capture device can be assigned more than one focus area, for example, four focus areas.

[0029] This offers the advantage that the occupant can determine which object in the vehicle's surroundings should be focused on, i.e., which object the image capture device should focus on. The desired object is then also displayed sharply in the digital image.

[0030] The recording and analysis of eye movements, particularly gaze direction, can preferably be performed using an eye tracker. This device records the eye movements of a vehicle occupant, including fixations, saccades, and regressions. The eye tracker can be a mobile or external device. A mobile device can, for example, be a head-mounted device attached to the operator's head, such as glasses. An external device, also known as a remote eye tracker, can be installed inside the vehicle without touching the occupant.Regardless of whether the eye tracker is designed as a mobile or external device, the eye tracker can include the following components: at least one light source, which emits in particular infrared light and preferably creates a light spot on the cornea of ​​the occupant, at least one camera that records the reflection patterns on the cornea, and at least one field-of-view camera, for example the image capture device, which records the area over which the operator is looking.

[0031] Furthermore, it is planned that in step h) the dimensions of the focus field will be determined depending on the fixation duration of the operator's gaze direction.

[0032] Put another way, the dimension of at least one focus area can be determined depending on how long the operator fixes an object within the first image area defined by the first gesture.

[0033] This offers the advantage that targeted focusing or sharpening on a specific object in the vicinity of the vehicle is possible, for example by reducing the focus area the longer the occupant's gaze rests on this object or on a point of this object.

[0034] To determine the fixation duration, a gaze plot, i.e. a graphic representation of the sequence and duration of fixations, or a heat map, i.e. a graphic representation in which the duration of fixations is highlighted in color, can be generated from the operator's eye movements.

[0035] The invention also includes further embodiments that offer additional advantages.

[0036] One embodiment provides that, in step d), if there is a plurality of image capture devices of the motor vehicle, in step e) the image capture device whose orientation and detectable image angle at least partially correspond to one of the directions attributable to the first gesture and the defined image angle is controlled.

[0037] In other words, if the vehicle has more than one image capture device, in step d) the image capture device can be activated that is oriented in a direction that at least partially coincides with a direction correlated with the first gesture. That is, at least one point in the vehicle interior can be determined to which a direction in which the first gesture is executed can be traced. Simultaneously, the image capture device can be activated that can at least partially capture an angle of view that corresponds at least partially to the angle of view determined by the gesture and the occupant's position.

[0038] This offers the advantage that the occupant can determine which environmental situation should be captured by performing the first gesture in a direction. That is, the occupant can decide whether a digital image should be captured of a situation located behind, to the side, or in front of the vehicle in the direction of travel.

[0039] The direction associated with the first gesture, hereinafter also referred to as the gesture direction, can be determined, for example, by calculating the center point of the image area defined by the first gesture. If, for instance, the gesture is a frame gesture shaped like a rectangle, diagonals can be drawn through the diagonally opposite corners of the rectangle. The intersection of these two diagonals can then represent the center point of the first image area. A perpendicular line through this center point onto the two-dimensional surface defined by the rectangle can then correspond to the gesture direction. Alternatively, a vector can be defined from the previously determined occupant position through the center point. The orientation of this vector can then correspond to the gesture direction.

[0040] Another embodiment provides that in step f) the image acquisition device in step d) is controlled in such a way that at least one geometric point attributable to the digital image to be acquired is aligned with a geometric point attributable to the first gesture.

[0041] In other words, the image capture device used to capture the digital image can be aligned such that a geometric point of the digital image to be captured by the image capture device coincides with a geometric point correlated with the first gesture. That is, before the digital image is captured, the image capture device can be aligned so that a point attributable to the camera's field of view coincides with a point in the vehicle interior attributable to the first gesture.

[0042] This offers the advantage that the camera can be precisely aimed at a desired situation in the vicinity of the vehicle, so that an unwanted part of the vehicle's surroundings is not shown in the captured digital image.

[0043] The center point of the first image section can serve as a geometric point. If the gesture is, for example, a framing gesture, one of the corner points of the first image section can also be understood as a geometric point. Preferably, the image capture device, i.e., the camera, is mounted to rotate or swivel in three dimensions. Accordingly, the camera can be swiveled so that the center point of the image section that the camera can capture coincides with the center point of the image section defined by the first gesture.

[0044] Another embodiment provides in step i) the capture of at least one digital image and in step j) the sending of the captured digital image to a vehicle-internal display device.

[0045] In other words, a digital image can be captured by the image capture device and then transmitted wirelessly or via cable from the capture device to an in-vehicle display device via an interface.

[0046] This offers the advantage that, after setting all relevant parameters of the image capture device, a digital image is taken and this image is made available to the vehicle for display.

[0047] The captured image can be transmitted via a wired connection, for example, using a data bus. Alternatively, the digital image can be transmitted wirelessly to the display device, for example, using WLAN, Bluetooth, UMTS, DSM, or similar transmission methods. The display device can be implemented as a digital screen, such as a windshield, side window, sunroof, or rear window. It is also conceivable that the display device could be a screen located in the center console of the vehicle.

[0048] Another embodiment provides that in step k) the method also includes displaying the captured digital image on the vehicle's internal display device and in step I) the capture and evaluation of a free second gesture performed by the operator inside the vehicle, wherein the second gesture results in the digital image being edited on the display device.

[0049] This means that after the digital image is displayed on the screen, a further gesture from one of the vehicle's occupants can be detected, and this second gesture can change the display of the digital image. Specifically, the displayed digital image can be enlarged, reduced in size, rotated, cropped, its color changed, moved, sent, or deleted.

[0050] This offers the advantage that the captured digital image can be subsequently reviewed and edited by the occupant, for example, if only part of the vehicle's surroundings depicted in the digital image is of interest to the operator.

[0051] Preferably, the second gesture can differ from the first. For example, to move the digital image on the display device, a swipe in one direction with an extended index finger or with the flat of the hand could be used. To enlarge or reduce the digital image, a space could be created between the thumb and index finger, and this space then enlarged or reduced. To rotate, a clockwise or counterclockwise twisting motion of the hand and / or forearm could be used. To send, a hand gesture resembling a wave could be used. To delete, a hand gesture similar to crumpling paper or a fist with the thumb extended downwards could be used.

[0052] Another embodiment provides in step m) the storage of the digital image in a vehicle-internal and / or vehicle-external storage device.

[0053] In other words, the digital image captured by the image capture device can be sent to and stored in a vehicle-internal and / or vehicle-external storage device.

[0054] This offers the advantage that the digital image is also available at a later time and / or on another device, such as a smartphone.

[0055] In particular, the digital image can be transmitted wirelessly or via cable from the image capture device or the vehicle's internal display to the storage device. The storage device can, for example, be implemented as a data storage device, especially a data carrier, within the vehicle. The aforementioned external storage device can, for example, be a USB storage medium connected to a USB port in the vehicle. It is also conceivable that the external storage device could be a smartphone connected to the vehicle, for example, via Bluetooth, Wi-Fi, or a USB cable.

[0056] Another embodiment provides in step n) that in step i) a large number of digital images are captured, wherein the large number of digital images represents a video sequence.

[0057] In other words, a series of digital images can be captured which, taken together, constitute a video film.

[0058] This offers the advantage that not only snapshots of a situation in the vicinity of the vehicle can be captured, but also situations that change over time, such as another vehicle passing by. Preferably, the image capture device is designed as a camera with video recording function or as a video camera.

[0059] The invention also provides a motor vehicle with an image capture device for capturing a digital image of the vehicle's surroundings. The motor vehicle comprises a gesture recognition device designed to recognize and evaluate a first gesture freely performed by an operator within the vehicle's interior, wherein the first gesture defines a first image section; a capture device designed to capture at least one position attributable to the first gesture and at least one position attributable to the operator within the vehicle; a computing device designed to define an image angle depending on the captured positions; and a control device designed to control the image capture device, depending on the defined image angle, to capture at least one digital image.

[0060] In other words, the vehicle may be equipped with a gesture recognition device that can be configured to capture and analyze a contactless movement pattern of a vehicle occupant. This initial movement pattern allows the occupant to define a section of the vehicle's visible environment, which is later to be captured as a digital image. Furthermore, a detection device may be provided that can identify at least one point within the vehicle interior that corresponds to the initial section defined by the first movement pattern. The detection device may also be configured to identify at least one point within the vehicle interior that indicates the operator's position within the vehicle. For clarity, the following section describes the position correlated with the first gesture.The position attributable to the first gesture is also called the gesture position. Similarly, the position correlated with the occupant or the position attributable to the operator is referred to as the occupant position. The vehicle may also be equipped with a computing unit that can receive the gesture position and the occupant position from the detection device and use this information to generate a first angle that can define a camera viewing angle to be captured by the image capture device. Finally, a control unit may also be provided that can receive the first angle from the computing unit and transmit it to an image capture device, enabling the image capture device to capture a digital image based on the first angle.

[0061] This offers the advantage of making it easy to capture a digital image from inside a motor vehicle.

[0062] Preferably, the image acquisition device is located on the outside of the vehicle, i.e., on an exterior surface, and not inside the vehicle. The image acquisition device can be mounted, for example, on the roof, the underbody, or on a side mirror of the vehicle. The interaction and function of the aforementioned vehicle components can be found in the input data described for the method.

[0063] The invention also includes further developments of the motor vehicle according to the invention, which have features already described in connection with the further developments of the method according to the invention. For this reason, the corresponding further developments of the motor vehicle according to the invention are not described again here.

[0064] The invention also includes combinations of the features of the described embodiments.

[0065] The following are exemplary embodiments of the invention described. This is illustrated by: Fig. 1 A flowchart of the process steps of an embodiment of the invention for capturing a digital image of the environment of a motor vehicle; Fig. 2 a schematic representation of the determination of a first image section depending on a non-contact first gesture by an operator; Fig. 3 a schematic representation of determining a viewing angle depending on a non-contact first gesture by an operator and a direction in which the first gesture is performed; and Fig. 4 a schematic representation of setting a focus field depending on the viewing direction of an operator;

[0066] The exemplary embodiments described below are preferred embodiments of the invention. In these exemplary embodiments, the described components each represent individual features of the invention, which can be considered independently of one another and each further develops the invention independently. Therefore, the disclosure is intended to include combinations of features of the embodiments other than those shown. Furthermore, the described embodiments can also be supplemented by further features of the invention already described. In the figures, identical reference numerals denote functionally equivalent elements.

[0067] The Fig. Figure 1 shows a flowchart of the individual process steps of an embodiment of the invention for capturing a digital image of the surroundings of a motor vehicle. After starting the process in step 100, gesture recognition and gaze direction recognition are preferably activated simultaneously. In gesture recognition, a non-contact gesture performed by an occupant of the motor vehicle is captured and evaluated in step 102. Here and in the following, "occupant" is used synonymously with "operator". In step 104, a first image section is determined, which was defined by the non-tactile gesture. Subsequently, as in the Fig. Figure 1 shows that, simultaneously in steps 106 and 107, a gesture position and an occupant position are recorded. The gesture position corresponds to a point in space that can be derived from the movement pattern of the first gesture. Similarly, the occupant position describes a point in space that can be derived from the occupant's position and orientation. From the recorded gesture position and occupant position in steps 106 and 107, a viewing angle is calculated in step 108 of the gesture recognition process.

[0068] Advantageously, as described above, steps 110 and 112 of gaze direction detection are executed simultaneously with steps 102 to 108 of gesture recognition. In gaze direction detection, step 110 captures and evaluates the gaze direction of a vehicle occupant. Depending on this captured gaze direction, at least one focus area for the image capture device is then defined in step 112. The dimensions of the focus area can be adjusted, for example, based on the duration of fixation on a point within the initial image section. Thus, the longer the occupant views a specific point within the initial image section, the larger or smaller the dimensions of the focus area can be. The number of focus areas can also depend on how many points within the initial image section the occupant fixates on.

[0069] After the viewing angle has been determined in step 108 and at least one focus area has been determined in step 112, at least one of the vehicle's image acquisition devices can be activated in step 114. Preferably, the image acquisition device that can capture at least part of the determined viewing angle is activated. In the next step 116, a digital image is captured by the image acquisition device. This captured image can then be sent to a display device in step 118. Finally, to allow the occupant to view the captured digital image, it can be displayed on a display device in step 120 before the process is completed in step 122.

[0070] Fig. Figure 2 shows how, depending on a defined movement pattern of the occupant, an initial image section 24 is determined, which is to be captured by an image acquisition device 35, for example a camera. In the Fig. Figure 2 schematically shows a view from inside a motor vehicle 20 through a window 25 onto the surroundings of the motor vehicle 22. The window 25 also serves as a display device. In this embodiment, the window 25 is designed as a windshield. Alternatively, the window 25 could also be designed as a side window, rear window, or roof window.

[0071] The motor vehicle may be in motion, as shown here, for example in a lane of a country road. On the opposite lane of the country road, another motor vehicle may be approaching, which is perceived by the occupant as an object in the surroundings 23.

[0072] The in Fig. The following situation can serve as an example for the embodiment shown in Figure 2. An occupant, such as a driver or passenger, is sitting in the motor vehicle and is therefore located in the vehicle interior 20. Through the window 25, the occupant sees an object in the vicinity of the motor vehicle 23, namely an oncoming motor vehicle. The occupant wants to photograph this oncoming motor vehicle and therefore performs a first gesture 21 in the vehicle interior 20. The first gesture 21 can be, as shown in the Fig. Figure 2 illustrates a framing gesture. The occupant, with their index finger extended and without touching the window 25, draws a rectangle around the object in the surroundings 23 in the air. By drawing this rectangle in the air without contact, the occupant defines a first image section 24. The portion of the vehicle's surroundings 22 that lies within this first image section 24 is then to be captured as a digital image by a camera. The camera is preferably located on the outside of the vehicle, for example, on the roof of the vehicle.

[0073] To recognize and evaluate the first gesture 21 of the occupant, a gesture recognition device (in the vehicle interior 20) can be installed. Fig. (2 not shown) must be set up. If the first gesture 21 was correctly recognized and evaluated by the gesture recognition device, the occupant position and at least one gesture position, in particular the positions of two diagonally opposite corner points of the rectangle defined by the first image section 24, can be determined via a detection device, which is also arranged in the vehicle interior 20. From the three detected positions in the vehicle interior 20, a triangle can be formed starting from the occupant position, which defines a field of view for the camera. This field of view can then be transmitted to a control unit, which controls the camera on the outside of the vehicle in such a way that the defined field of view corresponds to the camera's field of view, i.e., the angle from which the camera photographs the area around the vehicle 22.

[0074] After the camera has captured the digital image 27 of the object in the vicinity of the motor vehicle 23, the digital image 27 can be sent to a display device, in particular the screen 25, and displayed there. Preferably, the screen 25 is designed as a digital windshield. In addition to the digital image 27, a function module 26 can also be displayed on the screen 25. Thus, if the occupant performs a second gesture (in Fig. (2 not shown), for example, tapping the function module 26 with an extended index finger without touching the screen, allows the digital image 27 to be sent to a mobile device (smartphone) of the occupant. The smartphone can serve as an external storage device for the vehicle.

[0075] After the digital image 27 has been sent to the occupant's smartphone, the digital image 27 is preferably no longer automatically displayed on the screen 25. Alternatively, it would also be conceivable for the occupant to perform a further gesture to remove the digital image 27 from the display device.

[0076] Fig. Figure 3 shows how a viewing angle α and a direction in which a non-tactile first gesture 21 is performed by an operator 40 can be determined. For this purpose, the Fig. Figure 3 shows an operator 40 from a bird's-eye view. Specifically, the operator 40 is located inside a motor vehicle. The operator 40 can then perform a first gesture 21 inside the vehicle interior 20. In the illustrated embodiment, the first gesture 21 corresponds to a framing gesture, and the operator 40 draws a rectangle in the air inside the vehicle interior 20 with the outstretched index finger of one hand, without making contact. The drawn rectangle defines a first image section 24. The first image section 24 is characterized by its four corner points, namely the first corner point A, the second corner point B, the third corner point C, and the fourth corner point D.The center point M of the first image section 24 can be determined via the intersection of the diagonals, namely the first diagonal AB, which runs from the first corner point A to the second corner point B, and the second diagonal CD, which runs from the third corner point C to the fourth corner point D.

[0077] To determine the viewing angle α, at least one position attributable to the operator 40 and at least one position attributable to the first gesture 21 within the vehicle interior can first be determined using a detection device. The detection device can, for example, comprise one or more cameras or one or more sensors arranged within the vehicle interior 20, thus monitoring the vehicle interior 20 at least temporarily and at least partially. In addition to the cameras and / or sensors, the detection device can also include a processing unit designed to evaluate the images of the vehicle interior captured, for example, by the cameras, in order to determine at least one position attributable to gesture 21 and at least one position attributable to the operator 40 within the vehicle interior 20.

[0078] In this embodiment in Fig. 3. Two points, namely the first corner point A and the second corner point B, are defined as positions assignable to the first gesture. The intersection point S is defined as the position assignable to the operator. The intersection point S is a point on the bridge of the nose between the operator's eyes 40.

[0079] In a different one than the one in Fig. In the embodiment shown in Figure 3, the third corner point C and the fourth corner point D could also be identified as the position attributable to the first gesture. It is also conceivable that the center point M of the first image section, depicted as a rectangle, could be identified as the position attributable to the first gesture 21 within the vehicle interior 20. Furthermore, the position attributable to the operator 40 could also be defined by the center point of the operator 40's head.

[0080] From the intersection point S, two lines, namely the first line SA and the second line SB, are determined and extend to the first vertex A and the second vertex B. The first line SA and the second line SB form the sides of an angle. The angle of view α is defined between lines SA and SB. Determining lines SA and SB and calculating the angle of view α can be performed, for example, by a computer. The determined angle of view α can then be sent by the computer to a control unit, which in turn controls the camera to capture at least one digital image. Preferably, the angle of view α corresponds to the camera's angle of view, i.e., the angle set on the camera to photograph a specific area around the vehicle.

[0081] If the vehicle has not just one image capture device, but several, for example, three image capture devices mounted externally, a direction attributable to the first gesture can be determined in addition to the image angle α. For example, the aforementioned capture device can determine the center point M of the first image section 24, which is shaped as a rectangle. Subsequently, a direction vector R can be determined by the processing unit from the previously determined position attributable to the operator 40, in this case, from the intersection point S. This direction vector R can then indicate the direction in which the first gesture is executed and thus represents a direction attributable to the first gesture.

[0082] Depending on the direction in which the direction vector R points, one of the three cameras of the vehicle can now be selected to capture at least one digital image 27 of the vehicle's surroundings 22. The direction and the field of view α can be transmitted from the computer to the control unit. The control unit can then activate the camera whose orientation and captureable field of view at least partially correspond to the direction vector R and the field of view α.

[0083] Fig. Figure 4 shows how a focus field 33 of the image acquisition device 35 can be determined based on the operator's gaze direction 31. For this purpose, in Fig. Figure 4 shows the eye of an operator 30 and the hand of an operator, with which the operator performs a first gesture 21, from a side view. The operator is preferably located in the interior of a motor vehicle 20 and looks out through a window 25 from the interior of the motor vehicle 20 towards the surroundings of the motor vehicle 22. An image capture device 35 is arranged on the outside of the motor vehicle, which is designed to capture a digital image 27 of the surroundings of the motor vehicle 22 depending on the direction of view 31 and the first gesture 21 of the operator.

[0084] The in Fig. The exemplary embodiment shown in Figure 4 could be based on the following situation. An occupant, who preferably also represents the operator 40, is sitting in a motor vehicle. The occupant wants to capture a digital image of the vehicle's surroundings. The occupant therefore performs a first gesture 21, which preferably represents a frame and thus defines the first image section 24. How the image capture device 35 with a viewing angle α can be controlled depending on the first gesture 21 is described, among other things, in Figure 4. Fig. 2 and Fig. 3 shown.

[0085] In addition to gesture recognition, the vehicle interior 20 can also be equipped with a gaze direction detection device (eye tracker) that enables the operator's eye movements to be recorded and a gaze direction 31 to be determined from them. The eye tracker can, for example, be designed as glasses worn by the operator. Alternatively, the eye tracker can also be designed as an external device located in the vehicle.

[0086] The gaze direction can be detected, for example, by electrooculograms or the corneal reflex method using an eye tracker. Once the gaze direction 31 has been detected by the eye tracker, it can be transmitted, along with the field of view α, to a control unit that controls the image acquisition device 35. In addition to the camera field of view γ, the control unit also sets a focus field 33 for the image acquisition device 35. Preferably, the camera field of view γ corresponds to the defined field of view α. The focus field 33 is determined based on the area or point in space that the operator is focusing on from inside the vehicle interior 20.

[0087] The dimensions of the focus field 33 can depend, for example, on the fixation duration, that is, how long the operator fixes their gaze on a specific point. The camera's focus field can be reduced, for instance, if the operator fixes their gaze on a particular point in the vehicle's surroundings for more than one second. Overall, a camera image section 32 is thus defined by the camera's viewing angle γ in the plane in which the operator fixes their gaze on a certain point in the surroundings. All objects located within the focus field 33 can be displayed sharply by the camera on a digital image 27. Once the image acquisition device 35 has captured a digital image of the surroundings, this digital image 27 can be transmitted by the image acquisition device 35 to a display device, in this embodiment the screen 25.The digital image 27 captured by the image acquisition device 35 is now displayed on the disc 25, which can be designed as a digital windscreen.

[0088] In summary, the present invention describes the possibility of taking photographs or videos, i.e., at least a digital image 27, of the vehicle's surroundings from inside a motor vehicle 20 without using a smartphone or digital camera. By means of a finger gesture, i.e., the first gesture 21, in the air, without touching the windshield, the occupant, i.e., the operator 40, can define a desired image frame, i.e., a first image section 24, and save it as a digital photograph. The selected image frame can then be displayed as visual feedback on the windshield or on a display element, i.e., the display device in the vehicle.

[0089] The photo or video section of the surroundings can be defined by a frame gesture, which can be represented by the first gesture 21. A combination of eye tracking and gesture recognition can be used to determine the photo / video frame selected by the user, i.e., the operator 40, i.e., the first image section 24. The desired section, i.e., the desired camera image section 32, can be determined by a combination of the gaze direction 31, the field of view, and the hand gesture, i.e., the operator 40's first gesture 21. The photo or video section is displayed on the digital screen 25 or on a display in the vehicle, which is designed as a display device. This corresponds to the visual feedback of the selected area. The digital image can then be enlarged, reduced, and moved using various gestures, such as the second gesture.

[0090] Overall, the invention demonstrates how a frame gesture for capturing digital images or video sequences from a motor vehicle is made possible as simply as possible, without the use of additional devices such as digital cameras and smartphones.

Claims

[1] Method for capturing a digital image (27) of the environment of a motor vehicle (22), comprising the following steps a) Capturing and evaluating a first gesture (21) performed freely in a motor vehicle interior (20) by an operator (40), wherein the first gesture (21) defines a first image section (24), b) Detection of at least one position attributable to the first gesture (21) and at least one position attributable to the operator (40) in the motor vehicle (22), c) Determining a field of view (α) depending on the positions captured in step b), and d) Controlling an image acquisition device (35) depending on the specified image angle (α) to acquire at least one digital image (27), characterized by the steps g) Detecting and evaluating the gaze direction (31) of the operator (40) in addition to the first gesture (21), whereby a focus field (33) for the controlled image acquisition device (35) is determined depending on the gaze direction (31), and h) Determining the dimensions of the focus field (33) depending on the fixation duration of the gaze direction (31) of the operator (40). [2] Method according to any one of the preceding claims, characterized by , that e) in step d) where there is a large number of image capture devices of the motor vehicle (22), the image capture device (35) is controlled whose orientation and detectable image angle correspond at least partially to a direction attributable to the first gesture (21) and the specified image angle (α). [3] Method according to any of the preceding claims characterized by , that f) the image acquisition device (35) in step d) is controlled in such a way that at least one geometric point attributable to the digital image (27) to be acquired is aligned with a geometric point attributable to the first gesture (21). [4] Method according to any one of the preceding claims, characterized by i) Capturing at least one digital image (27), and j) Sending the captured digital image (27) to an in-vehicle display device. [5] Method according to claim 4, characterized by k) Displaying the captured digital image (27) on the vehicle's internal display device, and l) Capturing and evaluating a second gesture of the operator (40) freely performed in the vehicle interior (20), wherein the second gesture results in the processing of the digital image (27) on the display device. [6] Method according to any one of the preceding claims, characterized by m) Storing the digital image (27) in an in-vehicle and / or external storage device. [7] Method according to claim 4, characterized by , that n) in step i) a large number of digital images (27) are captured, wherein the large number of digital images (27) represents a video sequence. [8] Motor vehicle (22) with an image capture device for capturing a digital image (27) of the environment of the motor vehicle (22), comprising - a gesture recognition device designed to recognize and evaluate a first gesture (21) performed freely in the vehicle interior (20) by an operator (40), wherein the first gesture (21) defines a first image section (24), - a detection device designed to detect at least one position attributable to the first gesture (21) and at least one position attributable to the operator (40) in the motor vehicle (22), - a computing device designed to determine a field of view (α) depending on the detected positions, and - a control device designed to control the image acquisition device (35) depending on the specified image angle (α) to acquire at least one digital image (27), characterized by , that the image acquisition device (35) is configured to capture and evaluate, in addition to the first gesture (21), a gaze direction (31) of the operator (40), whereby a focus field (33) for the controlled image acquisition device (35) is defined depending on the gaze direction (31), and to determine the dimensions of the focus field (33) depending on the fixation duration of the gaze direction (31) of the operator (40).

Images