Method for operating an access system in a motor vehicle, device for carrying out said method and a motor vehicle equipped with such a device

Abstract

To provide a method for locking and unlocking and / or automatically operating vehicle flaps and / or doors without requiring manual interaction with the vehicle. The invention relates to a method for operating an access system (10) for a motor vehicle (3), in which the foot (5) of at least one person (6) is detected using a camera arrangement (4), a direction (8) of the foot (5) is determined using a software tool, and a gaze direction (9) of the person (6) is determined based on the determined direction (8). If the gaze direction (9) is directed towards at least one flap and / or door (11) of the motor vehicle (3), the access system (10) unlocks and / or automatically opens the at least one flap and / or door (11). On the other hand, if the gaze direction (9) is not directed towards the at least one flap and / or door (11), the access system (10) locks and / or automatically closes the flap and / or door (11). The invention relates to an apparatus (25) for carrying out the method as specifically mentioned, as well as to a motor vehicle (3) equipped with such an apparatus (25).

Description

[Technical field]

[0001] The present invention relates to a method for operating an access system of a motor vehicle according to the preamble of claim 1. The invention also relates to a device, in particular for carrying out said method, as well as to a motor vehicle preferably equipped with such a device. [Background technology]

[0002] A method of the initially mentioned kind is described in the document DE 10 200 03 133 A1, whereby the orientation of a person's feet can be used to activate an access system for locking and unlocking and / or automatically operating the flaps and / or doors of a vehicle. The drawback here is that using the orientation of the feet to determine whether the person mentioned really wants access to the vehicle is unreliable and can lead to false activation of the access system.

[0003] A further method for operating an access system in a motor vehicle is known from DE 10 2004 0133 A1. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Specification of German Utility Model No. 20 2019 104 268 U1 [Patent Document 2] DE 10 2016 108 702 A1 Summary of the Invention [Problem to be solved by the invention]

[0005] The object of the present invention is to provide an improved or at least one alternative embodiment for a method for operating an access system of a motor vehicle, in particular for locking and unlocking the flaps and / or doors of the motor vehicle and / or for automatic operation, without the need for manual contact with the motor vehicle. [Means for solving the problem]

[0006] According to the invention, this problem is solved in particular by the subject matter of the independent claims. Advantageous embodiments are the subject matter of the dependent claims and the description.

[0007] The basic idea of ​​the invention is based on the fact that an access system for locking and unlocking and / or automatically operating flaps and / or doors of a motor vehicle operates depending on the determined direction of a person's gaze.

[0008] For this purpose, according to the invention, a method for operating an access system of a motor vehicle is provided, in which at least one foot of a person approaching or standing near the motor vehicle is detected by a camera arrangement of the motor vehicle, then a software tool is used to first determine the orientation of the at least one foot detected by the camera arrangement, and then based on the determined orientation, the gaze direction of the person is determined. With the access system for locking and unlocking and / or automatically operating flaps and / or doors of the motor vehicle, at least one flap and / or door of the motor vehicle is unlocked and / or automatically operated open if the determined gaze direction is, in particular substantially or exactly, directed towards the at least one flap and / or door, respectively. Furthermore, with the mentioned access system, at least one flap and / or door of the motor vehicle is locked and / or automatically operated closed if the determined gaze direction is, in particular substantially or exactly, not directed towards the at least one flap and / or door, respectively. The method according to the invention thereby allows the mentioned person, when approaching the vehicle, to contactlessly activate the access system of the vehicle by simply directing his / her gaze towards the flap or door, thereby unlocking or locking the flap and / or door. This allows the person to relatively quickly and comfortably enter the vehicle or access the storage compartments covered by the flap of the vehicle. Furthermore, false activation of the access system can also be substantially eliminated, since it can be reliably determined whether the mentioned person is requesting access to the vehicle based on the determined gaze direction. Since the person does not have to or does not have to actively interact with the vehicle, this is further advantageously freeing up time for other activities, which can result in an improved comfortable experience achieved via the vehicle.

[0009] The person approaching the motor vehicle is preferably a user of the motor vehicle who is authorized to use the motor vehicle. This authorization may in particular be time-limited or may be permanently granted. The task of checking the authorization of this person can be performed actively in the manner described here, in particular by communication with a mobile terminal carried by the authorized user, which can be realized by a so-called intelligent key (smart key) or a mobile phone (smartphone) or a so-called CoD, i.e. a consumer owned smart device of the motor vehicle.

[0010] The term "foot" in the sense of the present invention is preferably the lowest part of a person's leg, which in particular comprises the metatarsus, which is connected to the tarsus adjacent to the lower leg, as well as the distal toes, which are located on the metatarsus.

[0011] Preferably, it is provided that in determining the orientation of at least one foot detected by the camera arrangement, its toe is determined. Furthermore, in this context, it may be provided that the determined gaze direction is oriented parallel or substantially parallel to a toe vector passing at least the determined toe of the detected foot. Preferably, the toe vector may also pass through the base of the detected foot or through the lower leg of the said person. This makes it possible to relatively simply determine the toe vector as well as the gaze direction of the person based on the determined foot.

[0012] The toes and / or metatarsus of a person's foot may preferably define a toe within the meaning of the present invention.

[0013] Furthermore, according to the invention, a classification object, in particular a classification circle, consisting of at least two categories, which advances in real time in association with the mentioned person is provided by means of a software tool for classifying the determined gaze direction. This classification object is preferably realized by a classification circle, which can be oriented horizontally in particular with respect to the ground on which the car is located. Furthermore, the mentioned person can be provided to be continuously centered with respect to the accompanying and advancing classification circle. The classification object or the classification circle thus realizes a software-generated auxiliary tool for quickly and reliably classifying the gaze direction of a determined person in real time.

[0014] Furthermore, according to the invention, the classification object, in particular the classification circle, is provided in such a way that it is divided into a first classification area facing the flap and / or door of the motor vehicle and a second classification area facing away from this flap and / or this door of the motor vehicle, such that if the determined direction of sight is in particular substantially or exactly in the first classification area of ​​the classification circle, this flap and / or this door of the motor vehicle is unlocked and / or automatically opened by means of the access system, and if the determined direction of sight is in the second classification area of ​​the classification circle, this flap and / or this door of the motor vehicle is locked and / or automatically closed by means of the access system. In other words, a determined direction of sight that is within the classification object, in particular the first classification area of ​​the classification circle, is permission for unlocking and / or automatically opening the respective flap or door of the motor vehicle, whereas a determined direction of sight that is not within the first classification area of ​​the classification circle is permission for locking and / or automatically closing the respective flap or door of the motor vehicle. The first classification area could therefore also be called the unlocking area and the second classification area the locking area.

[0015] Advantageously, the method can be characterized in that the first classification area, in particular when viewed in a horizontal plane, subtends an angle of at least 10° and at most 180°, in particular an angle of at least 30° and / or at most 90°. Also advantageously, the second classification area can further subtend a corresponding angle, in a horizontal plane, that is complementary to the angle of the first classification area. Alternatively, the second classification area can subtend an angle of at least 180° and at most 350°, in particular an angle of at least 330° and / or at most 270°, in a horizontal plane. In other words, the first classification area subtends an angle of at least 10° and at most 180°, in particular an angle of at least 30° and / or at most 90°, between its opposite area ends in the horizontal plane. The second classification area can be formed complementary to this angle or can form a predefined angle between a minimum of 180° and a maximum of 350°, in particular an angle of at least 330° and / or an angle of at most 270°. In practice, the angular range of the first and second classification areas is relatively wide, so that it can be relatively reliably and quickly determined whether the determined gaze direction is in the first or second classification area. The classification object formed in this way, in particular the classification circle, can therefore ensure a method that is relatively robust against disturbances when detecting the foot and against determination uncertainties when determining the orientation.

[0016] The horizontal plane referred to may preferably be constituted by the longitudinal center axis of the vehicle and a lateral axis perpendicular to said axis. The horizontal plane referred to may preferably be oriented parallel to the ground on which the vehicle is placed.

[0017] It is obvious that a classification object, in particular a classification circle, with more than two classification areas can also be provided, for example a third classification area separating the first and second classification areas, and if the determined gaze direction is within the third classification area, then a further vehicle function, such as a comfort function, can be automatically activated.

[0018] Furthermore, it may be provided that, by means of a software tool for each flap and / or door of the vehicle, a control field is provided, in particular directly adjacent to the respective flap and / or door, which is in particular formed in two dimensions and planarly and in particular oriented horizontally, in which, if the mentioned person is not within the control field, the access system is preferably placed in a locked state in which activation of the access system is not possible, independently of the determined line of sight, and, if the mentioned person is within the control field, the access system is preferably placed in an unlocked state in which activation of the access system is possible depending on the determined line of sight. A safety function is thus presented which allows locking and unlocking and / or automatic operation of the flaps and / or doors of the vehicle only if the mentioned person is actually present near the vehicle.

[0019] Furthermore, the mentioned software tool may preferably be provided in a computing unit of the vehicle. In this case, the mentioned software tool can preferably be realized by means of computer-based vision or machine vision and / or by algorithms for recognizing and measuring the geometric structure and the movement of objects. The software tool is thus implemented in the mentioned vehicle, so that the proposed method can be executed in or by the vehicle. In principle, however, the software tool can also be externalized and implemented, for example, in a fixed server. The proposed method can then be executed by communication between the external and the vehicle, for example via wireless communication. Furthermore, the proposed method also allows a relatively fast and in particular real-time detection of at least one foot and / or toe of a person approaching or standing near the vehicle, since the image data detected by the camera arrangement are analyzed relatively efficiently. Advantageously, the computer-based vision can additionally detect the person's body position and / or location.

[0020] Furthermore, the method can be characterized in that the mentioned software tools and / or algorithms for recognizing and measuring the geometric structure and the movement of objects are trained with so-called real data and / or machine learning methods. The mentioned real data can then be determined in the vehicle, for example in a learning cycle of the method, which is carried out at the manufacturer's side. In particular machine learning methods, such as for example "Deep Learning", can be used here. The so-called real data preferably represent data that are determined in the vehicle, for example in a learning cycle of the method, which is carried out at the manufacturer's side. In this case, the carried out learning cycle preferably covers as widely as possible the possible practical scenarios. In particular, it is possible to carry out a learning cycle for all flaps and / or doors of the vehicle, whereby different ambient and weather conditions as well as time and light conditions are simulated with subjects with different body types, different clothes and / or fixed angular trajectories. The collected real data thus serves to train the software tools and / or the mentioned algorithms and to make them relatively robust.

[0021] According to the mentioned method, it is preferably provided that the acquired actual data can be corrected in the adaptation process, whereby, optionally, the acquired actual data is manually supplemented with attributes of the mentioned person. Such attributes can be formed, for example, by a boundary frame, a lane ID or a body position (posture). For example, it is possible to train the software tool and / or the mentioned algorithm after carrying out the adaptation process.

[0022] The software tools and / or mentioned algorithms may suitably provide data that includes the current location of the mentioned person, the direction of movement of the person, the speed of movement of the person, and the orientation of the person.

[0023] A further basic idea of ​​the invention is to provide an apparatus for carrying out the aforementioned method, in particular mountable on a motor vehicle, which apparatus is adapted to carry out the aforementioned method, comprising a camera arrangement mountable or mounted on the motor vehicle, a computing unit communicatively connected to the camera arrangement, adapted to carry out computing tasks and adapted to control an access system for locking and unlocking and / or automatically operating flaps and / or doors of the motor vehicle, and a computer-readable storage medium communicable with or formed by said computing unit, the latter comprising in particular commands which, when executed by the computing unit, command the proposed apparatus to carry out the mentioned method according to the preceding description. Hereby, an advantageous embodiment of the apparatus for carrying out the aforementioned method is presented.

[0024] In this context, it may be preferred if the camera arrangement is realised by at least one omnidirectional camera which can or is mounted on the motor vehicle. Exactly four omnidirectional cameras arranged on the motor vehicle can be shown by way of example. With such omnidirectional cameras, a 360° omnidirectional view around the motor vehicle can preferably be provided. The omnidirectional camera may in particular be formed by a so-called SVC camera or by an omnidirectional parking camera which is adapted to carry out the parking assistant function. It is likewise preferred to provide a 360° omnidirectional view around the motor vehicle also by using said omnidirectional parking or SVC cameras ("surround vision cameras", abbreviated SVC). Furthermore preferably, four omnidirectional cameras are arranged on the motor vehicle, in particular one in the area of ​​the front bumper, another in the area of ​​the rear bumper and one each in the area of ​​the A-pillar of the motor vehicle or in the area of ​​the motor vehicle side mirrors.

[0025] In particular, the at least one omnidirectional camera may be arranged such that a central axis of its field of view, passing through the center of its field of view, is directed obliquely to the longitudinal central axis of the vehicle and towards the ground on which the vehicle is located. In other words, the at least one omnidirectional camera is preferably directed such that its field of view points obliquely downwards towards the ground. This allows the at least one omnidirectional camera to detect obstacles in the vicinity of the vehicle, in particular on the ground side, as well as the feet and / or legs of persons near or approaching the vehicle. In the context of the present invention, the term "field of view" is understood to mean in particular a three-dimensional area in which the optical device can detect objects.

[0026] A further basic idea of ​​the invention is to provide a vehicle capable of carrying out the above-mentioned method. To that end, a vehicle is provided that is equipped with an apparatus according to the above description, which apparatus is adapted to carry out the above-mentioned method. A preferred embodiment of a vehicle for carrying out the above-mentioned method is thereby presented.

[0027] In summary, the invention preferably relates to a method for operating an access system of a motor vehicle, where at least one foot of a person is detected using a camera arrangement, the orientation of the foot is determined using a software tool, and then a gaze direction of the person is determined based on the determined orientation. If the gaze direction is towards at least one flap and / or door of the motor vehicle, the access system unlocks and / or automatically opens the at least one flap and / or door. Conversely, if the gaze direction is not towards the at least one flap and / or door, the access system locks and / or automatically closes the flap and / or door. The invention also relates to an apparatus for carrying out the particularly mentioned method, as well as to a motor vehicle preferably equipped with such an apparatus.

[0028] Further important features and advantages of the invention emerge from the dependent claims, the drawing and the corresponding description based on the drawing.

[0029] It is to be understood that the features listed above, and further described below, can not only be used in the respective combinations indicated, but can also be used in other combinations or alone, without departing from the scope of the present invention.

[0030] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention are illustrated in the drawings and described in detail in the following description, where like reference numbers indicate identical or similar or functionally identical parts. [Brief description of the drawings]

[0031] [Figure 1] 1 is a schematic plan view from above of a motor vehicle equipped with a device for carrying out the method according to the invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0032] 1 shows a top view of a motor vehicle, generally designated by the reference number 3. The motor vehicle has flaps and doors 11 which are operable by an access system 10 integrated in the motor vehicle 3 for locking and unlocking and / or automatically operating them, i.e. the flaps and doors 11 can be unlocked and / or automatically operated open, or locked and / or automatically operated closed, in the usual way by the access system 10.

[0033] In order to simplify access to the vehicle 3 for persons 6 (here represented by a block) authorized to use the vehicle 3, in particular to achieve this relatively quickly and comfortably and without the need to touch the vehicle 3, the vehicle 3 is exemplarily equipped with a device 25 which is adapted to carry out the method according to the invention. This device 25 exemplarily comprises four independent omnidirectional cameras 27 (represented by triangles) of the camera arrangement 4, a computing unit 24 (represented by a block) mounted on the vehicle 3 and a computer-readable storage medium 26 which is here integrated in the computing unit 24.

[0034] The four omnidirectional cameras 27 exemplarily jointly realize a 360° omnidirectional view of the surroundings of the motor vehicle 3, and are therefore also basically suitable for carrying out the parking assistant function of the motor vehicle 3. Exemplarily, one omnidirectional camera 27 is arranged in the area of ​​the front bumper of the motor vehicle 3, another omnidirectional camera 27 is arranged in the area of ​​the rear bumper of the motor vehicle 3 and one omnidirectional camera 27 is arranged in each case in the area of ​​the A-pillar of the motor vehicle 3. The omnidirectional cameras 27 can preferably be arranged such that a central axis of their field of view passing through the center of their field of view is directed obliquely to the longitudinal central axis 21 of the motor vehicle 3 and towards the ground 15 on which the motor vehicle 3 is arranged. This results in a so-called camera line of sight obliquely downwards onto the ground 15, so that the camera can detect obstacles in particular on the ground side near the motor vehicle 3, as well as the feet 5 and / or legs of a person 6 beside the motor vehicle 3 or approaching it.

[0035] The mentioned computing unit 24 is adapted to carry out computing tasks and to control the access system 10. To that end, this computing unit 24 is communicatively connected to the omnidirectional camera 27 of the camera arrangement 4 via a communication cable, which is shown as a line in Fig. 1. The mentioned storage medium 26 is here integrated in the computing unit 24 and contains commands executable by the computing unit 24, which, when executed by the computing unit 24, cause the device 25 to carry out the method according to the invention.

[0036] The method according to the invention exemplarily provides for the detection of the toe 12 of at least one foot 5 of a person 6 approaching or standing next to the motor vehicle 3 by the omnidirectional camera 27 of the camera arrangement 4, whereby a direction 8 of at least one foot 5 or toe 12 is first determined by a software tool (not shown) and then a gaze direction 9 of the person 6 is derived based on the determined direction 8. Here, exemplarily, it may be based on the idea that the determined gaze direction 9 of the person 6 can be oriented parallel or at least substantially parallel to a toe vector 13 passing through at least the determined toe 12. It is then checked whether the determined gaze direction 9 is directed towards a flap and / or a door 11. If so, at least this flap and / or door 11 of the motor vehicle 3 is unlocked and / or automatically opened by the access system 10. On the other hand, if the determined gaze direction 9 is not directed towards this flap and / or door 11, the flap and / or door 11 is locked and / or automatically closed by the access system 10. The method according to the invention thereby enables the access system 10 to be contactlessly operable by the mentioned person 6, and thus enabling the person 6 to unlock or lock the flap and / or door 11 simply by directing their gaze towards it.

[0037] In order to simplify and make robust the checks carried out to see whether the determined gaze direction 9 is directed towards the flap and / or door 11 of the motor vehicle 3, a classification object that proceeds in real time with the mentioned person 6 may be provided by means of a software tool, which classification object is realized according to this embodiment by way of example by a classification circle 14 for classifying the detected gaze direction 9, shown in dotted lines in FIG. 1. The classification circle 14 is by way of example oriented horizontally with respect to the ground 15. Furthermore, the accompanying classification circle 14 is provided so as to be continuously centered with respect to the person 6. Furthermore, according to FIG. 1, the classification circle 14 is divided into a first classification area 16 facing towards the flap and / or door 11 of the motor vehicle 3 and a second classification area 17 facing away from this flap and / or this door 11 of the motor vehicle 3, i.e. formed of two classes. The first classification area 16 has an angle of exemplarily 90° between its area ends. The complementary second classification area 17 subtends an angle of 270° between its area ends. The operation of the access system 10 is performed with the intention that the flap and / or door 11 is unlocked and / or automatically opened by the access system 10 if the determined line of sight 9 is within the first classification area 16 of the classification circle 14, and that this flap and / or this door 11 is locked and / or automatically closed by the access system 10 if the determined line of sight 9 is not within the first classification area 16, i.e. within the second classification area 17 of the classification circle 14.

[0038] 1 furthermore provides a control field 23, indicated by a dashed line, arranged at one flap and / or door 11, by means of which a defined distance between the motor vehicle 3 and the person 6 can be checked, such that if the mentioned person 6 is not within the control field 23, the access system 10 is preferably placed in a locked state in which activation of the access system 10 is not possible, independently of the determined line of sight 9, and if the mentioned person 6 is within the control field 23, the access system 10 is preferably placed in an unlocked state in which activation of the access system 10 is possible, depending on the detected line of sight 9. This presents a certain safety function, by means of which locking and unlocking and / or automatic operation of the flap and / or door 11 of the motor vehicle 3 is only possible if the mentioned person 6 is actually near the motor vehicle 3, i.e. within a defined or definable distance to the motor vehicle 3.

[0039] In the above, when a horizontal orientation is mentioned, this particularly means a horizontal plane 18, which is preferably formed by a longitudinal central axis 21 and a transverse axis 22 perpendicular to said axis. The horizontal plane 18 mentioned is exemplarily oriented parallel to the ground 15.

Claims

1. A camera arrangement (4) of a vehicle (3) detects at least one foot (5) of a person (6) approaching or standing near the vehicle (3); an orientation (8) of the at least one foot (5) detected by the camera arrangement (4) is determined; A method for operating an access system (10) of a vehicle (3), comprising the steps of: Based on the determined orientation (8), a gaze direction (9) of the person (6) is determined; using an access system (10) for locking and unlocking and / or automatically operating flaps and / or doors (11) of the motor vehicle (3), at least one flap and / or door (11) of the motor vehicle (3) is unlocked and / or automatically operated open if the determined line of sight (9) points in the direction of the at least one flap and / or door (11), respectively, and / or using the access system (10) at least one flap and / or door (11) of the motor vehicle (3) is locked and / or automatically operated closed if the determined line of sight (9) does not point in the direction of the at least one flap and / or door (11), respectively; a classification object, in particular a classification circle (14), consisting of at least two categories, which progresses in real time accompanying the person (6) by means of a software tool, is provided for classifying the determined gaze directions (9), the classification object, in particular the classification circle (14), is divided into a first classification area (16) facing the flaps and / or doors (11) of the motor vehicle (3) and a second classification area (17) facing away from the flaps and / or doors (11) of the motor vehicle (3), using said access system (10) to unlock and / or automatically open flaps and / or doors (11) of said motor vehicle (3) if said determined line of sight (9) is in said first classification area (16) of said classification circle (14), Using the access system (10), the flaps and / or doors (11) of the vehicle (3) are locked and / or automatically closed if the determined line of sight (9) is in the second classification area (17) of the classification circle (14). A method comprising:

2. In determining the orientation (8) of the at least one foot (5) detected by the camera arrangement (4), its toe (12) is determined.

2. The method of claim 1 .

3. The determined gaze direction (9) is oriented parallel to a toe vector (13) passing through at least the determined toe (12) of the detected foot (5).

3. The method of claim 2.

4. the first sorting area (16) subtends an angle of at least 10° and at most 180°, in particular at least 30° and / or at most 90°; the second classification area (17) makes a corresponding angle that is complementary to the angle made by the first classification area (16); or The second classification area (17) subtends an angle of at least 180° and at most 350°, in particular at least 330° and / or at most 270°, when viewed in a horizontal plane (18). The method according to any one of claims 1 to 3.

5. said software tool being provided in a computing unit (24) of said vehicle (3); and / or The software tools are implemented by computer-based visual or machine vision techniques and / or algorithms for recognizing and measuring the geometry and movement of objects. The method according to any one of claims 1 to 4.

6. said software tools and / or said algorithms for recognizing and measuring the geometry and motion of objects are trained using real data and / or machine learning techniques; and / or The actual data is determined on the vehicle (3) during a learning cycle of the method.

6. The method of claim 5.

7. After obtaining the actual data, the actual data is modified in a fitting process; Optionally, the captured real data is manually supplemented with attributes of the person (6).

7. The method of claim 6.

8. An apparatus (25) for carrying out the method according to any one of claims 1 to 7, comprising: a camera arrangement (4) that can be mounted or is mounted on a motor vehicle (3); a computing unit (24) communicatively connected to the camera arrangement (4) that can be mounted or is mounted on the motor vehicle (3), adapted to perform computing tasks and to control an access system (10) for locking and unlocking and / or automatically operating flaps and / or doors (11) of the motor vehicle (3); a computer readable storage medium (26) in communication with or formed by said computing unit (24) having commands which, when executed by said computing unit (24), instruct said device (25) to carry out the method of claims 1 to 7; An apparatus (25), comprising:

9. said camera arrangement (4) being realised by at least one omnidirectional camera (27) which can be mounted or is mounted on said vehicle (3); and / or Four omnidirectional cameras (27) are arranged on the vehicle (3).

9. The device (25) according to claim 8.

10. The at least one omnidirectional camera (27) has a central axis of view passing through the center of its field of view, which is oblique to the longitudinal central axis (21) of the vehicle (3) and is directed toward the ground (15) on which the vehicle (3) is placed.

10. The device (25) of claim 9.

11. A motor vehicle (3) comprising a device (25) according to claims 8 to 10, arranged to carry out the method according to any one of claims 1 to 7.

Images