A method for operating an access system in an automobile, a device for implementing said method, and an automobile equipped with such a device.

Abstract

To provide a method for locking and unlocking and / or automatically operating vehicle flaps and / or doors without requiring manual interaction with the vehicle. The invention relates to a method for operating an access system (10) for a motor vehicle (3), in which the foot (5) of at least one person (6) is detected using a camera arrangement (4), a direction (8) of the foot (5) is determined using a software tool, and a gaze direction (9) of the person (6) is determined based on the determined direction (8). If the gaze direction (9) is directed towards at least one flap and / or door (11) of the motor vehicle (3), the access system (10) unlocks and / or automatically opens the at least one flap and / or door (11). On the other hand, if the gaze direction (9) is not directed towards the at least one flap and / or door (11), the access system (10) locks and / or automatically closes the flap and / or door (11). The invention relates to an apparatus (25) for carrying out the method as specifically mentioned, as well as to a motor vehicle (3) equipped with such an apparatus (25).

Description

Technical Field

[0001] The present invention relates to a method of operating an access system for a motor vehicle as described in the first part of claim 1. The present invention also relates particularly to an apparatus for carrying out said method, and to a motor vehicle preferably equipped with such an apparatus.

Background Art

[0002] A method of the type mentioned at the beginning is described in the following Patent Document 1. According to this, an access system can be operated to lock and unlock and / or automatically operate a flap and / or door of a motor vehicle by utilizing the orientation of a person's foot. The drawback in this case is that it is not reliable to determine whether the person in question actually requests access to the motor vehicle by the orientation of the foot, which may cause malfunction of the access system.

[0003] From the following Patent Document 2, a further method for operating an access system for a motor vehicle is known.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] In contrast, the object of the present invention is to provide an improved embodiment or at least one alternative embodiment for a method of operating an access system for a motor vehicle. In particular, a method is provided in which manual contact with the motor vehicle is not required to lock and unlock and / or automatically operate a flap and / or door of the motor vehicle. [Means for solving the problem]

[0006] In the present invention, this problem is solved, in particular, by the subject matter of the independent claims. Advantageous embodiments are the subject matter of the dependent claims and the specification.

[0007] The basic concept of the present invention is based on an access system for locking and unlocking and / or automatically operating the flaps and / or doors of an automobile, which operates according to a determined line of sight of a person.

[0008] To this end, the present invention provides a method for operating an access system for a vehicle, wherein a camera arrangement in the vehicle detects at least one foot of a person approaching or standing near the vehicle, then a software tool is used to first determine the orientation of the at least one foot detected by the camera arrangement, and then, based on the determined orientation, the direction of the person's gaze is determined. Using the access system for locking and unlocking and / or automatically operating the vehicle's flaps and / or doors, if the determined direction of gaze is directed, in particular substantially or precisely, towards at least one flap and / or door, then at least one flap and / or door of the vehicle is unlocked and / or automatically opened. Furthermore, using the access system described above, if the determined direction of gaze is not directed, in particular substantially or precisely, towards at least one flap and / or door, then at least one flap and / or door of the vehicle is locked and / or automatically closed. As a result, the method according to the present invention allows the person referred to as the vehicle to activate the vehicle's access system contactlessly simply by looking at the flap or door when approaching the vehicle, thereby unlocking or locking the flap and / or door. This allows the person to enter the vehicle or access storage compartments covered by the vehicle's flap relatively quickly and comfortably. Furthermore, since it is possible to reliably determine whether the person referred to as the vehicle is requesting access based on the determined gaze direction, malfunctions of the access system can be substantially eliminated. Further advantage is that since the person does not need to actively touch the vehicle, or does not need to touch it substantially, time can be freed up for other things, and as a result, the comfort experience achieved through the vehicle can be improved.

[0009] A person approaching a vehicle is preferably a user of the vehicle who has been authorized to use it. This authorization may be time-limited or permanent. The process of checking this person's authorization can be actively performed in the methods described herein, particularly by communication with a mobile device carried by the authorized user, which can be a so-called intelligent key (smart key), a mobile phone (smartphone), or a so-called CoD, i.e., a consumer-owned smart device.

[0010] In the sense of this invention, the term "foot" preferably refers to the lowest part of a human leg. The foot has, in particular, a metatarsal that is connected to the tarsal, which is adjacent to the lower leg, and the distal toes located on the metatarsal.

[0011] Preferably, the orientation of at least one foot detected by the camera configuration is determined, and its toes are determined. Furthermore, in this context, the determined line of sight direction may be provided to be directed parallel or substantially parallel to a toe vector passing through the determined toes of at least the detected foot. Preferably, the toe vector may further pass through the tarsal region of the detected foot or through the lower leg of the person's leg. This makes it relatively easy to determine the toe vector and the person's line of sight based on the determined foot.

[0012] The toes and / or metatarsal of a human foot may preferably define the toe in the sense of the present invention.

[0013] Furthermore, based on the present invention, a software tool is provided to classify a determined gaze direction, comprising at least two categories of classification objects, particularly classification circles, that progress in real time in conjunction with the person being referred to. This classification object is preferably realized by a classification circle that can be oriented horizontally to the ground on which the vehicle is located. Furthermore, the person being referred to may be positioned to be continuously centered with respect to the progressing classification circle. Thus, the classification object or classification circle realizes a software generation aid tool for rapidly and reliably classifying the gaze direction of a determined person in real time.

[0014] Furthermore, according to the present invention, the classification object, in particular the classification circle, is provided to be divided into a first classification area facing the vehicle's flaps and / or doors, and a second classification area facing away from these flaps and / or doors of the vehicle, such that when a determined line of sight is in particular substantially or precisely within the first classification area of ​​the classification circle, the vehicle's flaps and / or doors are unlocked and / or automatically opened using the access system, and when a determined line of sight is in the second classification area of ​​the classification circle, the vehicle's flaps and / or doors are locked and / or automatically closed using the access system. In other words, a determined line of sight within the classification object, in particular within the first classification area of ​​the classification circle, is a permit to unlock and / or automatically open each of the vehicle's flaps or doors, while a determined line of sight not within the first classification area of ​​the classification circle is a permit to lock and / or automatically close each of the vehicle's flaps or doors. Thus, the first classification area could also be called the unlock area, and the second classification area could be called the lock area.

[0015] Preferably, the method is characterized in that the first classification region has an angle of a minimum of 10° to a maximum of 180°, particularly a minimum of 30° and / or a maximum of 90°, when viewed in the horizontal plane. Likewise, preferably, the second classification region can have a complementary corresponding angle to the angle of the first classification region when viewed in the horizontal plane. Alternatively, the second classification region may have an angle of a minimum of 180° to a maximum of 350°, particularly a minimum of 330° and / or a maximum of 270°, when viewed in the horizontal plane. In other words, the first classification region has an angle of a minimum of 10° to a maximum of 180°, particularly a minimum of 30° and / or a maximum of 90°, between its opposing ends in the horizontal plane. The second classification region is formed complementary to this angle, or it can form a predetermined angle from a minimum of 180° to a maximum of 350°, particularly an angle of a minimum of 330° and / or a maximum of 270°. In practice, since the angular range of the first and second classification regions is relatively wide, it can be determined relatively reliably and quickly whether the determined line of sight is in the first classification region or the second classification region. Therefore, the classification objects formed in this way, especially the classification circles, make it possible to ensure a relatively robust method against disturbances when detecting feet and uncertainty when determining orientation.

[0016] The horizontal plane mentioned may preferably consist of the longitudinal central axis of the vehicle and a transverse axis perpendicular to this axis. The horizontal plane mentioned may preferably be oriented parallel to the ground on which the vehicle is positioned.

[0017] It is self-evident that classification objects with more than two classification areas, especially classification circles, can also be provided. For example, a third classification area may be provided that separates the first and second classification areas, and if the determined gaze direction falls within the third classification area, another automotive function, such as a comfort function, can be automatically activated.

[0018] Furthermore, software tools for each flap and / or door of the vehicle may be provided to offer a control field that is particularly directly adjacent to each flap and / or door, is particularly two-dimensionally planar, and particularly horizontally oriented. In this case, if the person referred to is not within the control field, the access system is preferably locked, regardless of the determined line of sight, and if the person referred to is within the control field, the access system is preferably unlocked, depending on the determined line of sight, allowing the access system to be operated. This presents a safety feature that allows the locking and unlocking and / or automatic operation of the vehicle's flaps and / or doors only when the person referred to is actually near the vehicle.

[0019] More preferably, the software tool mentioned may be provided within the vehicle's computing unit. In this case, the software tool can be implemented, preferably by computer vision or machine vision techniques and / or by algorithms for recognizing and measuring the geometric structure and movement of objects. Thus, since this software tool is implemented in the vehicle, the proposed method can be performed in or by the vehicle. However, it is also possible to move this software tool externally and implement it, for example, on a fixed server. In that case, the proposed method can be performed by communicating between the external environment and the vehicle, for example, via wireless communication. Furthermore, since the image data detected by the camera configuration is analyzed relatively efficiently by the proposed method, it is also possible to detect at least one foot and / or toes of a person approaching or standing near the vehicle relatively quickly, and especially in real time. Preferably, the posture and / or position of a person can be additionally detected by computer vision.

[0020] Furthermore, this method can be characterized in that the software tools and / or algorithms for recognizing and measuring the geometric structure and motion of objects are trained using so-called real-world data and / or machine learning techniques. In this case, the real-world data can be determined, for example, by the automobile in the learning cycle of this method performed by the manufacturer. Here, machine learning techniques, such as "Deep Learning," can be used. The so-called real-world data preferably represents data determined by the automobile in the learning cycle of this method performed by the manufacturer. In this case, the performed learning cycle preferably covers as wide a range of conceivable practical scenarios as possible. In particular, it is possible to perform the learning cycle for all flaps and / or doors of the automobile, in which case different ambient and weather conditions as well as time and brightness conditions are simulated using subjects with various body types, different clothing and / or fixed-angle trajectories. Thus, the collected real-world data helps to train the software tools and / or the algorithms mentioned, making them relatively robust.

[0021] According to the method described, preferably, the real data acquired in the fitting process can be modified, in which case, optionally, the acquired real data is manually supplemented with the attributes of the person described. These attributes can be formed, for example, by boundary frames, lane IDs, or body position (posture). For example, after the fitting process is performed, it is possible to train the software tool and / or the algorithm described.

[0022] The software tool and / or algorithm mentioned can suitably provide data including the current location of the person mentioned, the direction of their movement, the speed of their movement, and their orientation.

[0023] A further basic idea of the present invention is to provide a device, particularly one that can be mounted in a motor vehicle, for carrying out the method described above. This device, which is adapted to carry out the method described above, has a camera arrangement that can be mounted in or is mounted in a motor vehicle, and a computing unit that can be mounted in or is mounted in a motor vehicle and is communicatively connected to the camera arrangement and is adapted to carry out manipulation operations and to control an access system for locking and unlocking flaps and / or doors of the motor vehicle and / or for automated operation. The device also has a computer-readable storage medium that is communicable with the computing unit or is formed by the computing unit, which, in particular when executed by the computing unit, has commands that order the proposed device to carry out the method mentioned based on the above description. Thereby, an advantageous embodiment of a device for carrying out the method described above is presented.

[0024] In this context, it may be suitable if the camera arrangement is implemented by at least one omnidirectional camera that can be mounted in or is mounted in a motor vehicle. Four exactly omnidirectional cameras arranged in a motor vehicle can be exemplarily shown. Using such omnidirectional cameras, preferably a 360° omnidirectional view around the motor vehicle can be provided. The omnidirectional camera may in particular be formed by a so-called SVC camera or an omnidirectional parking camera that is adapted to carry out a parking assistant function. Similarly preferably, it is also possible to provide a 360° omnidirectional view around the motor vehicle by using this omnidirectional parking camera or SVC camera (abbreviation SVC, "surround vision cameras"). More preferably, four omnidirectional cameras are arranged in the motor vehicle, particularly one in the area of the front bumper, another one in the area of the rear bumper, and one each in the area of the A-pillar of the motor vehicle or in the area of the side mirror of the motor vehicle.

[0025] In particular, at least one omnidirectional camera may be provided such that the central axis of the field of view passing through the center of its field of view is oblique with respect to the longitudinal central axis of the motor vehicle and is directed towards the ground on which the motor vehicle is located. In other words, at least one omnidirectional camera is preferably directed such that its field of view is directed obliquely downwards towards the ground. Thereby, at least one omnidirectional camera can detect obstacles near the motor vehicle, particularly on the ground side, as well as the feet and / or legs of a person standing beside the motor vehicle or approaching the same vehicle. In the present invention, the term "field of view" is particularly understood to mean the three-dimensional region in which an optical device can detect an object.

[0026] A further basic idea of the present invention is to provide a motor vehicle capable of executing the above-described method. For this purpose, a motor vehicle equipped with the device according to the above description is provided, and this device is adjusted to execute the method according to the above description. Thereby, a preferred embodiment of the motor vehicle for executing the above-described method is presented.

[0027] In summary, the present invention preferably relates to a method of operating an access system of a motor vehicle, in which at least one person's foot is detected using a camera arrangement, the direction of the foot is determined using a software tool, and then the line-of-sight direction of the person is determined based on the determined direction. When the line-of-sight direction is directed towards at least one flap and / or door of the motor vehicle, the access system unlocks and / or automatically opens at least one flap and / or door. On the contrary, when the line-of-sight direction is not directed towards at least one flap and / or door, the access system locks and / or automatically closes the flap and / or door. The present invention particularly relates to a device for executing the particularly mentioned method, and preferably to a motor vehicle equipped with such a device.

[0028] Further important features and advantages of the present invention will become apparent from the dependent claims, the drawings, and the corresponding description based on the drawings.

[0029] The features listed above, and the features further described below, should be understood to be usable not only in the combinations shown, but also in other combinations or individually, without departing from the scope of the present invention.

[0030] Preferred embodiments of the present invention are shown in the drawings and described in detail below, where the same reference numerals represent the same, similar, or functionally identical parts. [Brief explanation of the drawing]

[0031] [Figure 1] This is a schematic top view of an automobile equipped with a device for carrying out a method according to the present invention. [Modes for carrying out the invention]

[0032] Figure 1 is a top view of an automobile, the entire vehicle being denoted by reference numeral 3. The automobile has flaps and doors 11, which are operable by an access system 10 incorporated in the automobile 3 for locking and unlocking them and / or for automatic operation. In other words, the flaps and doors 11 can be unlocked and / or automatically opened, or locked and / or automatically closed, in a normal manner by the access system 10.

[0033] To simplify access to the vehicle 3 by persons 6 (indicated here by blocks) authorized to use the vehicle 3, particularly to be achieved relatively quickly and comfortably and without requiring them to touch the vehicle 3, the vehicle 3 is equipped with, exemplary, a device 25 adapted to perform a method according to the present invention. This device 25 exemplary comprises four independent omnidirectional cameras 27 (indicated by triangles) in a camera arrangement 4, a computing unit 24 (indicated by blocks) mounted on the vehicle 3, and a computer-readable storage medium 26 incorporated here in the computing unit 24.

[0034] The four omnidirectional cameras 27 jointly provide a 360° panoramic view of the area around the vehicle 3, and are therefore also suitable for performing the parking assistant function of the vehicle 3. For example, one omnidirectional camera 27 is positioned in the area of ​​the front bumper of the vehicle 3, another omnidirectional camera 27 is positioned in the area of ​​the rear bumper of the vehicle 3, and one omnidirectional camera 27 is positioned in the area of ​​each A-pillar of the vehicle 3. Preferably, the omnidirectional cameras 27 may be positioned such that the central axis of their field of view, passing through the center of their respective fields of view, is oblique to the longitudinal central axis 21 of the vehicle 3 and oriented toward the ground 15 on which the vehicle 3 is positioned. This results in a so-called camera line of sight that is obliquely below the ground 15, allowing the cameras to detect obstacles near the vehicle 3, particularly on the ground side, as well as the feet 5 and / or legs of a person 6 near the vehicle 3 or approaching the vehicle.

[0035] The arithmetic unit 24 mentioned is configured to perform calculations and control the access system 10. To this end, the arithmetic unit 24 is communicably connected to the omnidirectional camera 27 of camera arrangement 4 via a communication cable shown by a line in Figure 1. The storage medium 26 mentioned here is incorporated into the arithmetic unit 24 and contains commands that can be executed by the arithmetic unit 24, which, when executed by the arithmetic unit 24, cause the device 25 to perform a method according to the present invention.

[0036] The method according to the present invention provides, exemplary, that the omnidirectional camera 27 of camera arrangement 4 detects the toe 12 of at least one foot 5 of a person 6 approaching or standing near a vehicle 3, at which point the orientation 8 of at least one foot 5 or toe 12 is first determined by a software tool (not shown), and then the gaze direction 9 of the person 6 is derived based on the determined orientation 8. Hereinafter, exemplary, the gaze direction 9 of the person 6 to be determined may be based on the idea that it can be directed parallel to or at least substantially parallel to a toe vector 13 passing through at least the determined toe 12. Subsequently, it is checked whether the determined gaze direction 9 is directed towards a flap and / or door 11. If it is, the access system 10 unlocks and / or automatically opens at least this flap and / or door 11 of the vehicle 3. Conversely, if the determined line of sight direction 9 is not directed toward the flap and / or door 11, the access system 10 locks and / or automatically closes the flap and / or door 11. Thus, the method according to the present invention makes it possible for the access system 10 to be operated non-contact by the person 6 mentioned, and thus the person 6 can unlock or lock the flap and / or door 11 simply by directing their gaze toward the flap or door 11.

[0037] To simplify and make robust to disturbances the checks performed to determine whether the determined line of sight 9 is directed toward the flap and / or door 11 of the automobile 3, a classification object that progresses in real time accompanying the person 6 may be provided using a software tool, which, according to this embodiment, is realized by a classification circle 14 for classifying the detected line of sight 9, shown by a dotted line in Figure 1. The classification circle 14 is exemplary oriented horizontally with respect to the ground 15. Furthermore, the accompanying classification circle 14 is provided to be continuously centered with respect to the person 6. Furthermore, based on Figure 1, the classification circle 14 is divided into a first classification region 16 facing toward the flap and / or door 11 of the automobile 3 and a second classification region 17 facing away from this flap and / or door 11 of the automobile 3, i.e., it is formed of two sections. The first classification region 16 has an exemplary 90° angle between its edges. The complementary second classification region 17 has an angle of 270° between its edges. The operation of the access system 10 is carried out with the intention that when the determined line of sight 9 is within the first classification region 16 of the classification circle 14, the access system 10 will unlock and / or automatically open the flap and / or door 11, and when the determined line of sight 9 is not within the first classification region 16, i.e., within the second classification region 17 of the classification circle 14, the access system 10 will lock and / or automatically close the flap and / or door 11.

[0038] Figure 1 further shows a control field 23, indicated by a dashed line, located on one flap and / or door 11, which can be used to check a specified distance between the vehicle 3 and person 6. If the person 6 is not within the control field 23, the access system 10 is preferably locked, regardless of the determined line of sight 9, making it impossible to operate. If the person 6 is within the control field 23, the access system 10 is preferably unlocked, depending on the detected line of sight 9, making it possible to operate. This provides a certain safety feature, which ensures that the locking and unlocking, and / or automatic operation, of the vehicle 3's flap and / or door 11 is only possible when the person 6 is actually near the vehicle 3, i.e., within a specified or achievable distance to the vehicle 3.

[0039] In the preceding description, when referring to horizontal orientation, this specifically means a horizontal plane 18 formed by a front-to-back central axis 21 and a transverse axis 22 perpendicular to this axis. The horizontal plane 18 referred to is, exemplarily, oriented parallel to the ground 15.

Claims

1. The camera placement (4) of the vehicle (3) detects at least one foot (5) of a person approaching the vehicle (3) or a person (6) standing near the vehicle. The orientation (8) of at least one foot (5) detected by the camera arrangement (4) is determined. A method for operating the access system (10) of the aforementioned automobile (3), Based on the determined orientation (8), the line of sight (9) of the person (6) is determined. Using the access system (10) for locking and / or automatically operating the flaps and / or doors (11) of the vehicle (3), if the determined line of sight (9) is directed toward at least one flap and / or door (11), at least one flap and / or door (11) of the vehicle (3) is unlocked and / or automatically opened, and / or using the access system (10), if the determined line of sight (9) is not directed toward at least one flap and / or door (11), at least one flap and / or door (11) of the vehicle (3) is locked and / or automatically closed. A software tool is provided to classify the determined gaze direction (9), which consists of at least two categories that progress in real time accompanying the person (6) and in which the person (6) is continuously positioned at the center. The classification circle (14) is divided into a first classification region (16) facing the flap and / or door (11) of the automobile (3) and a second classification region (17) facing away from the flap and / or door (11) of the automobile (3). Using the access system (10), if the determined line of sight direction (9) is in the first classification area (16) of the classification circle (14), the flaps and / or doors (11) of the automobile (3) are unlocked and / or automatically opened. Using the access system (10), if the determined line of sight direction (9) is in the second classification area (17) of the classification circle (14), the flaps and / or doors (11) of the automobile (3) are locked and / or automatically closed. The software tool is provided within the arithmetic unit (24) of the automobile (3), and / or The software tool is implemented by computer-based visual or machine-visual methods and / or by algorithms for recognizing and measuring the geometric structure and motion of objects. The software tool and / or the algorithm for recognizing and measuring the geometric structure and motion of an object are trained using real data and / or The aforementioned real data includes data from subjects with various body types, different clothing, and / or fixed-angle trajectories, different ambient and weather conditions, and time and light conditions. A method characterized by the following:

2. In determining the orientation (8) of at least one foot (5) detected by the camera arrangement (4), the toe (12) is determined. The method according to claim 1, characterized by the features described above.

3. The determined line of sight direction (9) is directed parallel to the toe vector (13) that passes through the determined toe (12) of the detected foot (5). The method according to claim 2, characterized by the features described above.

4. The first classification region (16) has angles ranging from a minimum of 10° to a maximum of 180°, particularly angles of a minimum of 30° and / or a maximum of 90°. The second classification region (17) has a corresponding angle that is complementary to the angle formed by the first classification region (16), or The second classification region (17) is, when viewed in the horizontal plane (18), an angle from a minimum of 180° to a maximum of 350°, particularly an angle of a maximum of 330° and / or a minimum of 270°. The method according to claim 1, characterized by the features described above.

5. After acquiring the aforementioned actual data, the actual data is modified in the adaptation process. The acquired actual data is manually supplemented with the attributes of the person (6). The method according to claim 1, characterized by the features described above.

6. Apparatus (25) for carrying out the method described in claim 1, Camera arrangement (4) that can be mounted on or is mounted on an automobile (3), A computing unit (24) that can be mounted in or is mounted in the vehicle (3), which is configured to perform computational tasks and to control an access system (10) for locking and unlocking and / or automatically operating the flaps and / or doors (11) of the vehicle (3), and is communicatively connected to the camera arrangement (4), During execution by the arithmetic unit (24), a storage medium (26) that can communicate with the arithmetic unit (24) or can be read by a computer formed by the arithmetic unit (24) has a command that instructs the device (25) to execute the method described in claim 1. A device (25) characterized by comprising the above.

7. The camera arrangement (4) is realized by at least one omnidirectional camera (27) that can be mounted on or is mounted on the automobile (3), and / or Four omnidirectional cameras (27) are positioned on the vehicle (3). The apparatus (25) according to claim 6, characterized in that it is a device.

8. The at least one omnidirectional camera (27) has a central field of view axis that passes through the center of its field of view, which is oriented diagonally with respect to the front-to-rear central axis (21) of the vehicle (3) and toward the ground (15) on which the vehicle (3) is positioned. The apparatus (25) according to claim 7, characterized in that it is a device.

9. A motor vehicle (3) comprising the apparatus (25) according to claims 6 to 8, which is configured to carry out the method according to any one of claims 1 to 5.

Images