Method for actuating an access system of a motor vehicle, device for carrying out the method and motor vehicle having such a device
Patent Information
- Application Number
- EP2022834855
- Authority / Receiving Office
- EP · EP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2022-01-25
- Filing Date
- 2022-12-06
- Publication Date
- 2024-12-04
AI Technical Summary
Existing methods for actuating motor vehicle access systems, such as those using foot orientation, are unreliable in determining user intent, leading to erroneous actuation of flaps and doors.
A method that uses a camera arrangement to detect a person's foot and determine their viewing direction, employing a software tool to classify the direction within specific angular ranges to unlock or lock vehicle flaps and doors contactlessly, based on the user's gaze direction.
Enables reliable and contactless actuation of vehicle access systems, ensuring accurate assessment of user intent and increased comfort by allowing access without manual interaction, reducing erroneous actuations.
Smart Images

Figure 1.1
Abstract
Description
[0001] Method for operating an access system of a motor vehicle, device for carrying out the method and motor vehicle with such a device
[0002] The invention relates to a method for actuating an access system of a motor vehicle according to the preamble of claim 1. The invention also relates in particular to a device for carrying out the method and expediently to a motor vehicle with such a device.
[0003] A method of the type mentioned above is described in DE 202019 104268 U1. According to this method, the orientation of a person's feet can be used to activate an access system for locking and unlocking and / or automatically operating hatches and / or doors of a motor vehicle. The disadvantage of this is that the orientation of the feet cannot be used to reliably determine whether the person in question actually requests access to the motor vehicle, which can result in incorrect operation of the access system.
[0004] From DE 102016 108 702 A1 another method for operating an access system of a motor vehicle is known.
[0005] The object of the invention is to provide an improved or at least a different embodiment for a method for operating an access system of a motor vehicle. In particular, the aim is to provide a method in which no manual interaction with the motor vehicle is necessary for locking and unlocking and / or automatically operating flaps and / or doors of a motor vehicle.
[0006] In the present invention, this object is achieved in particular by the subject matter of the independent claims. Advantageous embodiments are the subject matter of the dependent claims and the description. The basic idea of the invention is to provide an access system for locking and unlocking and / or automatically operating flaps and / or doors of a motor vehicle depending on a determined line of sight of a person.
[0007] For this purpose, according to the invention, a method for actuating an access system of a motor vehicle is provided, in the context of which method at least one foot of a person approaching the motor vehicle or located near it is detected by means of a camera arrangement of the motor vehicle, then an orientation of the at least one foot detected by the camera arrangement is first determined by means of a software tool, and based on the determined orientation a viewing direction of the person is then determined. By means of the access system for locking and unlocking and / or automatically actuating flaps and / or doors of the motor vehicle, at least one flap and / or door of the motor vehicle is unlocked and / or automatically opened if the determined viewing direction is directed, in particular substantially or exactly, towards the respective at least one flap and / or door.Furthermore, by means of said access system, at least one flap and / or door of the motor vehicle is locked and / or automatically closed if the determined line of sight is not directed, in particular substantially or exactly, towards the respective at least one flap and / or door. The method according to the invention therefore enables the access system of the motor vehicle to be operated by the said person without contact, and thus flaps and / or doors can be unlocked or locked by simply directing their gaze towards a flap or door when approaching the motor vehicle. The person can thus gain access to the motor vehicle or to storage spaces concealed by flaps relatively quickly and conveniently. Furthermore, based on the determined line of sight, it can be reliably assessed whether the said person requests access to the motor vehicle or not, thereby virtually eliminating incorrect operation of the access system.Since the person does not have to interact actively with the motor vehicle, or practically does not have to interact actively with it, there is still advantageous time to occupy themselves with other things, so that the result is an increase in the comfort experience achieved by the motor vehicle.
[0008] The person approaching the motor vehicle is expediently a user of the motor vehicle authorized to use the motor vehicle. The authorization can be granted to the person, in particular, for a limited period of time or permanently. The person's authorization can be actively checked within the framework of the method described above, in particular by means of communication with a mobile device carried by the authorized user, which can be implemented, for example, by a so-called intelligent key (smartkey) or a mobile phone (smartphone) or a so-called CoD, i.e. a smart device owned by the user of the motor vehicle (from the English consumer owned smart device).
[0009] For the purposes of the invention, the term "foot" expediently refers to the lowermost portion of a person's leg. A foot, in particular, has a tarsus directly adjacent to the lower leg, a metatarsus adjoining the tarsus, and distal toes arranged on the metatarsus.
[0010] It is expediently provided that, as part of determining the orientation of at least one foot detected by the camera arrangement, the toe tip is determined. In this context, it can further be provided that the determined viewing direction is aligned parallel or substantially parallel to a toe tip vector running at least through the determined toe tip of the detected foot. Expediently, the toe tip vector can further run through a tarsus of the detected foot or through a lower leg of a leg of the said person. This allows the toe tip vector and the person's viewing direction to be determined relatively easily based on the determined foot.
[0011] The toes and / or metatarsus of a person's foot may conveniently define a toe within the meaning of the invention.
[0012] According to the invention, it is further provided that the software tool provides an at least two-class classification object, in particular a classification circle, which moves along with the said person in real time and is used to classify the determined line of sight. The classification object is expediently implemented by a classification circle that can be aligned horizontally, in particular with respect to a surface on which the motor vehicle is arranged. Furthermore, it can be provided that the said person is continuously positioned centrally with respect to the moving classification circle. The classification object or the classification circle thus implements a software-generated tool for the rapid and reliable classification of the determined line of sight of the person in real time.According to the invention, it is further provided that the classification object, in particular the classification circle, is divided into a first class area facing a flap and / or a door of the motor vehicle and a second class area facing away from this flap and / or this door of the motor vehicle, wherein this flap and / or this door of the motor vehicle is unlocked and / or automatically opened by means of the access system if the determined viewing direction lies, in particular substantially or exactly, within the first class area of the classification circle, and wherein this flap and / or this door of the motor vehicle is locked and / or automatically closed by means of the access system if the determined viewing direction lies within the second class area of the classification circle.In other words, the determined viewing directions lying within the first class range of the classification object, in particular the classification circle, constitute releases for unlocking and / or automatically opening the respective flap or door of the motor vehicle, while the determined viewing directions not lying within the first class range of the classification circle constitute releases for locking and / or automatically closing the respective flap or door of the motor vehicle. The first class range could therefore be referred to as the release range, and the second class range as the blocking range.
[0013] The method can expediently be characterized in that the first class area, in particular viewed in a horizontal plane, spans an angle of at least 10° to a maximum of 180°, in particular an angle of at least 30° and / or an angle of a maximum of 90°. Furthermore, the second class area, likewise expediently viewed in a horizontal plane, can span an opposite angle that is complementary to the angle spanned by the first class area. Alternatively, the second class area, viewed in a horizontal plane, can span an angle of at least 180° to a maximum of 350°, in particular an angle of at least 330° and / or an angle of a maximum of 270°. In other words, the first class area spans an angle of at least 10° to a maximum of 180°, in particular an angle of at least 30° and / or a maximum of 90°, in a horizontal plane between its opposite area edges.The second class range can either be designed to complement this angle or span a predetermined angle of at least 180° to a maximum of 350°, in particular an angle of at least 330° and / or an angle of a maximum of 270°. In practice, due to the relatively large angular widths of the first or second class range, it can be determined relatively reliably and quickly whether the determined viewing direction lies within the first or second class range. A classification object designed accordingly, in particular a classification circle, can therefore ensure a method that is relatively robust against disturbances in the detection of the foot and uncertainties in determining the orientation.
[0014] Said horizontal plane can expediently be defined by a central longitudinal axis of the motor vehicle and a transverse axis perpendicular to the same. Said horizontal plane can expediently be aligned parallel to a surface on which said motor vehicle is arranged.
[0015] It goes without saying that the classification object, in particular the classification circle, can be provided with more than two class areas, for example, a third class area separating the first class area from the second class area can be provided, wherein, if the determined viewing direction lies within the third class area, further motor vehicle functions such as comfort functions can be activated automatically.
[0016] Furthermore, it can be provided that, by means of the software tool, a control field is provided for each flap and / or door of the motor vehicle, in particular directly adjacent to the respective flap and / or door, in particular with a two-dimensional planar design and furthermore in particular horizontally aligned, wherein the access system is expediently placed in a locked state in which no actuation of the access system is possible regardless of the determined viewing direction if the said person is not located within the control field, and wherein the access system is expediently placed in an enabled state in which actuation of the access system is possible depending on the determined viewing direction if the said person is located within the control field.This provides a security function that allows the locking and unlocking and / or automatic operation of the vehicle's flaps and / or doors only when the person in question is actually present at the vehicle.
[0017] Furthermore, it can be expediently provided that the said software tool is provided in a computing unit of the motor vehicle. In this case, the said software tool can preferably be implemented using methods of computer vision or machine vision and / or algorithms for recognizing and measuring geometric structures of objects and their movements. The software tool is thus implemented on the said motor vehicle, so that the proposed method can be executed entirely on or by the motor vehicle. In principle, however, it is also possible to outsource the software tool externally and, for example, implement it on a stationary server, in which case communication can then take place, for example, via radio communication between the external device and the motor vehicle in order to execute the proposed method.Furthermore, the proposed methods allow the image data captured by the camera arrangement to be analyzed relatively efficiently, so that at least one foot and / or the tip of a person approaching or standing near the motor vehicle can be detected relatively quickly and, in particular, in real time. Computer-based vision can also expediently be used to determine a person's posture and / or position.
[0018] Furthermore, the method can be characterized by the fact that the said software tool and / or the algorithms for recognizing and measuring geometric structures of objects as well as their movements are trained using so-called real ground data and / or machine learning methods. The said real ground data can be acquired, for example, during learning runs of the method on the motor vehicle conducted by the manufacturer. In particular, machine learning methods such as "deep learning" can be used. The so-called real ground data expediently represents data acquired, for example, during learning runs of the method on the motor vehicle conducted by the manufacturer. The learning runs carried out expediently cover the broadest possible spectrum of conceivable practical scenarios.In particular, learning runs can be conducted for all flaps and / or doors of the vehicle, working with test subjects with varying body types, different clothing, and / or trajectories, as well as simulating different environmental and weather conditions, as well as time and lighting conditions. The collected real-ground data can therefore help train the software tool and / or the aforementioned algorithms and make them relatively robust.
[0019] According to the said method, it is also expediently provided that the acquired real-ground data can be revised as part of an adaptation process, whereby the acquired real-ground data can optionally be manually supplemented with attributes of the said person. Such attributes can be formed, for example, by bounding boxes, lane IDs, or body postures. For example, after the adaptation process has been carried out, the software tool and / or the said algorithm can be trained.
[0020] The software tool and / or said algorithm may conveniently provide data including the location of said person, their direction of movement, their speed of movement and their orientation.
[0021] A further basic idea of the invention is to provide a device that can be mounted in particular on a motor vehicle for carrying out the method described above.This device, designed to carry out the method described above, comprises a camera arrangement that is mountable or mounted on a motor vehicle, a computing unit that is mountable or mounted on the motor vehicle and configured to carry out computing operations and to control an access system for locking and unlocking and / or automatically actuating flaps and / or doors of the motor vehicle, which computing unit is communicatively connected to the camera arrangement, and a computer-readable storage medium that communicates with the computing unit or is formed by the computing unit, the latter comprising, in particular, instructions that, when executed by the computing unit, cause the proposed device to carry out said method in accordance with the above description. This provides a favorable embodiment for a device for carrying out the method described above.
[0022] In this context, it may be expedient for the camera arrangement to be implemented by at least one surround-view camera that is mountable or mounted on the motor vehicle. For example, this could involve exactly four surround-view cameras mounted on the motor vehicle. Such a surround-view camera can expediently provide a 360° view around the motor vehicle. A surround-view camera can also be formed, in particular, by a so-called SVC camera or by a surround-view parking camera configured to perform parking assistance functions. These surround-view parking cameras, or SVC cameras (from the English "surround vision cameras," aka SVC), can also expediently provide a 360° view around the motor vehicle.Furthermore, four all-round view cameras are expediently arranged on the motor vehicle, in particular one in the area of a front bumper, one further in the area of a rear bumper, and one each in the area of an A-pillar of the motor vehicle or a side mirror of the motor vehicle. In particular, it can be provided that at least one all-round view camera is aligned such that a central axis of view passing through its field of view is obliquely aligned with respect to a central longitudinal axis of the motor vehicle and in the direction of a ground on which the motor vehicle is arranged. In other words, the at least one all-round view camera is expediently aligned such that its field of view points obliquely downwards towards the ground.This allows the at least one surround-view camera to detect, in particular, ground-level obstacles located near the motor vehicle, as well as the feet and / or legs of a person on the motor vehicle or moving toward it. The invention defines the term "field of view" as the three-dimensional area of an optical device within which it can detect bodies.
[0023] A further basic idea of the invention is to provide a motor vehicle that can carry out the method described above. For this purpose, a motor vehicle is provided that is equipped with a device according to the preceding description, which is configured to carry out the method according to the preceding description. This provides a preferred embodiment for a motor vehicle for carrying out the method described above.
[0024] In summary, the invention expediently relates to a method for operating an access system of a motor vehicle, wherein at least one foot of a person is detected by means of a camera arrangement, an orientation of the foot is determined by means of a software tool, and a viewing direction of the person is then determined based on the determined orientation. If the viewing direction is directed towards at least one flap and / or door of the motor vehicle, at least one flap and / or door is unlocked and / or automatically opened by means of the access system. If, on the other hand, the viewing direction is not directed towards at least one flap and / or door, the access system locks and / or doors and / or automatically closes them. The invention relates in particular to a device for carrying out said method and expediently to a motor vehicle having such a device.Further important features and advantages of the invention emerge from the subclaims, from the drawings and from the associated description of the figures based on the drawings.
[0025] It is understood that the features mentioned above and those to be explained below can be used not only in the combination specified in each case, but also in other combinations or on their own, without departing from the scope of the present invention.
[0026] A preferred embodiment of the invention is illustrated in the drawing and explained in more detail in the following description, wherein the same reference numerals refer to the same or similar or functionally identical components.
[0027] It shows schematically:
[0028] Fig. 1 shows a simplified plan view of a motor vehicle with a device for carrying out the method according to the invention.
[0029] Fig. 1 shows a plan view of a motor vehicle, designated overall by reference numeral 3. It has flaps and doors 11 that can be actuated by means of an access system 10 integrated in the motor vehicle 3 for locking and unlocking and / or automatically actuating the flaps and doors 11. This means that the flaps and doors 11 can be unlocked and / or automatically opened, or locked and / or automatically closed, using the access system 10 in the usual manner.
[0030] In order to facilitate access to the motor vehicle 3 for a person 6 authorized to use the motor vehicle 3, who is indicated here by a small box, in particular to make this access relatively quick, convenient and in particular without the need to touch the motor vehicle 3 manually, the motor vehicle 3 is equipped, by way of example, with a device 25 designed to carry out the method according to the invention. This device has, purely by way of example, four separate panoramic view cameras 27 of a camera arrangement 4, indicated by small triangles, a computing unit 24 installed in the motor vehicle 3, indicated by a small box, and a computer-readable storage medium 26 integrated into the computing unit 24. The four panoramic view cameras 27 together, by way of example, realize a 360° panoramic view around the motor vehicle 3, so that they are also suitable in principle for carrying out parking assistance functions of the motor vehicle 3.By way of example, one all-round view camera 27 is arranged in the area of a front bumper of the motor vehicle 3, another all-round view camera 27 is arranged in the area of a rear bumper of the motor vehicle 3, and one all-round view camera 27 is each arranged in the area of an A-pillar of the motor vehicle 3. It can expediently be provided that the all-round view cameras 27 are aligned such that their field of view center axes, which centrally penetrate the respective field of view, are oriented obliquely with respect to a central longitudinal axis 21 of the motor vehicle 3 and in the direction of a ground surface 15 on which the motor vehicle 3 is arranged. This results in a camera view obliquely downwards onto the ground surface 15, so that they can detect, in particular, ground-side obstacles located in the vicinity of the motor vehicle 3, as well as feet 5 and / or legs of the person 6 located at the motor vehicle 3 or moving towards it.
[0031] The aforementioned computing unit 24 is configured to perform computing operations and control the access system 10. For this purpose, it is communicatively connected to the panoramic cameras 27 of the camera array 4 via communication lines indicated by lines in Fig. 1. The aforementioned storage medium 26 is integrated into the computing unit 24 and comprises commands executable by the computing unit 24, which, when executed by the computing unit 24, cause the device 25 to execute the method according to the invention.
[0032] According to the method according to the invention, it is provided by way of example that at least one toe tip 12 of a foot 5 of a person 6 approaching the motor vehicle 3 or located near it is detected by means of the all-round view cameras 27 of the camera arrangement 4, wherein an orientation 8 of the at least one foot 5 or the toe tip 12 is first determined by means of a software tool (not illustrated), and a viewing direction 9 of the person 6 is then derived based on the determined orientation 8. By way of example, one can base this on the consideration that the desired viewing direction 9 of the person 6 to be determined can be aligned parallel or at least substantially parallel to a toe tip vector 13 running at least through the determined toe tip 12. Subsequently, a check is carried out to determine whether the determined viewing direction 9 is directed towards a flap and / or door 11.If this is the case, at least this flap and / or door 11 of the motor vehicle 3 is unlocked and / or automatically opened by means of the access system 10. If, however, the determined viewing direction 9 is not directed towards this flap and / or door 11, at least this one flap and / or door 11 is locked and / or automatically closed by means of the access system 10. The method according to the invention thus enables the access system 10 to be operated contactlessly by the said person 6, so that flaps and / or doors 11 can be unlocked or locked by the person 6 simply directing their gaze towards a flap or door 11.
[0033] In order to simplify the check to be carried out as to whether the determined viewing direction 9 is directed towards a flap and / or door 11 of the motor vehicle 3 and to make it robust against disturbances, it can be provided that a classification object is provided by means of the software tool which moves along with the said person 6 in real time, which according to the present embodiment is implemented by way of example by a classification circle 14, shown in Fig. 1 by means of a dotted line, for classifying the determined viewing direction 9. The classification circle 14 is, for example, aligned horizontally with respect to the ground 15. Furthermore, it is provided that the moving classification circle 14 is arranged continuously centrally with respect to the person 6.1, it can further be seen that the classification circle 14 is designed as a two-class area, being divided into a first class area 16 facing a flap and / or a door 11 of the motor vehicle 3, and a second class area 17 facing away from this flap and / or this door 11 of the motor vehicle 3. The first class area 16, for example, spans an angle of 90° between its area edges. The complementarily designed second class area 17 describes an angle of 270° between its area edges.The access system 10 is now actuated in the sense that a flap and / or door 11 is unlocked and / or automatically opened by means of the access system 10 if the determined viewing direction 9 lies within the first class range 16 of the classification circle 14, and that this flap and / or door 11 is locked and / or automatically closed by means of the access system 10 if the determined viewing direction 9 does not lie within the first class range 16, i.e. in particular within the second class range 17 of the classification circle 14.
[0034] In Fig. 1, a control field 23 is furthermore assigned to a flap and / or door 11 and drawn with a dashed line, by means of which a predetermined distance between the motor vehicle 3 and a person 6 can be checked by expediently setting the access system 10 in a locked state, in which no actuation of the access system 10 is possible regardless of the determined viewing direction 9, if the said person 6 is not located within the control field 23, and wherein the access system 10 is expediently set in an enabled state, in which actuation of the access system 10 is possible depending on the determined viewing direction 9, if the said person 6 is located within the control field 23.This provides a type of security function by means of which the locking and unlocking and / or automatic operation of flaps and / or doors 11 of the motor vehicle 3 is only possible if the said person 6 is actually located at the motor vehicle 3 - namely at a predetermined or predeterminable distance from the motor vehicle 3.
[0035] As far as a horizontal alignment was mentioned above, this refers in particular to a horizontal plane 18, which is expediently spanned by the central longitudinal axis 21 and a transverse axis 22 perpendicular to the same. Said horizontal plane 18 is, for example, aligned parallel to the substrate 15.
Claims
Patent claims Method for actuating an access system (10) of a motor vehicle (3), within the framework of which - by means of a camera arrangement (4) of the motor vehicle (3) at least one foot (5) of a person (6) approaching or located on the motor vehicle (3) is detected, - wherein an orientation (8) of the at least one foot (5) detected by the camera arrangement (4) is determined by means of a software tool, characterized in that - based on the determined orientation (8) a direction of gaze (9) of the person (6) is determined, - that by means of an access system (10) for locking and unlocking and / or automatically operating flaps and / or doors (11) of the motor vehicle (3), at least one flap and / or door (11) of the motor vehicle (3) is unlocked and / or automatically opened when the determined direction of view (9) is directed towards the respective at least one flap and / or door (11), and / or - that at least one flap and / or door (11) of the motor vehicle (3) is locked and / or automatically closed by means of the access system (10) if the determined direction of view (9) is not directed towards the respective at least one flap and / or door (11), - that by means of the software tool a classification object, at least two-class, in particular a classification circle (14), which runs in real time with the said person (6), is provided for the classification of the determined gaze direction (9), - the classification object, in particular the classification circle (14), is divided into a first class area (16) facing a flap and / or a door (11) of the motor vehicle (3) and a second class area (17) facing away from this flap and / or this door (11) of the motor vehicle (3), - that by means of the access system (10) this flap and / or this door (11) of the motor vehicle (3) is unlocked and / or automatically opened when the determined direction of view (9) lies within the first class area (16) of the classification circle (14), - that by means of the access system (10) this flap and / or this door (11) of the motor vehicle (3) is locked and / or automatically closed when the determined direction of view (9) lies within the second class area (17) of the classification circle (14). Method according to claim 1, characterized in that - within the framework of determining the orientation (8) of the at least one foot (5) detected by the camera arrangement (4), the toe (12) of which is determined. Method according to claim 2, characterized in that - the determined gaze direction (9) is aligned parallel to a toe vector (13) running at least through the determined toe (12) of the detected foot (5). Method according to one of the preceding claims, characterized in that - the first class area (16) spans an angle of at least 10° to a maximum of 180°, in particular an angle of at least 30° and / or an angle of a maximum of 90°, and / or - the second class area (17) spans an opposite angle which is complementary to the angle spanned by the first class area (16), or - the second class area (17) in a horizontal plane (18) spans an angle of at least 180° to a maximum of 350°, in particular an angle of at least 330° and / or an angle of a maximum of 270°. Method according to one of the preceding claims, characterized in that - the said software tool is provided in a computing unit (24) of the motor vehicle (3), and / or - the said software tool through methods of computer-based vision or machine vision and / or through recognition algorithms and The method is used to measure the geometric structures of objects and their movements. The method according to claim 5 is characterized in that... - the aforementioned software tool and / or the algorithms for recognizing and measuring the geometric structures of objects and their movements are trained using real-world ground data and / or machine learning methods, and / or - the real-world ground data are determined during learning runs of the method on the motor vehicle (3). Method according to claim 6, characterized in that - after the actual soil data has been recorded, the actual soil data will be revised as part of an adjustment process, whereby - optionally, the recorded real-world ground data can be manually supplemented with attributes of the said person (6). Device (25) for carrying out the method according to one of the preceding claims, characterized by, - a camera arrangement (4) that can be mounted or is mounted on a motor vehicle (3), - a computing unit (24) equipped for performing calculations and controlling an access system (10) for locking and unlocking and / or automatically operating flaps and / or doors (11) of the motor vehicle (3), which can be mounted or is mounted on the motor vehicle (3) and which is communicatively connected to the camera arrangement (4), as well as - a computer-readable storage medium (26) communicating with or formed by the computing unit (24), which comprises instructions that, when executed by the computing unit (24), cause the device (25) to execute the method according to the preceding claims. Device (25) according to claim 8, characterized in that - the camera arrangement (4) is realized by at least one all-round view camera (27) that can be mounted on or is mounted on the motor vehicle (3), and / or - four surround-view cameras (27) are arranged on the motor vehicle (3). Device (25) according to claim 9, characterized in that - at least one all-round view camera (27) is oriented such that a central axis of the field of view, which passes centrally through its field of view, is inclined with respect to a central longitudinal axis (21) of the motor vehicle (3) and is oriented towards a surface (15) on which the motor vehicle (3) is arranged. Motor vehicle (3) with a device (25) according to claims 8 to 10, which is configured to carry out the method according to one of claims 1 to 7.
