METHOD AND DEVICE FOR DETERMINING POSITION INFORMATION, VEHICLE AND COMPUTER PROGRAM PRODUCT

Abstract

A method for determining position information of an object that has at least one surface reflective to electromagnetic radiation and is located in a vehicle comprises the following: Detection of electromagnetic radiation reflected from at least one reflective surface of the object, originating from one or more vehicle-mounted sources of electromagnetic radiation, by at least one sensor, in particular at least one camera; Determining, recording, or retrieving construction details of the object; and Determining position information of the object from the reflected electromagnetic radiation using the object's construction details and position information from one or more vehicle-mounted sources of electromagnetic radiation and at least one sensor.

Description

[0001] The present invention relates to a method and a device for determining positional information of an object located in a vehicle. The invention further extends to a vehicle with such a device, as well as a corresponding computer program or computer program product.

[0002] In the prior art, (road) vehicles are already known in which one or more cameras are installed to capture images of the vehicle's interior. Such cameras can be used, for example, to determine the position of a vehicle occupant's eyes, particularly that of the driver. Infrared radiation (IR radiation), which is invisible to the human eye, can be used for this purpose.

[0003] CN 103886307 A discloses a method for determining the position of a driver's pupil and their gaze direction for fatigue detection. Other methods and devices for determining the position of objects that are not part of the human body are also known in the prior art. In prior art methods and devices known to the inventors, the position of objects in a plane perpendicular to a line extending through the object and the camera can be determined with relatively high accuracy.

[0004] The inventors have recognized that existing methods or devices for determining the distance of an object from a camera installed in a vehicle are often not possible or may deliver relatively inaccurate results.

[0005] Against this background, it is a task of the present revelation to at least mitigate the problems discussed above.

[0006] A solution to this problem is achieved according to the teaching of the independent claims. Various embodiments and further developments of the solution are the subject of the dependent claims.

[0007] A first aspect of the present disclosure relates to a (computer-implemented) method for determining position information of an object that has at least one surface reflective to electromagnetic radiation and that is located in a vehicle, wherein the method comprises: Detection of electromagnetic radiation reflected from at least one reflective surface of the object, originating from one or more vehicle-mounted sources of electromagnetic radiation, by at least one sensor, in particular at least one camera;

[0008] Determining, recording, or retrieving construction details of the object; and

[0009] Determining position information of the object from the reflected electromagnetic radiation using the object's construction details and position information from one or more vehicle-mounted sources of electromagnetic radiation and at least one sensor.

[0010] The first aspect of the present disclosure is based on the following considerations of the inventors.

[0011] For some objects that may be used in a vehicle, it is desirable or even necessary to determine the object's position relatively precisely, for example, to locate it with millimeter accuracy, not only with respect to its position on a plane perpendicular to a line extending through the object and the camera (or other sensor) (this is also referred to here as the xy-position), but also with respect to the distance between the object and the camera, i.e., along such a line (this is also referred to here as the z-position). In other words, the inventors have recognized that the position of such an object often needs to be determined in three dimensions (relative to a vehicle-fixed coordinate system).

[0012] One example of this is AR glasses (augmented reality glasses). Their position relative to the vehicle often needs to be determined (within millimeters) so that vehicle-mounted content can be displayed correctly in the glasses. Other examples are explained below.

[0013] The reflective surface can be an (external) surface of the object. However, it can also be an internal surface, for example, an internal surface coated with a non-reflective layer that is (essentially) transparent to electromagnetic radiation.

[0014] For the purposes of this disclosure, a "reflective surface" is not only to be understood as a surface that reflects (essentially) 100% of the electromagnetic radiation, but also as a surface that is partially reflective, i.e., reflects a smaller percentage of the electromagnetic radiation, for example, up to 90, 80, 70, 60, 50, 40, 30, or 20% of the electromagnetic radiation. In principle, a percentage sufficient for reliable detection is adequate.

[0015] The electromagnetic radiation can be located outside the visible range and can be, in particular, infrared radiation.

[0016] The object's design details can include information about its shape or reflective surface. This information may be in the form of CAD (Computer-Aided Design) data, for example. Using these design details, the object's positional information can be determined, as described in more detail below.

[0017] For the purposes of this disclosure, "position information" means information that specifies the position of the object relative to a point fixed to the vehicle. This position information may include information relating to one, two, or three dimensions. In particular, the position information may specify at least a distance from a point fixed to the vehicle, for example, from the at least one sensor or from the at least one source of electromagnetic radiation.

[0018] The following describes various exemplary embodiments of the method, which, unless expressly excluded or technically impossible, can be combined with each other as desired and with the second, third, fourth and / or fifth aspects of the present disclosure described below.

[0019] The object can have at least two reflective surfaces that are not coplanar.

[0020] Both of these surfaces can be used to determine the object's position. Position information can be determined separately for each surface, and the results can be compared if necessary. This can increase the accuracy of the determination. Alternatively, or in addition, the object's position can be determined using both surfaces simultaneously. In particular, when the latter is done, the accuracy of the determination can be further improved.

[0021] The reflective surfaces can be flat.

[0022] At least one reflective surface can also exhibit curvature. This curvature can be in only one dimension, as would be the case with the surface of a cylinder with a circular cross-section. However, it can also be in two dimensions, as would be the case with the surface of a sphere. Furthermore, different areas of the reflective surface can have different curvatures. Information about the curvature(s) can be found in the design details.

[0023] Each of the one or more vehicle-mounted sources of electromagnetic radiation can be essentially a point source of electromagnetic radiation. These sources of electromagnetic radiation do not necessarily have to be sources of structured light (or equivalent sources outside the visible spectrum).

[0024] According to the disclosure, it is sufficient if the sum of the number of vehicle-mounted sources of electromagnetic radiation, of reflective surfaces and of sensors is at least four.

[0025] For example, two (or more) sources of electromagnetic radiation can be used in combination with at least one reflective surface (flat or curved) and at least one sensor. Likewise, it would be possible to use at least one source of electromagnetic radiation in combination with at least two reflective surfaces (flat or curved) and at least one sensor. Similarly, it would be possible to use at least one source of electromagnetic radiation in combination with at least one reflective surface (flat or curved) and at least two sensors.

[0026] In this context, a concrete example will be explained. When using two sources of electromagnetic radiation (at different, vehicle-fixed positions) in combination with a reflective surface and a sensor, the sensor, for example a pixel-based camera, can detect the electromagnetic radiation reflected from the two sources by the reflective surface. The sensor determines the positions of the reflected radiation on the sensor surface. If the reflective surface is farther away from the sensor and the sources of electromagnetic radiation, the positions of the reflected radiation on the sensor surface will be closer together than if the reflective surface is positioned closer to the sensor and the sources of electromagnetic radiation.From the positions of the reflected radiation on the sensor surface and taking into account the positions of the sources of electromagnetic radiation as well as the construction details of the object, the distance between the reflecting surface and the sensor can be deduced.

[0027] The same applies when using a) (only) one source of electromagnetic radiation and two non-coplanar reflecting surfaces, and (only) one sensor, or b) (only) one source of electromagnetic radiation and (only) one reflecting surface, and two sensors at different positions.

[0028] The disclosed method can even be applied using (only) one source of electromagnetic radiation, (only) one reflective surface, and (only) one sensor. However, for this to be possible, the reflective surface would need to have a suitable curvature, in particular, it would need to be concave at least in certain sections (on the side facing the source of electromagnetic radiation and the sensor).

[0029] The object could, for example, be AR glasses, headphones, or a smart device, especially a smartphone or smartwatch.

[0030] For the purposes of this disclosure, the term "headphones" also includes, for example, an earphone. In general, the disclosed method may be suitable for any object that has a surface reflective to the electromagnetic radiation to be used.

[0031] Specifically, the disclosed method can first determine the type of object (if the object is not already known). Some of the objects mentioned above have characteristics that can be detected, for example, by a camera. Based on these characteristics, a processor, possibly with the aid of external resources (with which data can be exchanged, for example, via the internet or a proprietary network, especially wirelessly), can determine which object it is. These characteristics might include, for example, a specific shape of a section of the object or an encoding (barcode, QR code, or the like).

[0032] The object may also be designed for wireless communication, allowing the type of object to be determined in this way.

[0033] Once the type of object is known, its construction details can be retrieved from a memory or an external resource so that they can be used in the disclosed procedure.

[0034] The object's position information can include the object's distance from at least one sensor used to detect the electromagnetic radiation reflected from the object's reflective surface.

[0035] Additionally or alternatively, the object's position information can include the object's distance from at least one source of electromagnetic radiation.

[0036] In both cases, the z-position could thus be determined. If necessary, the xy-position could also be determined, possibly using the same sensor (camera), as is already known in principle. In summary, the disclosed method allows the object's position in three dimensions within the three-dimensional space of a vehicle-fixed coordinate system to be determined. If necessary, polar coordinates (for example, relative to the sensor's position) could first be determined and then transformed into Cartesian coordinates.

[0037] The method can include determining a spatial relationship, in particular a distance, especially an angular distance between at least two of the reflections, in order to determine the position information of the object using the determined spatial relationship.

[0038] As mentioned previously, in the disclosed method, a smaller distance between two reflections indicates a greater distance between the object and the sensor or source of electromagnetic radiation. Conversely, a larger distance between two reflections indicates a smaller distance between the object and the sensor or source of electromagnetic radiation. Accordingly, the angle at which the reflections are detected on the sensor can be determined from the distance between the reflections on the sensor surface. From this, the distance between the sensor and the object can then be calculated.

[0039] Triangulation can be used to determine the object's position information. This involves taking into account the position of at least one sensor and at least one source of electromagnetic radiation. The corresponding calculations that can be used within the framework of triangulation are known in principle and are therefore not explained in detail here.

[0040] The procedure may also include the determination of orientation information that indicates a spatial orientation of the object, whereby the orientation information is additionally taken into account when determining the object's position information.

[0041] Several implementations are possible for determining the orientation information or the spatial orientation of the object, which can also be used in parallel to make the determination of the orientation even more precise.

[0042] One of these implementations can be used when the object is equipped with a device that allows it to determine its own orientation. Such a device could, for example, use a positioning system, such as a satellite-based one, to determine the object's orientation. The orientation determined in this way would be an orientation within a global reference system. This orientation information can then be transmitted to the vehicle.The vehicle – if it is itself equipped with a suitable device for determining the orientation of the vehicle in the global reference system – can then determine the orientation of the object relative to the vehicle by comparing the orientation information transmitted by the object (concerning the spatial orientation of the object in the global reference system) with the orientation information determined by the vehicle (concerning the spatial orientation of the vehicle in the global reference system).

[0043] In a second implementation, which can also be used when the object is not equipped with its own orientation-determining device, the vehicle determines the object's orientation based on the image information captured by the sensor (e.g., camera). It has already been noted that methods and devices are known in the art for determining the position of objects in a plane perpendicular to a line extending through the object and the camera with relatively high accuracy. If, as previously described, the object's design details are known, these details can be compared with the image information of the object captured by the sensor to determine the object's orientation relative to the vehicle.Here's a simplified example: Let's assume the object is a thin, rectangular disk. If a main surface of this rectangular disk is oriented precisely towards the sensor, opposite edges of the rectangle appear to be exactly the same length from the sensor's perspective. The sensor can detect this and conclude that a main surface of the rectangular disk is oriented precisely towards the sensor. If the disk is now tilted, opposite edges of the object no longer appear to be the same length from the sensor's perspective. It makes a difference whether the object (rectangle) is tilted around only one axis parallel to an edge of the rectangle, or around two such axes (perpendicular to each other) (or around an axis that is not parallel to an edge of the rectangle).When tilted around exactly one axis parallel to an edge of the rectangle, one pair of opposite edges appears to the sensor to be of equal length (but no longer parallel to each other), while the other pair of opposite edges no longer appears to be of equal length (but remains parallel to each other). When tilted around two axes parallel to an edge of the rectangle (or around one axis not parallel to an edge of the rectangle), opposite edges of both pairs generally do not appear to be of equal length to the sensor. The angles between adjacent edges also change due to the tilting, and in the two examples described, in different ways. This, too, can be detected by the sensor. By comparing the edge lengths and / or the angles between adjacent edges—as they appear to the sensor—the sensor (or the computer) can determine the direction of the tilt.a processor connected to the sensor) determines the orientation of the object relative to the sensor and thus in a vehicle-fixed reference system.

[0044] For a different object that is not in the shape of a flat, rectangular disc, determining the object's orientation may be more complex, but is possible, especially if the object's construction details are known.

[0045] A second aspect of the present disclosure relates to a device for use in a vehicle, wherein the device is provided for determining position information of an object which has at least one surface reflective to electromagnetic radiation and which is located in a vehicle, wherein the device comprises: one or more vehicle-mounted sources of electromagnetic radiation; at least one sensor for detecting electromagnetic radiation reflected from the at least one reflective surface of the object, originating from one or more vehicle-mounted sources of electromagnetic radiation; a communication interface and / or storage for determining, capturing, or retrieving design details of the object; and a processor configured to determine position information of the object from the reflected electromagnetic radiation using the design details of the object and position information from one or more vehicle-mounted sources of electromagnetic radiation and at least one sensor.

[0046] Most of the features of the second aspect of the present disclosure have already been explained above. As mentioned, the communication interface can be an interface for wireless communication between the disclosed device and the object. However, the (or another) communication interface can also serve for communication with other resources, particularly those outside the vehicle. For this purpose, the device can, for example, send a query to an external database via the mobile network. As part of this query, the device can transmit the (determined) type of object to the external resource and receive the corresponding design details of the object from the external resource in response.

[0047] A device according to the second aspect may be intended for installation in a vehicle.

[0048] In principle, all vehicle-mounted sources of electromagnetic radiation can be used within the scope of the present disclosure, as long as they leave a characteristic and known pattern (or characteristic and known patterns) visible in an image content detected by the sensor (or sensors) due to their position.

[0049] At least one source of electromagnetic radiation could emit it continuously. However, this is usually unnecessary. Instead, the electromagnetic radiation could be emitted at (regular) intervals. This could be implemented as follows: The at least one source of electromagnetic radiation emits it upon receiving a trigger signal, for example, during a defined time interval. The sensor is triggered accordingly, meaning it detects reflections of the electromagnetic radiation from that point onward and within the defined time interval. At other times outside the defined time interval, the sensor does not need to detect reflections.This can potentially increase accuracy, as electromagnetic radiation that may originate from other sources outside the specified time interval and could distort the results is not taken into account.

[0050] A third aspect of the present disclosure relates to a vehicle, in particular a motor vehicle, comprising a device according to the second aspect.

[0051] A fourth aspect of the present disclosure relates to a computer program which contains instructions which, when executed by a device according to the second aspect, cause the device to carry out the steps of a process according to the first aspect.

[0052] A fifth aspect of the present disclosure relates to a computer program product that has a computer-readable medium on which a computer program according to the fourth aspect is stored.

[0053] A previously described computer program or computer program product may optionally be used in conjunction with a device according to the second aspect of the present disclosure or in a vehicle according to the third aspect of the present disclosure.

[0054] The computer program can be stored, in particular, on a non-volatile data carrier. Preferably, this is a data carrier in the form of an optical data carrier or a flash memory module. This can be advantageous if the computer program itself is to be handled independently of a processor platform on which the one or more programs are to be executed. In another implementation, the computer program can exist as a file on a data processing unit, in particular on a server, and be downloadable via a data connection, for example, the Internet or a dedicated data connection, such as a proprietary or local network. Furthermore, the computer program can comprise a plurality of interacting individual program modules. The modules can, in particular, be configured, or at least be usable, in such a way that they function in the sense of distributed computing (i.e., distributed computing)."Distributed computing" is performed on different devices (computers or processor units) that are geographically separated and connected via a data network.

[0055] The features and advantages described in relation to the first aspect of the revelation and its advantageous design also apply, at least where technically appropriate, to the other aspects of the revelation and their advantageous design, and vice versa.

[0056] Any terms used herein, such as "comprises," "includes," "features," "has," "with," or any other variant thereof, are intended to cover non-exclusive inclusion. For example, a method or apparatus that includes or features a list of elements is not necessarily limited to those elements but may include other elements not expressly listed or inherent in such method or apparatus.

[0057] Furthermore, unless explicitly stated otherwise, "or" refers to an inclusive "or" and not an exclusive "or". For example, a condition A or B is satisfied by each of the following: A is true (or present) and B is false (or not present), A is false (or not present) and B is true (or present), and both A and B are true (or present).

[0058] The terms "ein" or "eine," as used here, are defined as "one or more" or "at least one." The terms "ein anderer" and "ein Weitere," as well as any other variant thereof, are to be understood as "at least one more."

[0059] The term "plural" as used here, as well as any other variant thereof, is to be understood in the sense of "two or more".

[0060] The term "connected" and any variant thereof is to be understood in the sense of a connection for data transmission or transmission of electrical and / or optical signals - at least when the context does not indicate that it is a purely mechanical connection.

[0061] The terms "configured" or "set up" to perform a specific function (and their respective variations) are used here to mean that the device in question is already in a configuration or setting in which it can perform the function, or at least that it is adjustable—i.e., configurable—so that it can perform the function after appropriate adjustment. Configuration can be achieved, for example, by adjusting parameters of a process sequence or by using switches or similar devices to activate or deactivate functionalities or settings. In particular, the device can have several predefined configurations or operating modes, allowing configuration to be performed by selecting one of these configurations or operating modes.

[0062] Further features, advantages and applications of the revelation will be revealed in the following description in connection with the figures, in which the same reference signs are consistently used for the same or corresponding elements of the revelation.

[0063] For the sake of clarity, the figures are at least partially schematic or greatly simplified.

[0064] In the drawing: Fig. Figure 1 schematically shows an interior view of a motor vehicle according to one embodiment of the present disclosure. Fig. Figure 2 schematically shows a possible beam path according to one embodiment of the present disclosure. Fig. Figure 3 schematically shows an image content captured by a camera according to one embodiment of the present disclosure. Fig. Figure 4 schematically shows an image content captured by a camera according to one embodiment of the present disclosure. Fig. Figure 5 schematically shows a possible beam path according to one embodiment of the present disclosure. Fig. Figure 6 schematically shows a possible beam path according to one embodiment of the present disclosure. Fig. Figure 7 schematically shows an image content captured by a camera according to one embodiment of the present disclosure. Fig. Figure 8 schematically shows a possible beam path according to one embodiment of the present disclosure. Fig. Figure 9 schematically shows an image content captured by a camera according to one embodiment of the present disclosure. Fig. Figure 10 shows a flowchart to illustrate a procedure according to one embodiment of the present disclosure.

[0065] Fig. Figure 1 schematically shows an interior view of a motor vehicle 1 according to one embodiment of the present disclosure. The in Fig. The view shown in Figure 1 corresponds approximately to the view from a central position within the vehicle 1 looking forward through the windshield 12 of the vehicle 1. The vehicle 1 has a rearview mirror 11, as is usual, which is mounted on or near the vehicle's headliner. The vehicle 1 also has an on-board computer 8 or processor 8. This can, for example, be integrated into the dashboard 13 in such a way that it is not visible from the vehicle interior. A person 4 (passenger 4) is seated in the right front seat. A steering wheel (without a reference symbol) is indicated in the area of ​​the driver's seat.

[0066] The vehicle 1 has a camera 2, which in the example shown is attached to the underside of the rearview mirror 11. However, the camera 2 could also be mounted at other (suitable) locations on the vehicle 1. Next to the camera 2 are two essentially point-like sources 3 of electromagnetic radiation, for example, IR diodes (IR LEDs) 3. The camera 2 and the sources 3 of electromagnetic radiation are connected to the on-board computer 8 for data transmission, either via a wired connection or via a wireless communication interface 10 of the on-board computer 8. For the sake of clarity, connecting wires are not shown in the figures. The on-board computer 8 may also have a memory 9, which is described below. The on-board computer 8, the camera 2, and the sources 3 of electromagnetic radiation together constitute a device 15 for determining position information of an object 5.

[0067] In the Fig. In the example shown, person 4 is wearing headphones 5. These headphones have a surface that reflects at least part of the electromagnetic radiation from the sources 3. In this example, these headphones 5 are an object 5 whose position information is to be determined by the device 15. The position information specifies the position of object 5 in a vehicle-fixed reference system, for example, as Cartesian or polar coordinates. The vehicle-fixed reference system can originate from the camera 2, but this is not mandatory.

[0068] Fig. Figure 2 schematically shows a possible beam path according to one embodiment of the present disclosure. Fig. Figure 2 again shows two sources 3 of electromagnetic radiation and the camera 2. Also shown is a reflective surface 6 of the object 5. We first consider the solid lines and not the dashed lines. Electromagnetic radiation emanating from the sources 3, especially in an essentially omnidirectional manner, is reflected by the reflective surface 6, see reflections 7 and 7a. The reflected radiation is detected by the camera 2. Radiation that is reflected in such a way that the reflected radiation does not reach the camera 2 is in Fig. 2 not shown and not to be considered further within the scope of the present disclosure.

[0069] Fig. Figure 3 schematically shows an image content captured by camera 2 according to the design as described above. Fig. 2. Camera 2 "sees" the reflections 7 and 7a of the two sources 3 of electromagnetic radiation. Fig. 2, as they are reflected at the reflective surface 6 of the object 5. From the camera 2's perspective, the reflections 7 and 7a are separated by a certain distance, indicated by the double arrow. This distance between the two reflections 7 and 7a is detected and determined by the camera. Furthermore, the camera can determine the type of object 5. This can be done, for example, through wireless communication via the communication interface 10 of the on-board computer 8 and a communication interface 14 of the object 5. Additionally or alternatively, the camera 2 can also determine the type of object 5 based on certain characteristics that the camera 2 detects.

[0070] Furthermore, the camera can determine the position in the xy direction, as is known from the state of the art.

[0071] Once the type of object 5 is known to the device 15, in particular to the on-board computer 8, the on-board computer 8 can determine, acquire, or retrieve design details of the object 5. For example, if the on-board computer 8 has recognized that the object 5 is a smartphone 5 or, if applicable, a specific type of smartphone 5, the on-board computer 8 can assume that the reflective surface 6 is a plane. This would be an example of the on-board computer 8 itself determining design details of the object 5.

[0072] If necessary, the memory 9 of the on-board computer 8 contains construction details of object 5. Based on the determined type of object 5, the on-board computer 8 can therefore retrieve these construction details from memory 9.

[0073] In particular, if the on-board computer cannot itself determine 8 design details of the object 5 and has not stored them in memory 9, the on-board computer 8 can request design details of the object 5 from another resource via the communication interface, in particular via a data network such as the Internet 10, and record them upon receipt of the requested design details.

[0074] Using the xy-position of object 5 determined by camera 2 or the on-board computer 8, the determined distance between reflections 7 and 7a (from the perspective of camera 2), and the construction details of object 5, the on-board computer 8 can now determine the z-position of object 5, for example, its distance from camera 2. For this purpose, the on-board computer 8 can use triangulation. Thus, the device 15 can determine positional information for object 5, in particular three independent coordinates in a vehicle-fixed reference system.

[0075] If necessary, as explained above, the orientation of object 5 in relation to vehicle 1 is also determined. The determined orientation of object 5, or orientation information indicating the spatial orientation of object 5 in relation to vehicle 1, is then taken into account when determining the position information of object 5.

[0076] We will now consider the dashed lines of the Fig. 2, as well as the Fig. 4. The reflective surface of object 5 is now further away from camera 2 than was the case with the solid lines, as described above.

[0077] For better differentiation, the reflective surface of object 5 is now labelled 6a. The electromagnetic radiation emanating from the sources 3 is reflected at the reflective surface 6a (reflections 7b, 7c) and detected by camera 2. Because the reflective surface 6a is farther away than the reflective surface 6, the resulting reflection from camera 2 is approximately as shown in Fig. 4. Image content shown. Reflections 7b and 7c are closer together here than in Fig. 3 was the case. The distance between reflections 7b and 7c is again indicated by a double arrow.

[0078] By triangulation and using the xy-position of object 5 determined by camera 2 or the on-board computer 8, the determined distance of reflections 7b and 7c (from the perspective of camera 2) and the construction details of object 5, the on-board computer 8 can now determine the z-position of object 5 in the area shown by dashed lines in Fig. Determine the indicated position, for example the distance from camera 2.

[0079] From the comparison of Fig. 2, Fig. 3 and Fig. 4. It can be seen that a larger distance between reflections (7, 7a in Fig. 3) indicates a smaller distance between object 5 and camera 2, a smaller distance between reflections (7b, 7c in Fig. 4) to a greater distance of object 5 from camera 2.

[0080] Fig. Figure 5 schematically shows a possible beam path according to one embodiment of the present disclosure. This embodiment can be considered a variant of the embodiment according to Fig. 2. To avoid repetition, only the differences are pointed out. In the execution according to Fig. In section 5, the device 15 has only one source 3 of electromagnetic radiation, but two cameras 2. Two situations are again depicted. Solid lines indicate a situation where the object 5 is relatively close to the cameras 2, and dashed lines indicate a situation where the object 5 is relatively far from the cameras 2.

[0081] Each of the cameras 2 captures only one reflection. In the situation with solid lines, these are reflections 7 and 7a; in the situation with dashed lines, they are reflections 7b and 7c.

[0082] By comparing the image content captured by cameras 2 and using the xy-position of object 5 determined by cameras 2 or the onboard computer 8, as well as the construction details of object 5, the onboard computer 8 can again determine the z-position of object 5. This can be done as before in connection with the Fig. 2 to 4 described, but now there are two image contents with one reflection each (7 or 7a; 7b or 7c) instead of one image content with two reflections.

[0083] Also from Fig. Figure 5 shows that the image content captured by the cameras 2 differs depending on whether the object 5 is in the closer position (reflective surface 6) or in the more distant position (reflective surface 6a).

[0084] Fig. Figure 6 schematically shows a possible beam path according to one embodiment of the present disclosure. In this embodiment, the device 15 has only one camera 2 and only one source 3 of electromagnetic radiation. For this purpose, the object 5 has two (separate) reflective surfaces 6 and 6a. In this example, the object 5 is a pair of glasses 5, in particular AR glasses 5. The beam path for a reflection 7 at the left lens 6 is indicated by the solid lines, and the beam path for a reflection 7a at the right lens 6a is indicated by the dashed lines.

[0085] Although the device 15 in this embodiment has only one camera 2 and only one source 3 of electromagnetic radiation, the camera 2 nevertheless detects two reflections 7, 7a (namely one each at the reflecting surfaces 6, 6a) and can use these to determine a distance of the object 5 from the camera 2, as described above.

[0086] Fig. Figure 7 schematically shows an image content captured by camera 2 according to the design as described above. Fig. 6. The spectacle lenses are indicated as reflective surfaces 6 and 6a. Reflection 7 was detected on the left spectacle lens (from the perspective of camera 2), i.e., the left reflective surface 6. Reflection 7a was detected on the right magnifying glass (from the perspective of camera 2), i.e., the right reflective surface 6a. The distance of object 5 from camera 2 can be determined as described above.

[0087] The method disclosed can generally provide reliable results if the sum of the number of vehicle-mounted sources 3 of electromagnetic radiation, of reflective surfaces 6, 6a and of sensors 2 or cameras 2 is at least four, as described in the explanatory notes. Fig. 2, Fig. 5 and Fig. 6 is the case. In some cases, however, a single implementation can also yield a (reliable) result if only one source 3 of electromagnetic radiation, only one camera 2, and only one reflective surface 6 are present. Such an example is in Fig. 8 shown.

[0088] Fig. Figure 8 schematically shows a possible beam path according to one embodiment of the present disclosure. In this example, the reflecting surface 6, which faces the camera 2 and the source 3 of electromagnetic radiation, has a concave shape. Depending on the position of a focal point of this concave surface 6, (exactly) two reflections 7, 7a can be detected by the camera 2 in this arrangement as well, and the position information of the object 5, which has the reflecting surface 6, can be determined from these. The calculation using triangulation can be carried out as described above.

[0089] Fig. Figure 9 schematically shows an image captured by a camera according to one embodiment of the present disclosure. The in Fig. The image content shown in Figure 9 would result, for example, if the object 5 is an AR headset 5 with two reflective surfaces 6, 6a, and the device 15 has two sources 3 of electromagnetic radiation and a camera 2. Of the reflections captured by the camera 2, reflections 7 and 7b are located in the area of ​​the left reflective surface 6. Reflections 7a and 7c are located in the area of ​​the right reflective surface 6a.

[0090] The position information of object 5, in particular the distance of object 5 from camera 2, can be obtained in the example shown. Fig. The distance between reflections 7 and 7b on the left reflecting surface 6 can be determined in three different ways. Triangulation can be used in each of these methods, as described previously. First, the distance between reflections 7 and 7b on the left reflecting surface 6 can be evaluated. This distance is indicated by the small double arrow on the left. Second, the distance between reflections 7a and 7c on the right reflecting surface 6a can be evaluated. This distance is indicated by the small double arrow on the right. Third, the distance between the positions of the two groups of reflections—7 and 7b on the one hand, and 7a and 7c on the other—can be evaluated. The distance between these two groups (or rather, the distance between the respective centers of these two groups) is indicated by the long double arrow.Furthermore, two or all three of these calculations can be used to determine the positional information of object 5, in particular the distance of object 5 from camera 2. If necessary, the determination of object 5's positional information can be made more reliable by using two or three of these calculations. For example, an average value, which may be weighted, can be determined from the two or three calculations.

[0091] While based on the Fig. In sections 1 to 9, exemplary embodiments have been explained for the sake of clarity, in which a maximum of two sources 3 of electromagnetic radiation, two cameras, and two reflective surfaces 6, 6a are present, or in which the total number of sources 3 of electromagnetic radiation, cameras 2, and reflective surfaces 6, 6a is a maximum of five. However, embodiments are also possible in which a larger number of sources 3 of electromagnetic radiation, cameras 2, and / or reflective surfaces 6, 6a are present. This allows for redundancy and, if necessary, higher accuracy can be achieved through averaging or similar methods.

[0092] It should also be noted that the position of the camera(s) 2 and the sources 3 of electromagnetic radiation can, in principle, be chosen arbitrarily. In particular, the sources 3 of electromagnetic radiation do not have to be arranged (near) the at least one camera 2 or even symmetrically with respect to this camera 2 (or vice versa when using only one source 3 of electromagnetic radiation and two or more cameras 2). If the position of the at least one camera 2 and the at least one source 3 of electromagnetic radiation used is known in a vehicle-fixed reference frame, the position information of the object 5 can be determined according to the disclosure.

[0093] The camera(s) 2 and source(s) 3 of electromagnetic radiation do not need to lie on a straight line or even in a single plane.

[0094] Fig.Figure 10 shows a flowchart to illustrate a procedure according to one embodiment of the present disclosure.

[0095] After the start (20) of the method for determining position information of object 5, in step 21 electromagnetic radiation originating from one or more vehicle-mounted sources 3 of electromagnetic radiation and reflected by at least one reflective surface 6, 6a of object 5 is detected by at least one sensor 2, in particular at least one camera 2. In step 22, construction details of object 5 are determined, detected, or retrieved. Finally, in step 23, position information of object 5 is determined from the reflected electromagnetic radiation using the construction details of object 5 and position information from the one or more vehicle-mounted sources 3 of electromagnetic radiation and the at least one sensor 2. The method can then end (24).

[0096] While at least one exemplary embodiment has been described above, it should be noted that a large number of variations exist. It should also be noted that the described exemplary embodiments are merely non-limiting examples, and it is not intended to restrict the scope, applicability, or configuration of the devices and methods described herein. Rather, the preceding description will provide guidance for those skilled in the art in implementing at least one exemplary embodiment. It is understood that various modifications to the function and arrangement of the elements described in an exemplary embodiment can be made without derogating from the subject matter defined in the appended claims and their legal equivalents.The features described herein may be combined with one another in any way, unless expressly excluded or technically impossible. Likewise, features described primarily in connection with one aspect disclosed herein may also represent features of the other aspects disclosed herein. Furthermore, all aspects and features disclosed herein, either individually or in combination, are to be considered aspects of the present invention. Reference symbol list 1 vehicle 2 Sensor, Camera 3 Source of electromagnetic radiation 4th person, passenger 5. Object, AR glasses 6, 6a reflective surface 7, 7a-7c Reflection 8 processor, on-board computer 9 storage 10 Communication interface 11 Rearview mirrors 12 Windscreen 13 Dashboard 14 Communication interface of object 5 15 Device (for determining position information) 20-24 process steps QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] CN 103886307 A

[0003]

Claims

[1] Method for determining position information of an object (5) which has at least one surface (6, 6a) reflecting electromagnetic radiation and which is located in a vehicle (1), wherein the method comprises: Detection of electromagnetic radiation reflected from at least one reflective surface (6, 6a) of the object (5), which originates from one or more vehicle-mounted sources (3) of electromagnetic radiation, by at least one sensor (2), in particular at least one camera (2); Determining, recording or retrieving construction details of the object (5); and Determining position information of the object (5) from the reflected electromagnetic radiation using the design details of the object (5) and position information of the one or more vehicle-mounted sources (3) of electromagnetic radiation and the at least one sensor (2). [2] Method according to claim 1, wherein the object (5) has at least two reflective surfaces (6, 6a) which are not coplanar. [3] Method according to claim 1 or 2, wherein the at least one reflective surface (6, 6a) has a curvature. [4] Method according to any of the preceding claims, wherein each of the one or more vehicle-mounted sources (3) of electromagnetic radiation is a substantially point-like source (3) of electromagnetic radiation. [5] Method according to one of the preceding claims, wherein the sum of the number of vehicle-mounted sources (3) of electromagnetic radiation, of reflecting surfaces (6, 6a) and of sensors (2) is at least four. [6] Method according to any of the preceding claims, wherein the object (5) comprises AR glasses (5), headphones (5) or a smart device (5), in particular a smartphone (5) or a smartwatch (5). [7] Method according to one of the preceding claims, wherein the position information of the object (5) includes a distance of the object (5) from the at least one sensor (2) used for detecting the electromagnetic radiation reflected at the reflecting surface (6, 6a) of the object (5). [8] Method according to one of the preceding claims, wherein the method comprises determining a spatial relationship, in particular a distance, in particular an angular distance between at least two of the reflections (7, 7a, 7b, 7c) to each other, in order to determine the position information of the object (5) using the determined spatial relationship. [9] Method according to one of the preceding claims, wherein the method further comprises determining orientation information indicating a spatial orientation of the object (5), and wherein the orientation information is additionally taken into account when determining the position information of the object (5). [10] Device (15) for use in a vehicle (1), wherein the device (15) is provided for determining position information of an object (5) which has at least one surface (6, 6a) reflecting electromagnetic radiation and which is located in the vehicle (1), wherein the device (15) comprises: one or more vehicle-mounted sources (3) of electromagnetic radiation; at least one sensor (2) for detecting electromagnetic radiation reflected from the at least one reflective surface (6, 6a) of the object (5), which originates from one or more vehicle-mounted sources (3) of electromagnetic radiation; a communication interface and / or a memory (9) for determining, recording or retrieving design details of the object (5); and a processor (8) configured to determine position information of the object (5) from the reflected electromagnetic radiation using the design details of the object (5) and position information of one or more vehicle-mounted sources (3) of electromagnetic radiation and of the at least one sensor (2). [11] Vehicle (1), in particular motor vehicle (1), comprising a device (15) according to claim 10. [12] Computer program comprising instructions which, when executed by a device (15) according to claim 10, cause the device to perform the steps of a method according to any one of claims 1 to 9. [13] Computer program product comprising a computer-readable medium on which a computer program according to claim 12 is stored.

Images