SYSTEM FOR DETERMINING THE DISTANCE OF AN OBJECT

Description

[0001] The invention relates to a system for determining the distance of an object with respect to a reference position.

[0002] The measurement of the distance or separation of an object relative to a sensor is usually performed by measuring the time-of-flight of sensor signals or using frequency-modulated methods. Furthermore, algorithmic methods for distance measurement are known, which, for example, use so-called "disparity maps" of a stereo camera to generate a three-dimensional image of the stereo camera's surroundings. Based on this three-dimensional image, the distance of objects relative to the stereo camera can be determined.

[0003] However, these known distance measurement methods require dedicated sensors or cameras that can only be used for distance measurement. However, such 3D sensors, such as time-of-flight (TOF) cameras or stereo cameras, are expensive compared to monocular cameras and also have a relatively low lateral resolution. The low lateral resolution of these sensors compared to monocular cameras prevents their use, for example, in the detection of object features required for advanced image analysis, such as object classification and identification, or in fine-grained direction or pose estimation of objects or people.

[0004] Furthermore, while approaches are known that allow estimating the distance of an object using a monocular camera, these approaches are often based on predefined assumptions, for example, regarding the height of people. However, these assumptions can lead to errors in distance measurements, for example, if the assumptions do not correspond to the actual object or person present. For example, the assumption regarding a person's height may not be correct if the person is not in a standing posture.

[0005] From CN 111 561 906 A a system with the features according to the preamble of claim 1 is known.

[0006] US 2017 / 0031369 A1 describes a system with features according to a similar technology.

[0007] CN 106 127 163 A describes a system for determining the distance of an object relative to a reference position. The system comprises an optical sensor located at the reference position. Predetermined identification features of an object are detected in an image captured by the sensor. Furthermore, predetermined geometric data of the object are received in order to use this data to determine the distance of the object. If the distance between the sensor and a detected target person falls below a threshold value and the target person cannot be assigned to a specific location, the system issues an error message.

[0008] An object of the invention is to provide a system which enables a reliable determination of the distance of an object relative to a reference position with low requirements regarding the sensor used.

[0009] This object is achieved by a system having the features of claim 1. Advantageous developments of the invention are specified in the subclaims, the description and the drawings.

[0010] The system is designed to determine the distance of an object relative to a reference position and comprises an optical sensor arranged at the reference position and having a predefined field of view, as well as an electronic processing unit. The electronic processing unit is configured to receive an image captured by the optical sensor as well as predetermined identification features and predetermined geometric data of the object. Furthermore, the electronic processing unit is configured to detect the object based on the predetermined identification features in the field of view of the optical sensor, to identify at least two key positions of the object in the image, and to determine the distance between the key positions in the image.The electronic processing unit determines the distance of the object with respect to the reference position based on the distance of the key positions in the image and on the basis of the predetermined geometric data of the object.

[0011] The optical sensor only needs to be able to capture a two-dimensional image of its surroundings. This image can be used to determine the distance between the object's key positions and, ultimately, the distance between the object and the reference position, if the object's identification features and geometric data are available. Consequently, there are few requirements regarding the optical sensor's technical specifications.

[0012] A prerequisite for distance determination using the system is that certain properties of the object, i.e., the identification features and the geometric data, are known in advance. The determined distance between the key positions is used together with the prior information about the object to determine the distance of the object relative to the reference position. For example, known properties of the optical sensor, such as its imaging behavior, can be used.

[0013] The predetermined identification features can include, for example, a person's face, which is pre-captured as a camera image, characteristic dimensions or shapes of an object, or a radio frequency identification tag (RFID tag). Furthermore, the identification features can also include specific poses or movement sequences of a person, which are pre-captured by a camera and provided in the form of a camera image or an image sequence. Thus, the object can be classified using the identification features not only with regard to the object type, but also, for example, with regard to the person's posture. For example, a person may have different identification features when standing than when sitting.

[0014] The predetermined object data, on the other hand, refers to the dimensions of the object. The predetermined object data can also include dimensions between certain features of a person that depend on the person's posture, for example, the distance between a person's shoulder and knee. If the identification feature is designed accordingly (e.g., as an RFID tag) and a corresponding reader is used, the object data can also be stored there and used for identification.

[0015] The at least two key positions of the object can include the respective mapping of characteristic, easily recognizable points on the object, which can be detected, for example, using known algorithms for pattern and / or object recognition. Object positions that are specified on the object in "object space" are thus assigned, using the optical mapping, to the respective key positions in the image, which are thus located in "image space." For people, the specified object positions can include, for example, the positions of joints, such as the positions of a shoulder and a knee.

[0016] The distance between key positions in the image can therefore also be referred to as the distance in image space, which can be determined, for example, by counting the number of pixels in the image that are located along a straight line between the key positions. Given a known pixel size, the distance between key positions in the image can thus be determined based on the number of pixels between the key positions.

[0017] Based on the predetermined geometric data of the object, the actual distance between at least two predetermined object positions can be determined, which can also be referred to as the distance in object space. Finally, given known imaging properties of the optical sensor, the distance between the object and the reference position can be determined by relating the distance of the key positions in the image, i.e., in image space, to the actual distance of the predetermined object positions in object space.

[0018] One advantage of the system according to the invention is that a simple, cost-effective optical sensor can be used. Compared to expensive 3D sensors such as time-of-flight sensors or stereo cameras, such a simple sensor can exhibit high resolution, for example, in the lateral direction. This allows the sensor to be used not only for distance measurement, but also for other tasks in the field of image analysis or, for example, for implementing gesture control.

[0019] Using advance information about the object whose distance is to be determined relative to the reference position, the reliability of distance determination can be improved compared to approaches based on fixed assumptions, for example, regarding the height of a person. Furthermore, the system allows for dynamic tracking of a detected or classified object within the field of view of the optical sensor.

[0020] The electronic processing unit is further configured to output an error message if the object cannot be detected based on the predetermined identification features in the field of view of the optical sensor and / or if the key positions of the object cannot be identified in the image. In an industrial environment, such an error message can be used, for example, to suppress the movement of objects in such an environment that may be associated with a hazard.

[0021] In the event that an object is recognizable in the image, but the object and / or the key positions of the object are not clearly identifiable in the image, the electronic processing unit is additionally designed to assign a predefined distance to two selected points in the image, which replaces the distance of the predetermined object positions, i.e. in object space, when determining the distance between the object and the reference position. The predefined distance can be specified as a standard value in the system and can include, for example, a standard size for people. In this case, the precisely determined distance between the object positions, which is contained in the geometric data and is normally assigned to the key positions, is replaced by a standard value.Although this limits the accuracy of the system, it still allows distance determination even for unidentifiable key positions.

[0022] The electronic processing unit can further be configured to additionally determine the distance of the object relative to the reference position based on parameters of the optical sensor. The parameters of the optical sensor can relate to its imaging properties. Precise knowledge of the parameters of the optical sensor can increase the reliability of the system in determining the distance relative to the reference position.

[0023] The optical sensor can comprise a monocular camera. The monocular camera is a low-cost optical sensor with high lateral resolution. Furthermore, the monocular camera can be a surveillance camera that is already installed in a desired area, for example, within an industrial facility. This further reduces the system cost, since in such a case, it is only necessary to additionally implement the processing unit for determining the object's distance.

[0024] If the optical sensor is a monocular camera, the electronic processing unit can be further configured to determine the distance of the object relative to the reference position based on data relating to the imaging properties of the monocular camera. In this case, for example, the instantaneous focal length of the monocular camera can be taken into account to determine the distance of the object relative to the reference position if the distance of the key positions in image space has been determined and the distance of the predefined object positions in object space is known.

[0025] Separate markers can also be attached to the object, which define the predefined object positions and are assigned to the key positions. Accordingly, the geometric data can include a respective distance between any two of the predefined markers. The predefined markers can be attached, for example, to a person's clothing or to the outer surface of an object and can include, for example, suitable reflective surfaces or lights such as LEDs. The markers can then be identified in the image as key positions. Such predefined markers can facilitate the identification and determination of the respective distance and improve the reliability of the system.

[0026] The electronic processing unit can further be configured to recognize the predetermined identification features in the image captured by the optical sensor. This allows the object to be detected in the field of view of the optical sensor. This detection can be achieved, for example, by recognizing a person's face or their specific poses or movements. The image captured by the optical sensor therefore serves, on the one hand, to verify the presence of the object based on the identification features and, on the other hand, to determine the distance between the key positions in the image space.

[0027] Alternatively or additionally, the predetermined identification features may comprise at least one radio frequency identification (RFID) tag. The use of such a radio frequency identification tag can either increase the reliability of detecting the object in the field of view of the optical sensor or replace the detection of the object using the image captured by the optical sensor. In the latter case, no pattern or object recognition is required to be applied to the image captured by the optical sensor. This simplifies the application of the system.

[0028] The geometric data of the object can be determined in advance using a CAD model of the object and / or at least one 3D scanner. Using a CAD model and / or the 3D scanner can provide precise geometric data of the object to be detected in the field of view of the optical sensor and whose distance relative to the reference position is to be determined. Precise geometric data can improve the reliability of the system, since the determined distance of the object depends on the geometric data.

[0029] The electronic processing unit can also be configured to monitor the optical sensor based on a predefined distance between control positions arranged on one or more fixed objects. Such monitoring further improves the reliability of the system. When using a low-cost monocular camera, a so-called temperature response may also occur, i.e., a change in imaging properties with temperature. In this case, monitoring the camera using the predefined distances between fixed control positions allows the camera to be recalibrated based on the predefined distance.

[0030] Furthermore, the electronic processing unit can be configured to additionally determine a spatial position of the object based on a conversion of the distance of the object relative to the reference position using known imaging properties of the optical sensor. This conversion is also referred to as the "depth map to point cloud method" and can be expressed by the following formula: x , y , z , 1 = D u v * inv K * u , v , 1 , where u, v are the coordinates of a respective pixel in the image captured by the optical sensor, and K is an intrinsic camera matrix that describes the imaging properties of the optical sensor. "inv" denotes the inverse of the intrinsic camera matrix K, while D(u,v) denotes a "depth map value" at the respective pixel with the coordinates u, v. x, y, and z are the three Cartesian coordinates in object space that are assigned to the respective pixel in image space.

[0031] Furthermore, the three-dimensional position of the object can be determined, for example, by triangulation if the respective distance of one or more positions on the object relative to at least two reference positions is known. However, this requires the use of at least two optical sensors located at the respective reference position.

[0032] The three-dimensional position or spatial orientation of the object can be described by the three-dimensional coordinates of the object relative to a coordinate system whose origin is located at the reference position. The spatial orientation of the object represents additional information, i.e., in addition to the distance of the object relative to the reference position. The information regarding the spatial orientation can be relevant for other systems and applications related to the optical sensor.

[0033] Furthermore, if the respective spatial position of at least two objects in the sensor's field of view is determined by the system, the distance between the at least two objects can also be determined and tracked. If the distance between the at least two objects falls below a predetermined threshold, countermeasures can be initiated to prevent a collision between the objects, for example, by reducing the respective speed of the objects. In addition, an error message can be issued if the distance between the at least two objects falls below the predetermined threshold.

[0034] The invention is described below by way of example using an advantageous embodiment with reference to the accompanying figures. They show, schematically: Fig. 1 a system for determining the distance of an object and Fig. 2 details of an electronic processing unit of the system of Fig. 1 .

[0035] Fig. 1 schematically shows a system 11 for determining the distance of one or more objects 13 relative to a reference position 14. The system 11 comprises an optical sensor 15, which is designed as a monocular camera and is arranged at the reference position 14. More precisely, the reference position 14 is located in the imaging plane of the camera 15. The monocular camera 15 has a field of view 16 and outputs a two-dimensional image 17 of the object(s) 13. The system 11 further comprises an electronic processing unit 19, which is designed to receive the image 17 captured by the camera 15 and to process it to determine the distance between the object(s) 13 and the reference position 14.

[0036] The objects 13 captured in the image 17 by the camera 15 include persons 21 and / or objects 23. The respective person 21 and the respective object 23 each have predetermined identification features 25 which make it possible to detect the presence of the person 21 and / or the object 23 in the field of view 16 of the camera 15.

[0037] For persons 21, the identification features 25 include, for example, a camera image of the face of the person 21, which was previously captured by another system and is provided to the system 11. The identification features 25 alternatively or additionally include a radio frequency identification tag (RFID tag) of the respective person 21 or the respective object 23. By capturing data from the radio frequency identification tag, the presence of the respective object 13 in the field of view 16 of the camera 15 can also be verified. Furthermore, the identification features 25 can also be generated on the basis of CAD model data, which allow a unique identification of the respective object 13 based on distances or shapes defined by the CAD model data. Overall, the identification features 25 are provided as data for object recognition for the system 11 (see also Fig. 2 ).

[0038] Furthermore, the respective object 13 is assigned geometric data 27, which comprise the distance between at least two predetermined object positions 28 of the respective object 13. For a person 21, these predetermined object positions 28 comprise, for example, the position of a shoulder and a knee, so that the geometric data 27 comprise the distance between the shoulder and the knee of the respective person 21. For an object 23, the predetermined object positions 28 are located, for example, at corners or edges of the object 23, so that the geometric data 27 comprise, for example, a side or edge length of the object 23.

[0039] By means of the optical imaging by the camera 15, the predetermined object positions 28 located in the "object space" are assigned respective key positions 29 in the image 17, which are thus located in the "image space." The predetermined object positions 28 can also be defined by predefined markings (not shown) on the respective object 13, which thus define the key positions 29 in the image. If such markings are suitably applied to the clothing of the person 21 or to the outer surface of the object 23, the markings are easily recognizable as predetermined object positions 28 in the image 17. The markings can be, for example, colored markings, reflective surfaces, or lights. Furthermore, the distances between such markings can be easily determined and recorded as part of the geometric data 27.

[0040] The geometric data 27 are generated in advance by other systems and provided for the system 11. For persons 21 and objects 23, for example, 3D scanners can be used to generate a respective set of geometric data for the respective object 13, ie for the respective person 21 and the respective object 23. Furthermore, the geometric data 27 for objects 23 can in turn also be generated on the basis of CAD model data.

[0041] Fig. 2shows schematic details regarding the design of the electronic processing unit 19 and regarding the information that the electronic processing unit 19 uses to determine the distance between the respective object 13 and the camera 15 or the reference position 14. The electronic processing unit 19 comprises several modules 31 to 37, each implemented as software modules and / or as logic circuits. In detail, these modules of the electronic processing unit 19 comprise a module 31 for object classification, a module 33 for detecting key positions 29, a module 35 for determining the distance of the key positions 29 in the image space, and a module 37 for determining the distance between the respective object 13 and the camera 15 or the reference position 14.

[0042] Both the object classification module 31 and the key position detection module 33 each receive the two-dimensional image 17 captured by the monocular camera 15. The object classification module 31 additionally receives the identification features 25 of the respective objects 13 as object detection data. Based on the identification features or object detection data 25, the object classification module 31 determines whether a respective object 13 is present in the field of view 16 of the camera 15.

[0043] For this purpose, the object classification module 31 compares, for example, a camera image of the face of a specific person 21 with the two-dimensional image 17 in order to identify the face of the person 21 in the image 17. Alternatively or additionally, the identification and classification of a specific person 21 takes place using data, e.g., a radio frequency identification tag, which is received as object recognition data 25 from the object classification module 31. For objects 23, the object recognition data or identification features 25 include predefined shapes and / or details on dimensions, e.g., the ratio of length to width.

[0044] For persons, the object classification module 31 not only identifies the presence of the respective known person 21 in the field of view 16 of the camera 15, but also distinguishes between different poses or postures of the respective person 21. For example, the object classification module 31 identifies whether the respective person 21 is standing or sitting. In this case, the object classification by means of the module 31 includes, for example, the classes "seated person A" and "standing person A," i.e., two different classes for a specific person A identified based on a camera image of their face. Each class of an object 13 is further assigned a set of geometric data 27, which includes one or more distances between the predetermined object positions 28 of the respective object 13. In Fig. 1Furthermore, three different poses or postures of the person 21 are shown, to which three different object classes can correspond.

[0045] The key position recognition module 33 determines whether at least two predefined key positions 29 of the respective object 13 are recognizable in the image 17. The key positions 29 are, for example, the respective representation of the position of a shoulder and the position of a knee for a person 21. The key position recognition module 33 thus determines, for example, whether a shoulder and a knee of a person 21 are recognizable as key positions 29 in the image 17, as is the case in the image 17 of Fig. 1 which is output by the camera 15.

[0046] When the key positions 29 are recognizable in the image 17, the key position recognition module 33 forwards information regarding the key positions 29 to the module 35 for determining the distance between the key positions 29. In detail, this information includes at which pixels within the two-dimensional image 17 the respective key position 29 is located. Using this information, the module 35 determines the distance between each two key positions 29 within the image 17, i.e., in the image space. For this purpose, the module 35 counts the number of pixels along a straight line between each two key positions 29. If the pixel size in the image 17 is known, the number of pixels between the key positions 29 determines their distance in the image space, i.e., within the image 17.

[0047] The module 37 for determining the distance between object 13 and camera 15 or reference position 14 receives both the classification of the respective object 13 from the module 31 for object classification and one or more distances between two key positions 29 in the image space from the module 35. In addition, the module 37 receives data 39 on the imaging properties of the camera 15 and the geometric data 27 that were determined in advance and are assigned to the respective classified object 13. The geometric data 27 include respective distances between the predetermined object positions 28 in the object space, ie their actual distances that were determined in advance using another system.

[0048] Based on the distance between two key positions 29 in the image space, i.e. within the image 17 (cf. Fig. 1), based on the distance of the predetermined object positions 28 in object space that are assigned to these key positions 29, and based on the data 39 on the imaging properties of the camera, the module 37 determines the distance between the identified object 13 and the camera 15. In other words, the module 37 uses the relationship between the distance of the key positions 29 in image space and the distance of the predetermined object positions 28 in object space, which is determined by the known imaging properties of the camera 15, which include, for example, its focal length, to determine the distance between the identified object 13 and the camera 15 or the reference position 14 at which the camera is located. An output 41 of the electronic processing unit 19 comprises the distance between the identified object 13 and the camera 15 in object space, which was determined by means of the module 37.

[0049] If the object classification module 31 cannot identify or classify any of the objects 13, the electronic processing unit 19 outputs an error message as output 41. Such an error message can be used by other systems that are communicatively connected to the system 11, for example in an industrial environment, to suppress dangerous movements of objects and items in the environment of the system 11.

[0050] If, however, the object classification module 31 classifies or identifies a specific object 13 and the key position detection module 33 does not detect any of the predetermined key positions 29 of the identified object 13, the module 37 for determining the distance between the object 13 and the camera 15 can use a predefined distance contained in the geometric data 27, for example the height of a person 21, instead of the distance between the object positions 28 in the object space, and estimate the corresponding counterpart of this distance in the image 17 to thereby determine the distance between the object 13 and the camera 15.

[0051] Overall, the system 11 thus uses prior information from objects 13, i.e., the identification features 25 and the geometric data 27 provided by other systems, and the image 17 captured by the monocular camera 15, to determine the distance between the respective object 13 and the camera 15 or the reference position 14, given known imaging properties of the monocular camera 15. Since only a two-dimensional image 17 is required as input for the processing unit 19, there are few requirements for the monocular camera 15, so that it can be designed as a simple surveillance camera, which is often already present in a specific industrial infrastructure.

[0052] To improve the reliability of the system 11, the electronic processing unit 19 monitors the monocular camera 15 based on a predefined distance between control positions (not shown). The control positions are arranged on one or more fixed objects. When using a low-cost monocular camera 15, a so-called temperature gradient may occur, i.e., a change in the imaging properties with temperature. In this case, monitoring the monocular camera 15 using the predefined distances between fixed control positions allows the monocular camera 15 to be recalibrated based on the predefined distances.

[0053] The electronic processing unit 19 is further provided for determining a spatial position of the object 13 based on the distances between at least two predetermined positions with respect to the reference position 14. In detail, the three-dimensional position of the object 13 is determined by means of triangulation if at least two distances between predetermined positions on the object 13 are known. This allows three-dimensional coordinates of the object 13 to be determined with respect to a coordinate system whose origin is located at the reference position 14. The spatial position of the object 13 determined in this way represents additional information, i.e., in addition to the distance of the object 13 with respect to the reference position. The information regarding the spatial position can be relevant for other systems and applications in the environment of the monocular camera 15.

[0054] Furthermore, if the respective spatial position of at least two objects 13 in the field of view of the camera 15 can be determined by means of the system 11, the distance between the at least two objects 13 can also be determined and tracked. If the distance between the at least two objects 13 further falls below a predetermined threshold, countermeasures can be initiated to avoid a collision of the objects 13, for example by reducing the respective speed of the objects 13. In addition, an error message can be output if the distance between the at least two objects 13 falls below the predetermined threshold. List of reference symbols

[0055] 11System for determining the distance of an object 13Object 14Reference position 15Optical sensor, monocular camera 16Field of view 17Two-dimensional image 19Electronic processing unit 21Person 23Object 25Identification feature 27Geometric data 28Specified object position 29Key position 31Module for object classification 33Module for detecting key positions 35Module for determining the distance of key positions in image space 37Module for determining the distance between object and camera 39Data on the imaging properties of the camera 41Output of the electronic processing unit

Claims

1. A system (11) for determining the distance of an object (13) with respect to a reference position (14), said system (11) comprising: an optical sensor (15) which is arranged at the reference position (14) and which has a predefined field of view (16), an electronic processing unit (19) which is configured: to receive an image (17) acquired by the optical sensor (15), to receive predetermined identification features (25) and predetermined geometric data (27) of the object (13), to detect the object (13) in the field of view (16) of the optical sensor (15) based on the predetermined identification features (25), to identify at least two key positions (29) of the object (13) in the image (17), to determine the distance of the key positions (29) in the image (17), and to determine the distance of the object (13) with respect to the reference position (14) based on the distance of the key positions (29) in the image (17) and based on the predetermined geometric data (27) of the object (13), characterized in that the electronic processing unit (19) is additionally configured to output an error message if the object (13) cannot be detected in the field of view of the optical sensor (15) based on the predetermined identification features (25) and / or if the key positions (29) of the object (13) cannot be identified in the image (17), and in that the electronic processing unit (19) is additionally configured, if an object (19) is indeed recognizable in the image (17), but the object (19) and / or the key positions (29) of the object (13) is / are not clearly identifiable in the image (17), to assign a predefined distance to two selected points in the image (17), said predefined distance replacing a distance of predefined object positions, which is included in the geometric data (27) for an assignment to the distance of the key positions (29), when determining the distance between the object (13) and the reference position (14).

2. A system (11) according to claim 1, characterized in that the electronic processing unit (19) is further configured to additionally consider parameters of the optical sensor (15) when determining the distance of the object (13) with respect to the reference position.

3. A system (11) according to claim 1 or 2, characterized in that the optical sensor (15) comprises a monocular camera.

4. A system (11) according to claim 3, characterized in that the electronic processing unit (19) is further configured to consider data with respect to the imaging properties of the monocular camera (15) when determining the distance of the object (13) with respect to the reference position (14).

5. A system (11) according to any one of the preceding claims, characterized in that at least two predefined markings, which are assigned to the at least two key positions (29), are applied to the object (13) and the geometric data (27) comprise a respective distance between a respective two of the predefined markings.

6. A system (11) according to any one of the preceding claims, characterized in that the electronic processing unit (19) is further configured to recognize the predetermined identification features (25) in the image (17) acquired by the optical sensor (15) in order to detect the object (13) in the field of view of the optical sensor (15).

7. A system (11) according to any one of the preceding claims, characterized in that the geometric data (27) of the object (13) are defined in advance by means of a CAD model of the object (13) and / or using at least one 3D scanner.

8. A system (11) according to any one of the preceding claims, characterized in that the electronic processing unit (19) is further configured to monitor the optical sensor (15) on the basis of a predefined distance between control positions which are arranged at one or more fixed objects (13).

9. A system (11) according to any one of the preceding claims, characterized in that the electronic processing unit (19) is additionally configured to determine a spatial position of the object (13) by means of a conversion of the distance of the object with respect to the reference position using known imaging properties of the optical sensor (15).