Portable communication terminal

Abstract

Portable communication terminal (10) with a first camera (12) which is configured to record a spatial view of an object (20) by means of the first camera (12) and to provide first data associated with the view of the recorded object (20), with an object recognition unit (16) for processing the first data provided by the first camera (12) and determining a classification of the object (20) by comparing the first data with previously stored classification data for objects, and with a second camera (14) for two-dimensional recordings for recording a two-dimensional view of the object (20) and for providing second data associated with the two-dimensional view of the recorded object (20), a comparison unit (18) for comparing the second data with previously stored,two-dimensional view data associated with the classified object (20) and an orientation recognition unit (22) for determining an orientation of the classified object (20) based on comparison results supplied by the comparison unit (18), wherein the portable communication terminal (10) is designed to select one of the recorded objects by means of a user input when there are several objects (20) recorded by the first camera (12) and to determine supplementary data for the object (20) corresponding to the determined orientation and to output said data by means of the output unit (24), in particular superimposed on the superimposed first and / or second data.

Description

[0001] The present invention relates to a portable communication terminal comprising a first camera configured to record a spatial view of an object by means of the first camera and to provide first data associated with the view of the recorded object, and comprising an object recognition unit for processing the first data provided by the first camera and determining a classification of the object by comparing the first data with previously stored classification data for objects.Furthermore, the invention relates to a method for determining an orientation of an object by means of a portable communication terminal having a first camera for three-dimensional recordings, comprising the steps of: recording a spatial view of the object by means of the first camera and providing first data associated with the view of the recorded object, processing the first data provided by the first camera by means of an object recognition unit, by means of which a classification of the object is determined by comparing the first data with previously stored classification data for objects.

[0002] Portable communication devices with a camera for three-dimensional recordings are generally known, so that separate written proof is not required. Portable communication devices can be, for example, smartphones, augmented reality glasses, or the like. It is also known to use such a communication device to classify a recorded object, specifically according to its type. For example, the object can be a building, an object, in particular a vehicle, but also a mountain, a tower, combinations thereof, or the like. Generic communication devices are frequently used in the field of augmented reality. This allows additional data or information, in particular about recorded objects, to be obtained and provided automatically.For this purpose, it is often necessary to determine the orientation of the recorded object, especially if information or data on selectable sections of the object are to be selectively acquired and displayed. Due to technical limitations of such cameras, systems that use a camera that can record a spatial view of an object or that can capture a three-dimensional object are only inadequately able to capture the orientation of the three-dimensionally recorded object based solely on the data provided by the first camera. Three-dimensional capture using the first camera is particularly detrimental to resolution and / or color representation.For example, if a camera that captures the three-dimensional view of an object first captures a vehicle from the front and rear, the two images are often indistinguishable for an evaluation unit that processes the data from the first camera. As a result, additional data for each view cannot be determined and displayed as desired.

[0003] Furthermore, Seyed Eghbal Ghobadi from the Department of Electrical Engineering and Computer Science at the University of Siegen revealed the use of a 3D camera in conjunction with a 2D camera in his dissertation “Real Time Object Recognition and Tracking Using 2D / 3D Images” in the oral examination on September 16, 2010.

[0004] It is the object of the invention to further develop a generic portable communication terminal and a method of the generic type in such a way that the determination of the orientation of the object is improved.

[0005] As a solution, the invention proposes a portable communication terminal according to independent claim 1. Furthermore, a method for determining the orientation of an object is proposed in further independent claim 5. Further advantageous embodiments emerge from the features of the dependent claims.

[0006] On the communication terminal side, it is particularly proposed that the communication terminal has a second camera for two-dimensional recordings for recording a two-dimensional view of the object and for providing second data associated with the two-dimensional view of the recorded object, a comparison unit for comparing the second data with previously stored two-dimensional view data associated with the classified object, and an orientation detection unit for determining an orientation of the classified object on the basis of comparison results supplied by the comparison unit.

[0007] In terms of the method, it is proposed that a generic method is supplemented by the steps of: recording a two-dimensional view of the object by means of a second camera for two-dimensional recordings and providing second data associated with the two-dimensional view of the recorded object, comparing the second data with previously stored two-dimensional view data associated with the classified object by means of a comparison unit and determining the orientation of the classified object by means of an orientation recognition unit based on comparison results provided by the comparison unit.

[0008] An essential aspect of the invention is therefore that a second camera is provided that is suitable for providing two-dimensional second data. The second camera can thus provide supplementary data with information that enables the orientation of the object, in particular parts of the object, to be reliably detected. This, in turn, makes it possible to acquire orientation-related data and provide corresponding displays on the portable communications terminal. Determining the orientation can also include determining a section of the object, for example, a door, a window of a building, a vehicle, or the like.

[0009] The portable communications terminal can be, for example, a smartphone, a camera device, augmented reality glasses, combinations thereof, or the like. The first camera can be a TOF (Time of Flight) camera, for example. Such TOF cameras typically provide data in black and white format with limited resolution. The spatial view of an object is its three-dimensional view, which can be recorded using the first camera. Accordingly, the first camera provides first data associated with the view of the recorded object, which is provided to an evaluation unit of the portable communications terminal. Of course, the evaluation unit can also be arranged in a central facility to which the portable communications terminal is connected for communication purposes, for example, a data center and / or the like.

[0010] The portable communications terminal is connected to a communications network via a wireless communication connection, for example, a GSM network, a UMTS network, an LTE network, or the like. The portable communications terminal can access the evaluation unit, if provided externally, via this wireless communication connection. Furthermore, other units, in particular a database, can of course also be accessed.

[0011] The object recognition unit can process the first data provided by the first camera. The processing can include conversion into a predefined data format. Furthermore, additional data processing steps can be provided, for example, brightness correction, contrast correction, edge detection, and / or the like. The processed data can then be compared with previously stored classification data for objects. For this purpose, the portable communications terminal can have a database with corresponding classification data. Additionally or alternatively, it can also be provided that this database is not comprised by the portable communications terminal, but is located at a remote location, for example, in the central facility, which the portable communications terminal can access via its wireless communication connection.The object recognition unit performs the comparison and classifies the object. For example, the classification can be based on usage characteristics, spatial extents, combinations thereof, or the like. The classification can, for example, include classifying an object as a vehicle, in particular a passenger car, a truck, a building, a geographical feature such as a mountain, a river, or the like, or even combinations thereof.

[0012] The first data provided by the first camera is generally sufficient to classify the object with sufficient reliability. However, this data is generally not sufficient to recognize and assign the orientation of the object, in particular sections of the object, with sufficient accuracy. The second camera serves this purpose for two-dimensional recordings, by means of which a two-dimensional view of the object or a section of it is recorded. The second camera provides second data assigned to the recorded two-dimensional view of the recorded object. This second data is fed to a comparison unit, which compares the second data with previously stored two-dimensional view data assigned to the classified object.For this purpose, the database of the portable communications terminal and / or the remote database can provide two-dimensional view data associated with the classified object. This view data can, for example, be photographs of the object from a plurality of different views. Based on a comparison result provided by the comparison unit, an orientation recognition unit can determine the orientation of the classified object. For this purpose, the orientation recognition unit can, for example, retrieve supplementary data from the database regarding the two-dimensional view data, which indicates which view the respective view data represents.

[0013] This creates the possibility of supplying the portable communications terminal with additional data regarding the orientation of the classified object. This data can then be displayed using a display unit of the portable communications terminal. For example, in a vehicle, data regarding the machine equipment can be additionally displayed in a front view. In a passenger car, for example, in a rear view, additional information regarding a trunk, in particular the contents of the trunk, other important devices such as warning and auxiliary devices for road traffic, and their position can be provided.

[0014] Supplementary data for the object corresponding to the determined orientation is determined and output by the output unit, in particular superimposed on the superimposed first and / or second data. The supplementary data can, for example, be detailed data on specific object views or sections of the object. For example, if the object is a vehicle and the section of the object is a fender with a wheel, the dimensions of the wheel and the tire size can be additionally displayed accordingly.

[0015] If several objects are recorded using the first camera, one of the recorded objects is selected using a user input. This embodiment proves to be particularly advantageous if more than one object is displayed in a recording area of the first camera. Of course, it can be provided that an orientation is determined for each of the recorded objects. Typically, however, a single object is of particular interest to the user and is to be displayed for an output. This also promotes clarity, among other things. For this reason, it is possible for the user to use user input to select the object for which they would like further information. For example, the selection can be made by manually selecting the desired object on a touch-sensitive screen.In addition, it is of course also possible to select the object by entering control commands to control a selection cursor.

[0016] In particular, the orientation recognition unit can be communicatively coupled to a database, which database provides supplementary data for the objects. Furthermore, the communication terminal can comprise an output unit for outputting the supplementary data. The output unit can preferably be a visual output unit in the form of a screen or the like. Of course, the output unit can also be a printer, by means of which supplementary data can be printed out. In the case of augmented reality glasses, for example, it can be provided that the supplementary data is displayed visibly for the user of the glasses.

[0017] Furthermore, in one embodiment, the communication terminal can include a position sensor for detecting a position of the communication terminal. This makes it possible to provide additional information that can support object detection and orientation detection. For example, the position sensor can be used to detect a swivel position of the communication terminal and provide corresponding data. The provision can be made, for example, to the object detection unit and / or the orientation detection unit.

[0018] According to a further embodiment of the invention, it is proposed that the first and second data are output in a superimposed manner by means of an output unit. This makes it possible to provide the user with additional information regarding the object of interest. The superimposed display can comprise adjusting the scales of the images belonging to the data to be superimposed by means of an evaluation or processing unit. Of course, it can also be provided that the first and second data are shifted relative to one another, preferably until a display corresponding to the respective position of the recorded view is achieved3. This makes it possible to achieve a display that is easily understandable for the user.

[0019] According to an advantageous development, it is proposed that classification data only for predefined objects be taken into account. For example, it can be provided that classification data is only available for predefined objects in the database. This can prevent data from being provided for undesired, self-evident objects. For example, it can be provided that, for a vehicle parked in a hall, data is only desired for the vehicle. By limiting the classification data to classification data for vehicles, it can be avoided that a method according to the invention is carried out for the hall in which the vehicle is parked. This allows selectivity with regard to objects of interest. Furthermore, processing effort can be reduced.

[0020] For the implementation of the method according to the invention, it can further be provided that the position of the communication terminal is taken into account. The position of the communication terminal can, for example, be an orientation in space. For example, it can be the position of an inclination relative to a horizontal or a vertical line. The accuracy of the method can thereby be further increased.

[0021] Further advantages and features can be found in the following description of an exemplary embodiment with reference to the figures. The exemplary embodiment serves merely to illustrate the invention and is not limiting thereof. Reference numerals used in the figures denote identical components and functions.

[0022] They show: Fig. 1 a portable communication terminal according to the invention in a schematic cut-away side view; and Fig. 2 shows a schematic side view of an application of the portable communication terminal according to Fig. 1.

[0023] Fig. 1 shows a schematic side view of a portable communications terminal 10, here a smartphone, with a first camera 12 configured to capture a spatial view of an object 20 using the first camera 12 and to provide first data associated with the view of the captured object 20. The communications terminal 10 further comprises an object recognition unit 18 for processing the first data provided by the first camera 12 and for determining a classification of the object 20. For this purpose, the first data are compared with previously stored classification data for objects. Corresponding classification data for a wide variety of objects are stored in an external database (not shown), which the communications terminal 10 can retrieve via a wireless communication connection, here the LTE network.

[0024] The portable communication terminal 10 has a second camera 14 for two-dimensional recordings for recording a two-dimensional view of the object 20. The second camera 14 provides second data associated with the two-dimensional view of the recorded object 20. The communication terminal 10 further comprises a comparison unit 18, by means of which the second data are compared with pre-stored two-dimensional view data associated with the classified object 20. Finally, the communication terminal 10 comprises an orientation detection unit 22, which determines an orientation of the classified object 20 based on comparison results provided by the comparison unit 18.

[0025] Object 20 is a passenger car as described in Fig. 2. A user 28 carries the portable communication terminal 10, here the smartphone, with them and causes the first camera 12 to create a view of the front of the passenger vehicle 20. The first camera 12 provides a three-dimensional or spatial view of the passenger vehicle 20 and provides first data associated with this image. The first data are fed to the object recognition unit 16, which processes the first data and causes the smartphone 10 to transmit classification data on objects, here motor vehicles, in particular passenger vehicles, via an external database (not shown). The object recognition unit 16 determines a classification of the passenger vehicle 20 by comparing the first data with the previously stored classification data.As a result, the object recognition unit 16 classifies the object 20 as a passenger car, preferably also the type of passenger car.

[0026] However, with the first data, it is not yet possible to determine whether the front or rear section of the passenger vehicle 20 was recorded. The first camera 12 is designed as a TOF camera and only provides a comparatively rough representation in black and white.

[0027] To achieve a more precise classification, the smartphone 10 now has the second camera 14, which provides a conventional camera function of a digital camera. From the same view as before, a two-dimensional image of the passenger vehicle 20 is created with the second camera 14, and second data corresponding to the two-dimensional view of the recorded passenger vehicle 20 is provided. The comparison unit captures the second data and requests pre-stored two-dimensional view data associated with the classified object 20, here the passenger vehicle, from the database (not shown). The second data are compared with the view data, and the comparison result is transmitted to an orientation recognition unit 22 of the portable communication terminal 10.The orientation recognition unit 22 determines an orientation of the classified passenger vehicle 20 based on comparison results supplied by the comparison unit 18. This makes it possible to recognize that the captured view is a front view of the passenger vehicle 20. Now that the object and its orientation have been recognized, further data regarding the front view of the passenger vehicle 20 are requested from the smartphone 10. In the present case, corresponding data regarding the engine type is requested and simultaneously displayed for the user 28 on a screen (not shown) of the smartphone 10 with a superimposed view of the first and second data. The user 28 can then save this display in a storage unit (not shown) of the smartphone 10.

[0028] For the purpose of the superimposed display, the scales of the first and second data are adjusted and a correction is made regarding the position using data from the position sensor.

[0029] Of course, the user 28 can also move around the vehicle and initiate new recordings with the second camera 14. Accordingly, a comparison of new second data with the previously requested view data is carried out again. For example, if the Fig. If the left front wheel shown in Figure 2 is captured by the second camera 14, the comparison unit and the orientation unit recognize that it is the front left wheel of the passenger vehicle 20. The smartphone 10 then requests corresponding data about the wheel from the database, for example, dimensions of the tire and rim, tire pressure, and the like. This data is in turn displayed to the user 28 on the screen of the smartphone 10.

[0030] The exemplary embodiment serves merely to illustrate the invention and is not limiting it. Naturally, functions, particularly with regard to operation, can be designed as desired without departing from the spirit of the invention. Furthermore, the invention is not limited to motor vehicles, but can be used for any object, including buildings, landscapes, or the like. The invention combines the advantages of a conventional second camera with the advantages of a first camera that captures a three-dimensional view.

[0031] Finally, it should be noted that the advantages and features, as well as embodiments, described for the device according to the invention apply equally to corresponding methods and vice versa. Consequently, corresponding method features can be provided for device features and vice versa.

Claims

[1] Portable communication terminal (10) with a first camera (12) which is configured to record a spatial view of an object (20) by means of the first camera (12) and to provide first data associated with the view of the recorded object (20), with an object recognition unit (16) for processing the first data provided by the first camera (12) and determining a classification of the object (20) by comparing the first data with pre-stored classification data for objects, and with a second camera (14) for two-dimensional recordings for recording a two-dimensional view of the object (20) and for providing second data associated with the two-dimensional view of the recorded object (20), a comparison unit (18) for comparing the second data with pre-stored,two-dimensional view data associated with the classified object (20) and an orientation recognition unit (22) for determining an orientation of the classified object (20) based on comparison results supplied by the comparison unit (18), wherein the portable communication terminal (10) is designed to select one of the recorded objects by means of a user input when there are several objects (20) recorded by the first camera (12) and to determine supplementary data for the object (20) corresponding to the determined orientation and to output said data by means of the output unit (24), in particular superimposed on the superimposed first and / or second data. [2] Communication terminal according to claim 1, characterized by that the orientation recognition unit (22) is communicatively coupled to a database, which database provides supplementary data for the objects. [3] Communication terminal according to claim 2, characterized byan output unit (24) for outputting the supplementary data. [4] Communication terminal according to one of claims 1 to 3, characterized by a position sensor (26) for detecting a position of the communication terminal (10). [5] Method for determining an orientation of an object (20) by means of a portable communication terminal (10) having a first camera (12) for three-dimensional recording, comprising the steps: - recording a spatial view of the object (20) by means of the first camera (12) and providing first data associated with the view of the recorded object (20), - processing the first data provided by the first camera (12) by means of an object recognition unit (16), by means of which a classification of the object (20) is determined by comparing the first data with previously stored classification data on objects, - recording a two-dimensional view of the object (20) by means of a second camera (14) for two-dimensional recordings and providing second data associated with the two-dimensional view of the recorded object (20), - comparing the second data with previously stored two-dimensional view data associated with the classified object by means of a comparison unit (18), - Determining the orientation of the classified object by means of an orientation recognition unit (22) based on comparison results supplied by the comparison unit (18), wherein - if there are several objects (20) recorded by the first camera (12), one of the recorded objects is selected by means of a user input, and - supplementary data for the object (20) corresponding to the determined orientation are determined and output by means of the output unit (24), in particular superimposed on the superimposed first and / or second data. [6] Method according to claim 5, characterized by that the first and second data are output in a superimposed manner by means of an output unit (24). [7] Method according to one of claims 5 or 6, characterized by that classification data is only taken into account from given objects. [8] Method according to one of claims 5 to 7, characterized by that a position of the communication terminal (10) is taken into account.

Images