Method and apparatus for generating a communication channel - Patents.com

Abstract

The present invention relates to a method for configuring a mobile terminal of a user in a semi-automated arrangement of communication channels. The proposed method allows preselecting communication partners and generating communication channels with minimal technical complexity as a function of hardware parameters such as screen size and camera resolution. The technical complexity in terms of computational capacity, storage capacity and storage performance is reduced compared to known methods so that the proposed invention can be implemented, for example, in a smartphone. The invention is also directed to a correspondingly configured device. Also disclosed is a computer program product having control commands for implementing the disclosed method and for operating the disclosed device and arrangement.

Description

[Technical field]

[0001] The present invention relates to a method for generating a communication channel by configuring a user's mobile terminal in a semi-automatic setup. The proposed method allows preselecting a communication partner and establishing a communication channel with low technical effort depending on current hardware parameters such as screen size and camera resolution. The technical effort in terms of computational capacity, memory capacity and memory performance is reduced compared to known methods to such an extent that the proposed invention can be implemented on a smartphone. The invention is also directed to an apparatus configured accordingly. Furthermore, a computer program product is proposed having control instructions for implementing the proposed method or for operating the proposed apparatus and arrangement. [Background technology]

[0002] Patent document 1 discloses a head-up display system including a display unit including a projection unit arranged to be positioned adjacent to a windshield, the display unit further including a display screen and a control element.

[0003] US Patent No. 5,399,633 refers to different levels of panoramic views with performance of interactive digital media presentations with multiple views. US Patent No. 5,399,633 shows a method of image recognition with time delay, where a preview image is generated.

[0004] The prior art shows surveillance systems that can monitor large events, for example by pointing a camera at the side of a building. The camera is mounted on the ceiling so that it can see the area being monitored. This means that the prior art allows monitoring of a particular area regardless of the number of people present. This unlimited number of people results in a larger amount of data, assuming sufficient computing power is made available. Mobile devices in particular tend not to have the necessary computing power.

[0005] The prior art also assumes that the surveillance cameras themselves do not process the data, but rather transmit the captured image data to a remote computing unit, which usually has extensive hardware capabilities. The data is transmitted to at least one server, which takes over the data processing. In this respect, surveillance cameras are usually passive image sensors that provide image data according to their own viewing angle. Since active control units are installed at a distance from these cameras, the surveillance personnel must manually input the viewing angle and zoom factor into the cameras and perform the surveillance remotely. The prior art does not show data capture and processing on a single mobile device. There is no limit to the number of people, so many people can be recorded at once. All people and objects present in the camera's field of view are captured.

[0006] The prior art also shows that it is possible to identify individual people or objects from a number of objects in an image. For example, people are stored in a database and when a photograph of the person is taken, this photograph can be compared with the recorded data. If there is a match, information can be given indicating the specific location of the person.

[0007] The prior art also shows a facial recognition function that recognizes faces from an image, even if they are not stored in a database beforehand. In a typical smartphone, this is shown by marking the face on the photo with a rectangle. In this way, the number of people in the image can be determined.

[0008] Furthermore, the prior art shows the ability to perform object recognition during autonomous driving by pointing a camera at the road ahead, which allows for the interpretation of traffic signs, for example speed limits. Typically, only a small number of traffic signs are in the field of view of the on-board camera, so current hardware does not pose a problem. This image information is evaluated inside the car, so there is no technical problem of too much data. Also, in this scenario, passive cameras are generally proposed, capturing only traffic signs, which are then interpreted. [Prior art documents] [Patent documents]

[0009] [Patent Document 1] German Patent No. 102015214744 [Patent Document 2] DE 112014005165 A1 [Patent Document 3] US Patent Application Publication No. 2010 / 173678 Summary of the Invention [Problem to be solved by the invention]

[0010] The prior art faces the problem that there may be a potentially large number of people or objects to be recognized in the image source. Especially the hardware capacity of the mobile device is limited and often the bandwidth is also limited. The drawback of the prior art is that it is often not possible to transmit huge image data to a remote server. On the other hand, if the data is processed in the mobile device itself, extensive hardware capacity is required in the mobile device. The hardware capacity here refers to the processor performance, memory size and speed.

[0011] Furthermore, a problem with the current state of the art is that it uses a lot of battery power, especially with extensive image data, which can lead to heating in the terminal device and reduced battery life.

[0012] The aforementioned problems are exacerbated by the fact that image processing generally has real-time requirements, i.e., processing must occur immediately from the user's subjective point of view. In particular, camera movement generates large amounts of image data that must be processed immediately. [Means for solving the problem]

[0013] It is therefore an object of the invention to propose a method which allows a user with little technical effort to display in real time on his mobile device people or objects that he can contact. The object here is that the hardware requirements are so low that even mobile devices have this capability. It is further an object of the invention to provide a correspondingly equipped device. It is also an object of the invention to provide a computer program product with control commands for implementing the method or for operating the proposed device.

[0014] This object is solved by a method having the features of claim 1. Further advantageous embodiments are given in the dependent claims. Therefore, a method is proposed for configuring a user's mobile terminal for a semi-automatic setup of a communication channel. The method reads out at least one image sensor continuously to detect a number of potential communication partners. Depending on a pre-set number of potential communication partners, a zoom factor of the image sensor, a magnification factor or a focus value of at least one sensor of the plurality of image sensors is set. A selection of a communication partner is enabled by the provided registration information. A communication channel is set from a pre-defined set of communication partners. A focus value of at least one sensor of the plurality of image sensors is set. A selection of a communication partner is enabled by the provided registration information. A communication channel is set from a pre-defined set of communication channels to the communication partner's terminal according to a stored terminal identification or user ID of the enabled communication partner as a function of a user input of the user's mobile terminal.

[0015] The magnification factor can be physically set in the camera or simulated programmatically. If you don't have a hardware compatible lens, you can choose a virtual screen section that is zoomed, i.e. enlarged. Simple digital cameras without zoom lenses or cameras without a compatible lens can also be operated in such a way that they only use one image section in normal operation, i.e. the screen section can be virtually enlarged. Also the zoom factor or magnification factor (magnification factor) can be adjusted programmatically. Adjusting the magnification factor by an application means enlarging or reducing the image section.

[0016] The proposed method configures a mobile device such that a camera or image sensor is configured. The mobile terminal is, for example, a smartphone of the user, which is assisted in the configuration of the communication channel. As this is semi-automated, the user does not have to select from a predefined list, such as a phone book. Rather, the user uses the end device to scan the crowd and also to position the camera. The user is then shown potential communication partners, which he can select individually. For example, the user points the camera at a group of 20 people. All 20 users may be registered in the basic service. If only 15 users are registered in the proposed service, the magnification, focal length, or zoom factor is set so that a pre-set number of potential communication partners are displayed individually, i.e. exclusively. If the user inputs that he would like to see 5 potential communication partners, the zoom factor is set so that only 5 are displayed on the smartphone screen. These 5 people may or may not be registered in the underlying service. Possibly, all 5 people have saved their registration information, or only 3 out of the 5 people are registered. Registered people in groups of five are then shown which contacts they can contact, and once the user selects an actual contact, the potential contact becomes the actual contact.

[0017] For example, the user may see three potential communication partners in the enlarged image, with their faces displayed using rectangles. If there are five people in the image, but only three are registered, the user will see three rectangles for each potential communication partner's face. The user then taps on the rectangles to set up or establish a communication channel.

[0018] People can be detected by successively reading out at least one image sensor to detect the number of potential communication partners. The camera of the mobile end device is activated to capture images continuously. It is also possible to use multiple cameras that merge images together. Then, using image recognition, it can be determined how many people and objects are in each image. The user positions the camera to detect different numbers of communication partners. At this stage, these are still only potential communication partners. The camera images are therefore a continuous sequence of individual images that have been analysed.

[0019] The user has memorized the number of potential communication partners beforehand, and it is exactly this number that is displayed. For this purpose, in the proposed method, the magnification factor, focal length, or zoom factor of the image sensor are automatically set.

[0020] Specifically, a user may have specified five potential communication partners in advance. The user simply specifies this number. Now, when the user points the camera at a group of ten people, the image is adjusted so that only five people appear in the image. Due to the size (dimensions) of a smartphone screen, the number of potential communication partners cannot usually be set to more than ten. Since each communication partner is displayed for selection with a rectangle placed around their face, for example, the number of potential communication partners is severely limited.

[0021] According to the invention, it has been shown that the ideal number of potential communication partners is between 3 and 10 in this method. The faces can be displayed in such a way that the user can actually recognize the faces, i.e. identify the people. Moreover, the person or communication partner can be selected by manual input. Usually, the user selects the communication partner by tapping on the screen, which is not possible for more than 10 people.

[0022] That is, the dimensions of the screen are taken into account when controlling the camera or one or more sensors. If the number of potential contacts is displayed and the user moves the mobile device, the system branches back to the continuous reading process, so that the image section or zoom factor, magnification, or focal length is reset. For example, if only three potential contacts are displayed instead of the desired five, the system zooms out until the number of five potential contacts is displayed.

[0023] The selection of communication partners is then activated using the provided registration information. For example, if five potential communication partners are displayed, but only three are registered in the underlying service, then only these three are activated. Activation here means activating the creation of a communication channel. Thus, a communication channel can be established for these activated persons or objects. It is also possible to limit the number of potential communication partners so that only activated, i.e. registered, individuals or objects are displayed. For example, if five communication partners are displayed, but only three are registered, then the zoom factor is set so that at least three activated or registered communication partners are present in the processed image.

[0024] The registration information can be provided in such a way that user data is stored on a remote server and indicates which person or object would like to be contacted. Contact information can also be stored for this purpose. For example, a user can be registered with a photo and store an address for setting up a video channel. If a person in the image being processed is recognized by face recognition, it can be indicated that a video channel can be set up.

[0025] Thus, a potential communication partner can specify which communication channel he wants to use, for example by saving a phone number or by using a text service instead. A user who performs the proposed procedure is then shown that he is enabled to have a phone call or a text conversation or a chat with a potential communication partner, the potential communication partner being the person who is addressed by using a saved terminal device.

[0026] The proposed method generally refers to a human user communicating with a technical device. For example, the user is traveling in a car and the camera detects a certain number of license plates. The proposed method adjusts the camera so that the certain number of license plates are visible. The user (in this case the driver) then sees on his screen the license plates of the (other) drivers with whom he wants to communicate. For example, if a first driver and a second driver are registered with the proposed method, the drivers are allowed to contact each other on the highway such that the first driver's head-up display shows the other five drivers. This comparison is performed by image recognition of license plates. The first driver then selects a displayed potential communication partner by making a voice input. This potential communication partner becomes the actual communication partner by providing a communication channel with this driver.

[0027] The registration information may also include characteristics relating to potential communication partners. For example, the registration information may be used to enter data relating to communication partners. A potential element of the registration information is a location identifier of the potential communication partner. That is, someone may indicate that they only want to be contacted by people with license plates corresponding to the city in which they live. Thus, the registration information may include information that the driver only wants to communicate with people in a certain city. The corresponding services may be expanded as required. Thus, for example, the driver may set that only potential communication partners with a certain relationship status are displayed to him. In this way, it is technically easy to contact other people even if only a limited amount of screen space is available. The image sensor may be set to ensure that only a certain number of potential communication partners are displayed.

[0028] A particular advantage here is that potential contacts are automatically recognized and the camera is automatically configured, i.e. potential contacts are displayed without any input from the user. All the user has to do is align his / her mobile terminal and a suitable image with the contact is displayed. It is ultimately up to the user to choose which potential contacts he / she wants to contact.

[0029] According to one aspect of the invention, multiple image sensors are read out and the images are displayed on a display device. This has the advantage that a perspective or three-dimensional representation can be displayed. By spatially locating communication partners in this way, a spatial image can be created. Furthermore, further information can be added to the image, such as additional information about the communication partners or highlighting of the communication partners. Current mobile devices and cars usually have multiple cameras, so that these cameras can be combined.

[0030] According to a further aspect of the invention, the communication partners are detected according to the stored registration information, the stored image information, the read location sensor, the altimeter and / or the barometric pressure sensor. This has the advantage that only registered communication partners can be reached using the registration information. The image information can be compared with the actual image from the camera by face recognition or pattern matching, so that the user can be identified. It is also possible to specify the location of the communication partner. Here, spatial restrictions can be implemented, for example by defining a radius. The geographical location of the user can also be determined by the altimeter and the barometric pressure sensor. The barometric pressure sensor provides information about the altitude. This makes it possible, for example, to restrict the geographical area in which the communication partners are located. For example, only communication partners within a radius of 15 meters or even more are relevant.

[0031] According to a further aspect of the invention, the preset number of potential communication partners is specified as a function of the resolution and / or screen occupancy of the display device. This has the advantage that it is possible to specify how many communication partners are to be displayed for each resolution of the mobile terminal device. This makes it possible to reserve enough space to display the communication partners so that the user can actually identify and select them. In this way, a list of different potential resolutions of the screen and / or camera can be created and each resolution can be assigned a value indicating the number of potential communication partners. For example, for a standard screen resolution, five communication partners can be displayed. For higher resolutions, seven communication partners are also possible. In this way, the camera is set taking the screen into account and always the appropriate number of communication partners is displayed. The screen occupancy can be defined, for example, such that 15% of the screen is reserved for one communication partner. Since above 90%, 15% is not possible, only 90% of the screen occupancy is used and six communication partners are displayed. It is also possible to display only five communication partners on the entire screen by setting the screen occupancy to 20%. The camera is set so that its focal length or zoom factor captures just this number of parties.

[0032] According to a further aspect of the invention, the number of potential communication partners is set to a selection between 3 and 10. This has the advantage that a number of communication partners can be displayed on a typical screen size of a smartphone, allowing the user to identify the communication partners and to select the communication partners accordingly by touch screen input. Surprisingly, according to the invention, 3 to 10 has been found to be a suitable number for communication partner selection with typical camera configurations and screens. Empirically, it has been determined that if the number of communication partners displayed exceeds 10, the display becomes cluttered and it becomes technically impossible to accurately select communication partners using a touch screen and finger input.

[0033] According to a further aspect of the invention, the magnification or zoom factor is set such that the retrieved potential communication partners exactly correspond to a pre-set number of potential communication partners. This has the advantage that the camera is set with respect to the zoom factor such that the captured image is reduced or enlarged until the pre-set number of communication partners is present in the image. Thus, the camera is automatically adjusted.

[0034] According to a further aspect of the invention, the registration information comprises at least one image, at least one pattern, information on the license plate, facial features, biometric data, and / or at least one visual user identification. This has the advantage that existing communication partners can be compared with the stored image. If there is a sufficient match, it is assumed that the person in the stored image is actually in the real-time image taken. If the communication partner is not a person, a pattern can also be stored. The pattern can be easily captured using image recognition. Also, a car can be selected as a communication partner by optically storing part or all of the license plate. For example, if a driver wants to send a warning, the driver can enter the license plate information in text form as registration information, and other drivers can accurately select the car. Thus, a warning can be displayed on the display of the driver of the car.

[0035] According to a further aspect of the invention, the set of communication channels comprises an audio channel, a video channel, and / or a text communication channel. This has the advantage that a user can select from multiple formats and therefore set the appropriate communication channel. The communication channel can be a text conversation or chat, or an audiovisual message exchange.

[0036] According to a further aspect of the invention, the terminal or user identifier comprises an address, a telephone number, contact data and / or an alphanumeric string. This has the advantage that the terminal can be addressed in different ways and also that different terminal identifiers can exist. In addition to the telephone number, an address for video communication can also be stored. The user can choose how the communication channel is established.

[0037] According to a further aspect of the invention, user input of the mobile terminal device is provided by touching a touch-sensitive display unit and / or by voice input, which has the advantage that the user can specify which communication partner to select by selecting the communication partner on the display or by voice input in the car.

[0038] According to a further aspect of the invention, the mobile terminal device of the user is in the form of a telephone, electronic glasses with a user interface and / or a head-up display. This has the advantage that different hardware can be used. For example, the user can use electronic glasses that can display information on the lenses of the glasses. In this way, the user receives different information about the multiple communication partners that surround him. Furthermore, information indicating which communication partners are registered can also be used. Furthermore, by displaying a camera image on the inner display of the glasses, it is often also possible to zoom in or out on the communication partners with a desired magnification (factor).

[0039] According to a further aspect of the invention, the communication channel is established using Bluetooth, WLAN, the air interface of a mobile phone network, a point-to-point connection, and / or a telephone connection. This has the advantage that different communication technologies can be used, as well as a second communication technology can be used in case of failure of a first communication technology. Furthermore, these technologies can be prioritized, so that in case of failure of a higher priority technology, a lower priority technology can automatically take over.

[0040] This object is also solved by a device for configuring a mobile terminal of a user in a semi-automatic setup of communication channels. The device comprises a read-out unit configured for continuously reading out at least one image sensor for detecting the number of potential communication partners. The magnification of at least one of the image sensors is configured by the setting unit as a function of a pre-set number of potential communication partners. The device comprises an enabling unit configured for enabling the selection of communication partners by means of the provided registration information. The device comprises a communication interface configured for setting communication channels from the pre-set number of communication channels to the terminal devices of the communication partners according to the stored user IDs or terminal device IDs of the enabled communication partners. The terminal device identifiers of the enabled communication partners depend on a user input of the mobile terminal device of the user.

[0041] The terms magnification factor and zoom factor may be used interchangeably herein. The object is also solved by a computer program product comprising control commands for performing the proposed method or for operating the proposed apparatus.

[0042] According to the invention, it is particularly advantageous that the method can be used to operate the proposed devices and units. Furthermore, the proposed devices and units are suitable for implementing the methods according to the invention. The devices thus in each case implement structural features that are suitable for implementing the corresponding methods. However, the structural features can also be designed as process steps. The proposed methods also provide steps for carrying out the functions of the structural features. Furthermore, physical components can also be virtually provided or virtualized.

[0043] Further advantages, features and details of the present invention are apparent from the following description, which describes in detail the aspects of the present invention with reference to the figures. The features described in the claims and the specification may each be essential to the present invention, individually or in combination with each other. Similarly, the features described above and further described herein may be used individually or in combination with each other. Functionally similar or identical parts or components may be provided with the same reference signs. The terms "left", "right", "upper" and "lower" used in the description of the embodiments refer to the views of the drawings having legible figure designations or legible reference signs. The illustrated and described embodiments are not definitive and are used as examples to explain the present invention. The detailed description is for the reference of those skilled in the art, and therefore known circuits, structures and methods are not shown or described in detail herein, so as not to impede the understanding of the present specification. The figures are shown as follows: [Brief description of the drawings]

[0044] [Figure 1] 2 is a schematic flow chart of a proposed method for setting up (configuring) a user's mobile terminal in a semi-automatic setting of a communication channel according to an aspect of the present invention; [Diagram 2] FIG. 2 illustrates a user-orientated field of view, divided horizontally and vertically, in accordance with one aspect of the present invention. [Diagram 3] 1 illustrates a user-based field of view according to one aspect of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0045] Fig. 1 is a schematic flow chart showing a method for configuring a mobile terminal of a user in a semi-automated setup of communication channels. The method comprises a continuous reading (100) of at least one image sensor for detecting (102) a number of potential communication partners. A magnification factor of at least one of the image sensors is set (103) according to a pre-set (101) number of potential communication partners. The method further comprises enabling (104) a selection of communication partners by the provided registration information. The method further comprises setting (105) communication channels from the pre-set number of communication channels to the communication partner terminals according to the stored terminal identification information of the enabled (104) communication partners according to a user input of the mobile terminal of the user.

[0046] Figure 2 shows the field of view with the user at the base and divided into horizontal and vertical directions. The virtual centerline corresponds to the azimuth angle, i.e. the angle with respect to the North Pole. The compass direction is determined using a magnetometer or compass. The horizontal centerline is determined by the vertical orientation of the mobile phone, which is determined using a rotation sensor and / or a tilt sensor.

[0047] Where: W = width of field of view. W / 2 = half the width of the field of view.

[0048] H = field of view height. H / 2 = half the height of the field of view. r = distance (field of view radius) to the camera or home position.

[0049] a = vertical aperture angle of the camera. a / 2 = half the vertical aperture angle of the camera. b = horizontal aperture angle of the camera.

[0050] b / 2 = half the horizontal aperture angle of the camera. COmin = minimum value of geo coordinate. COmax = maximum value of geocoordinate.

[0051] HP=Home Position. Figure 3 shows the field of view from the user. To speed up data storage, the field of view is shown as a closed polygon. The accuracy of the altitude information from the GPS signal depends heavily on the quality of the received signal and the number of satellites received. If available, a signal from a pressure sensor (barometer) is used to improve the altitude information. The pressure at the target's location is also known, so that the pressure can be used to calculate the target's altitude in relation to the user's own altitude. To do this, sensor data such as pressure and temperature, GPS position and signal quality, etc. need to be sent to the server.

[0052] Where: P0=home position. P1~P5=geocoordinates (calculated in the WGS84 georeferencing system) at distance r and angle (COA / 4*X) from P0.

[0053] r=field radius. X=coordinate index. Azimuth = Azimuth angle.

[0054] COA=Horizontal Camera Aperture Angle (Aperture Angle). COA / 4 = 1 / 4 of the horizontal camera aperture angle. Polygon coordinate calculation:

[0055]

number

[0056] The function "at()" calculates the coordinates of a point P0 at a distance r and an azimuth angle according to the WGS84 georeferencing system.

[0057]

number

[0058] Not shown here is a data memory or computer readable medium with a computer program product having control instructions for performing the proposed methods or operating the proposed system arrangements.

Claims

1. A method for configuring a user's mobile terminal in a semi-automatic setup of a communication channel, the method comprising: a continuous reading step (100) by at least one image sensor for detecting (102) the number of potential communication partners, wherein the magnification of at least one of the image sensors is set (103) as a function of a preset number (101) of the potential communication partners, and the preset number of the potential communication partners is specified according to the resolution of a display device and the screen occupancy rate of the display device, the continuous reading step (100); a step of enabling (104) the selection of a communication partner based on provided registered information; and a step of setting (105) the communication channel from the preset number of communication channels to the terminal device of the communication partner according to the stored terminal device identifier of the enabled (104) communication partner in response to a user input of the user's mobile terminal device; A method comprising the above.

2. Some of the image sensors are read, and their images are combined and displayed on a display device, The method according to claim 1.

3. The communication partner is detected by at least one of stored image information, as a function of stored registered information, as a function of a read position sensor, as a function of an altimeter, and as a function of a barometric pressure sensor. The method according to claim 1.

4. The number of the potential communication partners is set by a selection from 3 to 10. The method according to any one of claims 1 to 3.

5. The magnification is set such that the read potential communication partners exactly correspond to the preset number of the potential communication partners. The method according to any one of claims 1 to 3.

6. The registered information is configured to include at least one of at least one image, at least one pattern, information from a number plate, facial features, biometric data, and at least one visual user ID. The method according to any one of claims 1 to 3.

7. The number of the communication channels includes at least one of an audio channel, a video channel, and a text communication channel. The method according to any one of claims 1 to 3.

8. The terminal device identifier includes at least one of an address, a telephone number, a user ID, contact data, and an alphanumeric string. The method according to any one of claims 1 to 3.

9. The user input of the mobile terminal device is performed by at least one of touching a touch-sensitive playback unit and voice input. The method according to any one of claims 1 to 3.

10. The user's mobile terminal device is in at least one form of a telephone, electronic glasses with a user interface, and a head-up display. The method according to any one of claims 1 to 3.

11. The communication channel can be established by using at least one of Bluetooth (registered trademark), WLAN, an air interface of a mobile radio network, a point-to-point connection, a Wi-Fi (registered trademark) direct connection, and a WLAN direct connection. The method according to any one of claims 1 to 3.

12. In a semi-automatic setup of a communication channel, an apparatus for setting a user's mobile terminal, the apparatus comprising: A reading unit configured for continuous reading (100) of at least one image sensor for detecting (102) the number of potential communication partners, wherein the magnification of at least one of the image sensors is set (103) by a setting unit as a function of a preset (101) number of the potential communication partners; An enabling unit arranged to enable (104) the selection of a communication partner based on provided registration information; A communication interface for setting (105) the communication channel from the preset number of communication channels to the terminal device of the communication partner according to the stored terminal device identifier of the enabled (104) communication partner in response to a user input of the user's mobile terminal device, wherein the preset number of the potential communication partners is specified according to the resolution of a display device and the screen occupancy rate of the display device; An apparatus comprising.

13. A computer program product comprising instructions for instructing a computer to execute the steps of the method according to any one of claims 1 to 3 when the program is executed by at least one computer.

14. A computer-readable storage medium comprising instructions that, when executed by at least one computer, direct the computer to perform the steps of the method according to any one of claims 1 to 3.

Images