Position determination method or system for position determination of an article

Abstract

Method for determining the position of at least one object (P1-P6), in particular at least one product positioned in a shelf (7), in a space (2), in which the method comprises: emitting a light signal from a light-emitting device (6) positioned adjacent to the object, the light signal being encoded with identification information for identifying the object, and detecting the scene by means of at least one freely movable camera (10), a digital scene image generated by means of the camera representing the scene, and generating object position data on the basis of a computerized position of a light signal occurring in the digital scene image relative to the camera and identification information emitted by means of the light signal and knowing an automatically determined orientation and position of the camera in space during the detection of the scene, wherein the object position data represents the position of an object in the scene relative to the space.

Description

Technical Field

[0001] The invention relates to a method and a system for determining the position of an object, in particular for detecting the position of a product. Background Art

[0002] A system for detecting the location of a large number of products in a business premises is known from US2021073489A1. However, since all shelves must be equipped with a special device for identifying the shelves, the so-called "identity information providing device", the system and the method disclosed in conjunction with the system have proven to be extremely complex, expensive and cost-intensive in practice. This special device is generally disclosed as a wireless communication device, which can be used as a separate device or as a component of an electronic price display. The device or the information conveyed thereby for identifying the shelf must be detected compulsorily in the case of this method.

[0003] The present invention therefore has the object of avoiding the problems discussed in the context of a system or a method carried out thereby. Summary of the invention

[0004] This object is achieved by a method according to claim 1. The subject of the invention is therefore a method for determining the position of at least one object in a space, in particular at least one product positioned in a shelf, wherein the method comprises emitting a light signal from a lighting device positioned adjacent to the object, the light signal being encoded with identification information for identifying the object, and detecting a scene by means of at least one freely movable camera, wherein a digital scene image generated by means of the camera represents the scene, and generating object position data based on a computerized determined position of the light signal appearing in the digital scene image relative to the camera and the identification information emitted with the light signal and knowing the automatically determined orientation and position of the camera in the space when detecting the scene, wherein the object position data represent the position of the object in the scene relative to the space.

[0005] This object is also achieved by a freely movable device according to claim 12. The invention therefore relates to a freely movable device, in particular a shopping cart or glasses or a mobile phone or a tablet, for determining the position of at least one object in space, in particular at least one product located in a shelf, wherein the position is displayed by a light signal emitted by a lighting device located adjacent to the object, the light signal encoding identification information by means of which the object can be identified, wherein the device carries at least one camera, which is designed to generate a digital scene image of a scene detected by the camera, and wherein the device, in particular the camera, is designed to determine in a computerized manner the position of the light signal occurring in the digital scene image relative to the camera and the identification information emitted by the light signal, and wherein the device, in particular the camera, is designed to at least assist in the automated determination of the orientation and position of the camera in space when detecting a scene.

[0006] This object is also achieved by a lighting device according to claim 13. The invention therefore relates to a lighting device, in particular a shelf label, which is designed to display product and / or price information, or a product separator, which is designed to separate different products, wherein the lighting device has a storage stage for storing identification information, by means of which objects, in particular products located in a shelf, can be identified, in the vicinity of which the lighting device is located, and a light signal generation stage, which is designed to encode a light signal as a function of the identification information and to emit the encoded light signal.

[0007] This object is also achieved by a system according to claim 14. The invention therefore relates to a system for determining the position of at least one object in a space, in particular at least one product positioned in a shelf, wherein the system has at least one lighting device according to the invention and at least one freely movable device according to the invention.

[0008] The measures according to the invention result in the following advantages: a system is created that is to be realized and operated more efficiently and at lower cost, since, compared to the system discussed at the outset, fewer devices are required to be fixedly installed at the shelf, which can be set up and maintained individually. This relates in particular to the fact that the device discussed at the outset, which emits information for identifying the shelf on which the device is installed, is not required. In addition, a significantly more efficient method is created, since in this case the focus is completely on the emitted light signal, which is actually located spatially corresponding to the respective location of the object or product in the shelf or has its origin there and is suitable for identifying the object. In addition, the use of a freely movable camera enables a time-dynamic and location-variable detection of the current location of the object in each case in the business premises. Thus, the position detection of the object in the store is possible without the camera-based detection of the object itself or the camera-based detection and evaluation of product-specific information conveyed by the object or shelf label, which product-specific information, for example, can be found in a digital record of the object (digital photo or video) or in a digital record of a paper shelf label or in a digital record of the screen content of an electronic shelf label with a large digital processing effort.

[0009] Further particularly advantageous embodiments and developments of the invention are apparent from the dependent claims and from the following description.

[0010] In general, the present invention enables the position of an object to be determined based on a light signal in the space of a business premises, i.e., the object is positioned. In order to identify the object, the mentioned identification information is used, which is conveyed by means of a light signal. The identification information is obtained by a previous logical association of the object with the light-emitting device assigned to the object. Specifically, in the case of a logical association, a so-called "binding" is involved. Here, a clear identification of the object is established by a clear identification of, for example, an electronic shelf display label (ESL for short, Electronic Shelf Label in English). For this purpose, a barcode scanner to be manually operated is usually used to scan the barcode imaged at the object and the barcode imaged on the screen of the ESL and is stored in a database as closely related. The same content also applies to so-called shelf separators or product separators, which separate different objects or products or product groups from each other in the shelf. The barcode that clearly identifies the separator can be imaged on the label of the separator, or it can also be displayed on the screen of the separator. Usually, either the ESL or the separator is installed in the shelf, but a mixed configuration is also possible. Therefore, the identification information can be a unique identifier of the ESL or separator or also a unique identifier of the product. Of course, other, for example meta information, can also be used here, which represents the logical association that has been created when binding, wherein the product or product group can be inferred again through this meta information.

[0011] The light emitting device that emits a light signal is therefore constructed as an ESL or a separator here. The light emitting device stores the corresponding identification information by means of identification data in its storage level, which is implemented by an EEPROM, for example. The identification data can be transmitted to the corresponding light emitting device from a central data processing device, for example, by radio or also by wire. In addition to other electronic components required for the corresponding functionality, the light emitting device also has a light signal generation level, by means of which a coded light signal can be emitted. For this purpose, the light generated by means of, for example, a light emitting diode is modulated according to the (predetermined) coding to be applied. In this case, the intensity or even the color, i.e. the spectral characteristics of the light, may be affected when applying RGB-LED. The light signal can be located in the visible spectrum of the light. However, it is preferred to use an invisible spectrum range so as not to distract customers and staff from their legacy activities unnecessarily with light signals. The light signal is emitted during the time interval required for at least emitting the entire identification information. The emission can be performed one or more times in sequence, especially according to an external request (for example, by a control signal of a central data processing device or a camera), or automatically for many times, such as, for example, in a manner controlled by an internal timer of the light emitting device.

[0012] As cameras, still-image cameras or video cameras can be used, which are constructed to create two-dimensional digital images of the scene in its detection area, i.e., digital scene images. Regardless of whether it is a still-image camera or a video camera, at least so many individual scene images or videos that last long enough as scene images are detected that an evaluation of identification information can be achieved, which identification information does appear during a time interval. Therefore, when determining the minimum duration of detection, at least the time range necessary for transmitting the identification information is taken into account. In the case of a light signal in which the identification information is encoded over a time period, for example, in the case of a still-image camera, at least so many images are detected within the time period set for transmitting that the identification information can be determined without problems from the changes in the light signal in the scene image.

[0013] The real scene captured by the camera includes the objects or subjects located in the scene and, if necessary, their changes in time or position.

[0014] The digital scene image is the optical imaging of the real scene through the camera's optics (lens optics, also lens) onto the camera's digital image capture unit, for which a so-called CCD is usually used, where CCD stands for "charge-coupled device", and the image of the scene onto it is thus digitized. The scene image is thus imaged onto the pixel matrix of the image capture unit.

[0015] The digital scene image is therefore a two-dimensional data structure acquired by a matrix of pixels of the CCD. Depending on the implementation of the electronic sensor and other post-processing camera electronics, a digital still image or a digital video sequence of the scene is generated, which constitutes the digital scene image. The resolution of the digital post-processing and / or the electronic image sensor defines the image resolution of the digital scene image (the total number of pixels or the number of columns (width) and rows (height) of the raster pattern). Therefore, for further processing, the digital scene image generated by the camera consists of a matrix of pixels, i.e., for example, 1500 pixels in the x-direction and 1000 pixels in the y-direction orthogonal thereto. Each of these pixels has a defined (i.e. known) pixel size (or in other words, the centers of adjacent pixels have a defined (i.e. known) pixel distance.

[0016] When determining the identification information in a computerized manner, a light signal is first searched for in the digital scene imaging. This can be found relatively easily because the light signal is essentially different from the otherwise more likely dark background of the digital scene image. For this purpose, for example, the characteristic color or characteristic intensity or brightness of the image points of the digital scene image can be checked. In order to search for the light signal, its temporal color change or brightness or intensity change to be expected according to the coding can also be used. Once such a light signal has been found, the temporal change (modulation) of the light signal parameters is analyzed in the case of knowing the coding scheme applied when the light signal is emitted, so as to decode the identification information therefrom.

[0017] For further processing of the light signal, the position of the light signal found, ie the pixel coordinates, in the digital scene image and the data representation of the determined identification information are stored in close association.

[0018] By detecting digital scene images using a freely movable camera, different digital scene images of the real scene or the camera's environment are formed over time from the positions and in different orientations, which digital scene images can all be evaluated individually or in the temporal and spatial context of each other in order to determine the position of one, but preferably multiple objects.

[0019] In systems which are preferably used in supermarkets, a relatively large number of lighting devices are typically used, depending on the quantity of different products. In the maximum expansion phase, this will be at least as many lighting devices as there are products or product groups in the relevant store. In such a system, the position of the product or product group is determined entirely based on the position determination of the light signal in a digital scene image and its identification by means of identification information, which digital scene image is acquired over time by means of a camera that is movable or moves in the business premises. The light signal, which can be clearly identified in the corresponding digital scene image by means of the identification information, allows the assignment of real objects, which are located in the space by definition near the lighting device. In order to locate an object in a space (such as a business premises), from which one or more digital scene images are detected by means of a camera, i.e. in order to obtain spatially related object position data, the position of the corresponding light signal is first determined relative to the camera (camera coordinate system), and then the position is transformed into a spatial coordinate system (spatial coordinate system), which has its origin at a defined location in the space and is located there with a defined orientation.

[0020] Various aspects of this generation of object position data are discussed below.

[0021] According to one aspect of the invention, only a single freely movable camera can be used, which is carried by a staff member through a space, such as a business premises. Preferably, the method is performed with the aid of a plurality of cameras that are moved independently of one another. These cameras can be moved independently of one another by the staff of the store, which speeds up the detection process when detecting various scene images in the entire spatial area.

[0022] In order to detect as large a spatial area as possible for each camera, a so-called fisheye lens can be used for this purpose. Of course, the lens or the lens itself can also be designed to be movable, that is, it can be swiveled relative to the camera. Alternatively, the camera can also be swiveled relative to a freely movable device. Alternatively, an omnidirectional camera can also be used.

[0023] For the purpose of moving the camera, the staff can be equipped with (protective) glasses, for example, and the corresponding camera is placed at the (protective) glasses. However, if the freely movable camera is placed at the shopping cart, it has been proved to be particularly advantageous. The following advantages arise, that is, the number of image detection cameras is no longer limited to the number of staff members moving in the store, but the individual movements of customers, which usually far exceed the number of staff members, are used for image detection. In this case, the customer moves a group of image detection cameras in the spatiality of the business premises. With the involvement of customers, the following advantages also arise: the following business areas can be determined from the image detection, that is, in the business areas, the products cause increased customer attention or here the products are actually taken from the shelves by customers, because it can be determined based on the detected scenes that these business areas appear more frequently or because they stay there for a longer time. Thus, important insights for warehousing, inventory detection, planning to supply goods to the shelves, and marketing to target groups or product-specific marketing can be obtained.

[0024] Regardless of whether the freely movable device is now a pair of glasses or the like or a shopping cart, it has proven to be particularly advantageous if a plurality of cameras are arranged at the freely movable device, which cameras move together with the freely movable device, i.e. in groups, and generate individual digital scene images from different detection positions and / or with different detection orientations. This enables: digital scene images of different scenes can be generated with each camera at the corresponding position where the freely movable device is exactly located. Thus, if, for example, one of the cameras is oriented to the left and the other to the right relative to the direction of travel of the shopping cart, i.e. its corresponding detection area is directed to the place where the shelf next to the shopping cart is located, then in the shelf aisle, the shelves that delimit the shelf aisle on the left and right sides of the shelf aisle can be detected at the same time. Of course, the same also applies to the glasses mentioned. In order to also ensure a problem-free, i.e. reliable detection of the entire height of the shelf, especially in the case of a shopping cart, a plurality of separate cameras can be arranged on one or each of these sides. In the case of a shopping cart, the cameras can be arranged one above the other, for example mounted on a rod, or can also be oriented differently, i.e. detecting the upper shelf area, the middle shelf area and the lower shelf area. Even in the case of glasses, a camera oriented toward one of the sides can have different detection areas arranged one above the other, which detect the entire image area from the bottom of the lowest shelf toward the bottom of the uppermost shelf. Alternatively, of course, wide-angle cameras can also be used in order to cover the same or similar total detection area. The individual detection areas can also overlap.

[0025] The ESL or separators forming the light-emitting device are usually operated by means of batteries. Therefore, it has proven to be particularly advantageous if a control signal transmitter that moves with at least one freely movable camera emits a control signal, in particular a radio-based control signal, and the light-emitting device located in the receiving area of ​​the control signal only emits a light signal when receiving the control signal, and the light-emitting device is configured to receive the control signal in a manner equipped with a receiver. In order to receive the control signal, a relatively simple receiver is required, which receiver can also evaluate the unique identifier of the control signal if necessary so as not to react to other radio signals. Thus, the light emission can be limited to a restricted spatial area around the control signal transmitter by a variety of light-emitting devices. Thus, for example, the effective range of the control signal of five to ten meters may be sufficient to activate only the light signal generating stage located in the environment of at least one camera for emitting the corresponding light signal. The effective range of the control signal should be determined so that the detection area of ​​the camera can be optimally utilized. Therefore, at least those light signal generating stages whose light signals can be detected by the camera under reasonable assumptions should be activated. This results in the advantage that the further light signal generating stages can remain inactive, which has a positive effect on the energy budget of the respective lighting device and in particular also on the energy budget of the entire system.

[0026] For this purpose, for example, radio technology can be used, wherein the transmitted control signals are at least partially shielded by the shelves, which are usually constructed of metal, so that mainly those lighting devices located in the same shelf aisle as the freely movable devices can receive the control signals.

[0027] Instead of or in addition to using radio control signals, the lighting device can be designed to emit a light signal only when a certain (time-related) criterion is met. In order to apply this criterion, the lighting device has a timing control stage, which is designed to control the time behavior of the light signal generating stage, i.e. (one or more) active or inactive phases. The light signal generating stage is of course designed to be controllable by the timing control stage and can be controlled by the timing control stage.

[0028] The timing control stage can for example receive time data containing information about the current time and / or have a clock generator, whereby the timing control stage determines the current time or the current date. However, this information can also be obtained by other means, such as a system clock or a system time or a system date that is globally available in the system.

[0029] The criterion may, for example, include that the light signal is emitted only if it has not previously been emitted within a certain time period. But the criterion may also, for example, include that the light signal may only be emitted within a certain time range. This time range may, for example, consist of one or more fixedly predefined time windows during a day or during a week or a month. Thus, the system may, for example, be set up so that the light signal is emitted only on certain working days. For example, the entire business premises may thus be inspected once during the week in order to verify the consistent positioning of the products. The emission of light signals in different shelf aisles in the business premises may also be assigned to different days of the week so that aisles close together do not emit light signals at the same time. Even when glasses are used, the criterion may, for example, be defined so that the light signal is emitted only within that time period in which the staff usually, i.e. foreseeably, walks through the business premises wearing glasses, in order, for example, to prepare the light signal for the next day's operation.

[0030] The method has proven to be particularly advantageous if the determination of the position relative to the camera includes the automatic determination of a dimension which can be derived from the digital scene image, wherein the dimension is determined with knowledge of the real dimensions of a reference object identified in the digital scene image, in particular in the form of an electronic shelf label, and the dimension is used for converting between the position information or the dimension determined in the digital scene image and the position information or the dimension in the scene detected by means of the camera.

[0031] For this purpose, reference objects contained in the scene image, preferably identically constructed reference objects, can be identified, the actual dimensions of which are known, and the dimensions for the scene image can be determined based on the knowledge of the actual dimensions.

[0032] However, in the context of the present invention, the reference objects are particularly preferably the light-emitting devices themselves, which are implemented as ESLs or separators with generally defined, i.e. previously known dimensions. These reference objects can simply be found in the digital scene image, i.e. located or delimited and subsequently analyzed for determining the dimensions, based on the light signal "present" by the reference object, which is indeed emitted by the reference object. In order to identify or find the reference objects in the digital scene image, pattern recognition can be used, which examines the digital scene image with regard to the characteristic appearance of the reference objects, such as a rectangle or square or even a proportion. In any case, the starting point for the search for the one or more reference objects is the light signal, which is generally emitted by the reference object and which therefore makes it particularly easy to find the light signal in the digital scene image.

[0033] However, the reference object may also be a "combined" reference object, which is composed of light signals from different light-emitting devices, for example, light signals from two light signal-emitting devices positioned at the bottoms of different shelves essentially overlapping each other, wherein the distance between the bottoms of the shelves in this case is a known real dimension used to determine the scale.

[0034] As has already been shown, the specific invention is preferably applied in the retail industry. In the retail industry, such a reference object can have a variety of manifestations. For example, it can be an entire shelf, the length and height of which are, for example, known. Since the shelf usually plays a dominant role in the scene image, it may be advantageous to use a shelf. However, smaller objects, such as shelf strips that form the front end of the shelf bottom, can also be used as reference objects. Shopping baskets set up in business premises for displaying goods are also suitable for this, provided that the shopping baskets are positioned in the detection area of ​​the corresponding camera. However, in the retail industry, shelves or shelf strips and shopping baskets are often provided by the most different manufacturers in a variety of sizes for the corresponding business premises of various retailers, often also under the condition of special structural presetting of the retailer. Therefore, the shopping basket is only suitable as a reference object in very narrow application cases.

[0035] Since there is basically a uniform size for electronic shelf labels, it is also very advantageous to use such electronic shelf labels as reference objects. Of course, electronic shelf labels exist in the most different and completely different sizes. However, it has been shown in practice that the sizes applied between different business premises or different retailers hardly change or only change within a predefined range. This is especially applicable to a large number of electronic shelf labels installed at one or more shelf rails of the shelf in a single commercial premises and from a single manufacturer, which is usually the case. Such electronic shelf labels usually exist at the shelf in only one to two (perhaps also three) different sizes. Since each of these electronic shelf labels must be matched with the same shelf rail, the electronic shelf labels usually differ only in width in their size or scale, while the height is often the same for, for example, two different types of electronic shelf labels. However, the opposite situation may also exist. Therefore, the actual size of the electronic shelf label can be classified as uniform across different types of shelf labels and across the installation site.

[0036] The choice of electronic shelf labels as reference objects has also proven to be particularly advantageous for another reason: Unlike the shelves themselves or shelf strips etc., shelf labels are always located at the very front of the shelves and can therefore be easily and unambiguously identified in the digital image captured by the camera by means of digital image processing and can therefore also be reliably evaluated.

[0037] The count of the relevant image points assigned to the reference object forms the basis for the dimension determination. The number of pixels thus determined can be correlated with the real area or real length of the reference object, thereby enabling scaling from the pixel system of the CCD sensor to the real scene. When carrying out the dimension determination, knowledge of the exact physical parameters of the CCD sensor can also be used, so that the actual distance of the image points, the pixel matrix or its size is precisely known. It is thus possible to convert from the counted pixels to the actual length or area measurement for the reference object imaged on the CCD sensor, from which the current dimension can be calculated and then scaling can be performed between the imaged length and area measurement at the CCD and the real scene.

[0038] Specifically, those pixels in the scene image that are assigned to the imaging of the reference object, ie, the pixels of the imaging of the reference object, can be determined by using at least one of the measures listed below, namely:

[0039] - Determine the number of pixels occupied by the image of the reference object in a two-dimensional manner. Thus, by counting the pixels, the area occupied by the image of the reference object in the scene image can be determined (specify the area as the sum of the counted pixels or specify the area as the total pixel area of ​​the counted pixels) and the dimensions can be calculated when the actual area of ​​the reference object is known (e.g. the area of ​​the front surface of the reference object specified in square millimeters).

[0040] - Determine the number of pixels occupied by the reference object image on the circumference side or the number of pixels surrounding the reference object image on the circumference side. The circumference of the reference object image in the scene image is thus determined by counting pixels either based on the pixels that are still occupied at the edge by the reference object image or based on the pixels that are directly adjacent to the reference object image. The dimension can be calculated knowing the actual circumference of the reference object (e.g. the circumference of the front side of the reference object).

[0041] - Determine the number of pixels occupied by the reference object image along one of its boundary lines or the number of pixels surrounding the reference object image adjacent to one of its boundary lines. Therefore, the length of the edge line is used as the basis for the dimension determination. This can be a straight-line edge line, such as a side of a rectangular or square structure of the reference object, which side can be given by the shell side, for example. Therefore, either count the pixels occupied along such a boundary line or count the pixels surrounding the reference object image adjacent to one of its boundary lines. The dimension can be calculated when the actual length of the boundary line of the reference object is known.

[0042] However, the described measures can also define a scale along the scene image, i.e. a scale that is related to the location in the scene image. Such a location-dependent scale may be necessary if, for example, the scene image distorts one of the proportions of the reference objects imaged there. This situation may occur, for example, when recording a scene with the aid of a camera that extends far to the left or right of the camera, which can occur, for example, in the case of aisles in a retail enterprise, in the case of an unfavorable orientation of the camera. Reference objects located near the camera are then imaged larger than reference objects located further away from the camera.

[0043] In conjunction with such perspective imaging, it is also possible to derive the course of the scale along the scene image from the distortion of a single reference object. However, this can usually only lead to improved results if the length of the reference object along the viewing angle to be evaluated is relatively short and there are disadvantageously only a few image points assigned to the reference object.

[0044] Therefore, preferably a plurality of such reference objects are used, which are ideally evenly distributed halfway along the perspective imaging, which is often the case in ESL, in order to be able to define the function describing the location-dependent dimension as accurately as possible. Such a dimension is also called a metric.

[0045] Based on the totality of the light signals identified in the digital scene image, a first data structure can be generated, using a scale for the scene image, which represents a two-dimensional digital map of the light signals in the real scene, specifying the dimension (s) required for two-dimensional mapping (e.g., measured in millimeters). For this purpose, for example, only linked dimension specifications (i.e., relative dimension specifications between adjacent light signals) can be created in order to specify the position of the light signals. It is also possible to create absolute dimension specifications measured from reference points in the scene image. This two-dimensional map obtained in this way, which is stored digitally, is then used to place the two-dimensional map in a three-dimensional context related to the camera, which will be discussed further below.

[0046] According to this aspect, determining the position relative to the camera comprises automatically determining the distance between the camera and the scene detected by the camera. The light signal identified in the digital scene image can thus be located three-dimensionally relative to the camera, i.e. in the camera coordinate system. Thus, the already known two-dimensional location of the light signal in the scene is extended by a third coordinate.

[0047] This distance can be estimated in a substantially known spatial arrangement, such as a shelf aisle, i.e., at, for example, half the aisle width. This distance can also be well limited using the so-called lens equation, since the scale or measure can indeed be defined with good accuracy by means of a reference object found in the digital scene image, the dimensions of which are known very accurately. Thus, the distance between the camera and the corresponding light-emitting device emitting light can be determined by automatic calculation, knowing the parameters of the optical imaging system of the camera. This can be done, for example, completely automatically by the computer of the camera, since the computer can retrieve the parameters of the optical imaging system from its memory, which parameters have been pre-programmed at the memory (for example, when the camera was manufactured or when it was put into operation), and since the computer does know the actual dimensions of the reference object, which are also available, such as pre-programmed. Thus, for example, the distance of the camera from the real object from which the light signal is emitted, i.e., ultimately the distance from the source of the light signal, can be calculated in the scene using the well-known lens equation, wherein in this case, of course, the imaging function corresponding to the actual lens of the camera can be applied. Thus, the position of the light signal can be determined in the spatial context of the camera coordinate system. It should also be mentioned here that the distance between the camera and the lighting device can also be determined by automatic determination with the aid of a distance sensor, wherein a lidar sensor or the like can be used for this purpose. Thus, an accurate direct distance measurement is possible, wherein the computer of the camera further processes the data transmitted by the lidar sensor in order to obtain a third coordinate in the spatial context of the camera coordinate system. It should also be mentioned that so-called "time of flight" sensors can also be used in order to automate this distance measurement.

[0048] Since the camera coordinate system changes its position and orientation as a function of the camera movement in space, but the position of the lighting device in space remains unchanged, the three-dimensional coordinate description calculated for a specific lighting device in the camera coordinate system has a variability or dynamics predetermined by the movement of the camera.

[0049] According to another aspect, it has proven to be advantageous if the method step of emitting light signals is performed substantially simultaneously by a plurality of light-emitting devices, and wherein the method step of determining the position of the light signals occurring in the digital scene image relative to the camera and the identification information emitted by the light signals in a computerized manner is performed for all light signals occurring in the digital scene image. This results in the advantage that a large number of light signals contained in the scene image are used collectively in order to thereby perform position determination for a large number of light signals in a single detection range of the camera. This accelerates and optimizes the position determination process.

[0050] After obtaining the three-dimensional position data of the light signal in the camera coordinate system, the three-dimensional position data is transformed into the spatial coordinate system of the space in order to thereby obtain object position data, since indeed each light signal identifies an object adjacent to the location where the light signal is emitted. Here, according to another aspect of the method, supplementary data is taken into account for generating the object position data, the supplementary data having at least one of the following types of data, namely:

[0051] - orientation data describing the orientation of the camera relative to a reference orientation in space;

[0052] - tilt data, which describes the tilt of the camera relative to a reference plane in space;

[0053] - Position data describing the position of the camera relative to a reference point in space.

[0054] These supplementary data can be generated with the aid of various sensors present on a camera or on a freely movable device, such as a shopping cart or a pair of glasses, as will be discussed below.

[0055] The orientation of the camera in the spatial coordinate system can be determined automatically by means of an orientation sensor, for which, for example, an electronic compass can be used, and the camera's computer further processes the data transmitted by the electronic compass or provides the data for conversion between coordinate systems. In addition, the inclination of the camera with respect to the horizontal plane represented by inclination data can also be understood as a component of the orientation. The inclination can be detected by means of an automatic determination by means of an inclination sensor, for which, for example, an electronic gyroscope can be used, and the camera's computer further processes the data transmitted by the electronic gyroscope or provides the data for conversion between coordinate systems.

[0056] The position of the camera in the spatial coordinate system can be determined by automatic radio-based position determination, in particular by means of “ultra-wideband radio technology” (UWB radio technology for short), wherein for this purpose preferably a fixed (at different points in the relevant spatial region) mounted UWB transmitter with a known position in the spatial coordinate system is used, and the camera has a UWB radio module, by means of which the position of the camera relative to the UWB transmitter can be determined in the case of a corresponding UWB transmitter in UWB radio communication with the camera, and position data are generated from this, which are further processed by a computer of the camera or are provided for conversion between coordinate systems. Since the camera used is preferably of compact design, the position of the camera in the spatial coordinate system determined in this way can be equated with the origin of the camera coordinate system in a good approximation. Otherwise, a correction to take into account the difference would have to be carried out.

[0057] The path traveled can also be detected by means of sensors (for example fastened to the rollers of a shopping cart, which detect the rolling movement of the rollers or wheels) or sensors for detecting acceleration, from which the path traveled and the direction of travel can be determined, and data describing the path traveled can be generated using these sensors, which are further processed by the computer of the camera or provided for conversion between coordinate systems. However, this requires a known starting point in the spatial coordinate system in order to be able to describe the path.

[0058] Thus, with the aid of the supplementary data describing different representation forms depending on the configuration of the system, on the one hand, a conversion into a spatial coordinate system can be carried out directly in the camera, and the position data of objects of the real scene in the spatial coordinate system thus obtained can be emitted by the camera for further processing by, for example, a central data processing device. On the other hand, the camera can also emit position data of the light signal in the spatial context of the camera coordinate system together with the supplementary data present at the camera, and the conversion into the spatial coordinate system can be carried out by, for example, a central data processing device.

[0059] Thus, the three-dimensional position of the individual lighting devices and the identification information conveyed by them is now known in the spatial coordinate system. Using the identification information, the corresponding position is now queried, for example, from a database of a (central) data processing device for the object actually involved, and the (central) data processing device then generates a data structure for all of the objects from the object position data determined for each object, which contains the three-dimensional position of each object in the business location in the spatial coordinate system. This is also referred to in the jargon as a three-dimensional floor plan of the store, which reproduces the positions of all products in the store as accurately as possible.

[0060] In the present case, as discussed, the floor plan is not generated manually but fully automatically based on the light signals of the electronic shelf labels or separators installed in the store and with the help of a freely movable camera that detects the light signals over time as it moves in the store.

[0061] It can also be provided that sequences of scene images are used together to determine the position of an object. These sequences may also involve overlapping detection areas. Thus, for example, the position can be determined for the same lighting device in successive scene images and the scene images can thus be coupled to one another by placing the positions of light signals in different scene images one above the other. This coupling of scene images results in an extension of the detection area of ​​a camera or of the cameras used.

[0062] In this case, already generated data that locates the light signal and describes its identification information can be used to interpret the newly generated scene image. By using the already generated image position data, i.e. the position and identification of the already detected light-emitting device, the newly generated scene image can be simply interpreted, into which the same light-emitting device is imaged. Light-emitting devices that have not been detected so far can be simply described relative to the position of known light-emitting devices. Dynamic light signal detection is thus also possible, in which the imaged light signal moves in the time sequence of the scene image. In this case, the position of the newly appearing light signal in the scene image sequence is described based on the already known light signal position. Therefore, this coupling allows the scene to be detected and described dynamically and at the same time safely and reliably.

[0063] Thus, for example, all lighting devices installed along a shelf aisle can be detected without problems, even if the respectively used camera cannot detect the entire image of the shelf aisle due to its detection range or due to its positioning or orientation relative to the shelves. This measure thus makes it possible to generate a complete digital scene image of the lighting devices installed along the shelf aisle.

[0064] It should also be mentioned that multiple fixed-position coordinate systems can be used to describe object position data. Thus, for example, a local fixed-position coordinate system can be defined in each aisle, so that the object position data are described in this coordinate system respectively. When the relationship between the coordinate systems is known, the dimensions and / or positions described in the various coordinate systems can then be simply converted to each other. Thus, the object position data can be converted from a local fixed-position coordinate system into a fixed-position spatial coordinate system that includes or describes the entire business premises.

[0065] It should also be mentioned that, in addition to the components already discussed, the freely movable device can also have a storage battery or battery for supplying energy to the electronic components. The freely movable device can also have a generator for converting kinetic energy into electrical energy. The electrical power provided by the generator can be used directly for supplying the electronic components or can be stored intermediately in the storage battery or battery. When using a shopping cart as a freely movable device, the generator is preferably connected to at least one wheel of the shopping cart or integrated therein, so that the rotational movement of the wheel drives the generator. Thus, the camera can also be operated autonomously at the freely movable device. In a manner equipped with its own energy source, the shopping cart can also supply its own drive device in order to assist the customer in an autonomous driving manner. In this regard, it can also be provided that, for example, at a common collection location for such a shopping cart, the energy source is charged in a contactless manner.

[0066] It should also be mentioned that when using a shopping cart, the shopping cart can have a screen, in particular a touch screen, and a data processing unit associated therewith, which allows the customer to interact with the shopping cart. For example, a digital shopping basket can be displayed on the screen, which reproduces which items have been placed in the shopping cart. The customer's shopping list can also be displayed thereby. Detailed information about the product can also be communicated thereby. For this purpose, the shopping cart can have a barcode scanner or an NFC communication module, which can scan or detect the product placed in the shopping cart. The shopping cart can also have another camera or use the previously discussed camera to detect the product placed in the shopping cart. The camera detects the product placed in the shopping cart for this purpose and creates a product image. Here, the product can be identified, for example, according to a barcode or QR code or the like. However, in this case, the object position data that is present in essentially real time also allows the product to be identified according to its structure in the product image. Through the currently generated object position data, the product image can be compared with a possible product selection that is reduced relative to the total variety, and the product is located in the scene or environment according to the object position data. Therefore, the object position data allows a faster and safer processing of the product image and the identification of the product located therein. Such a shopping cart can also have a scale that is provided for weighing the products in the shopping cart. For example, the data generated thereby can be used to determine weight-dependent prices. The inventory of goods in the shopping cart detected in this way can also be used in the same way for a reliable self-service checkout process, including payment for all goods (especially also weight-dependent goods). For this purpose, an NFC payment terminal can be provided at the shopping cart, with which payment can be processed, for example, via the NFC functionality of a mobile phone. In order to also be able to detect goods that are not equipped with an NFC chip, a barcode scanner can be provided in the shopping cart, whereby a self-service checkout process can be carried out in any case.

[0067] The shopping cart can also have an NFC communication module for detecting goods in the shelves, for example, in order to detect NFC tags fastened to objects when the shopping cart passes by. In this configuration, the shopping cart can be used not only to determine the position of the objects, but also to detect the number of objects actually present at the corresponding (shelf) position. It is thus possible to monitor the goods inventory in the store virtually continuously by a group of shopping carts moving in the business premises, and to transmit the detected goods inventory data from the shopping carts to the server via the radio communication module, where they are entered into the floor plan and can also be visualized there.

[0068] Since the server 8 is always informed of the location of the shopping cart by means of the supplementary data, it is also possible to provide the customer with location-specific marketing information by means of the screen while the shopping cart is moving.

[0069] Furthermore, a customer behavior analysis can be performed in the server 8 , since the movement patterns of the shopping cart are available to the server 8 in real time, and if necessary even contain data representing or explaining the inventory of goods in the shopping cart.

[0070] The shopping cart may also have visual signalling means (eg LEDs or a screen) by means of which it can be communicated that the shopping cart is occupied or that the user of the shopping cart requires assistance from personnel at the business location.

[0071] The shopping cart can also have a cost-effective lidar system, by means of which special attention can be drawn to the customer in an environment-specific manner, for example by providing information on a screen in an environment-dependent manner.

[0072] Finally, it should also be generally mentioned that the electronic device or equipment discussed certainly has electronic devices. Electronic devices can be constructed discretely or by integrated electronic devices or by a combination of the two. Microcomputers, microcontrollers, application-specific integrated circuits (ASICs) can be used in combination with analog or digital electronic peripheral components when necessary. Many of the mentioned functionalities of the device are implemented by means of software executed on the processor of the electronic device in cooperation with hardware components when necessary. Devices constructed for radio communication usually have an antenna configuration for sending and receiving radio signals as a component of a transceiver module and a modulator and / or demodulator. Electronic devices can also have an internal electrical power supply device, which can be implemented, for example, with a replaceable or rechargeable battery. These devices can also be supplied in a wired manner either by an external power supply part or by means of "Power over LAN". A radio-based power supply can also be set by means of "Power over WiFi".

[0073] These and other aspects of the invention emerge from the drawings discussed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] The present invention will be described in more detail again below based on embodiments with reference to the accompanying drawings, but the present invention is not limited to the embodiments. In this case, the same components are provided with the same reference numerals in different figures. In a schematic manner:

[0075] Figure 1 The basic structure of a system for carrying out the method according to the present invention is shown;

[0076] Figure 2 showing the appearance of identification information in different optical signals over time;

[0077] Figure 3 The system is shown for use in a shelf aisle in a business location. DETAILED DESCRIPTION

[0078] Figure 1 A system 1 is shown for carrying out a method for determining the position of products P1 to P6 constituting an object in a sales location, hereinafter referred to as space 2. A stationary, orthogonal, right-hand spatial coordinate system 3 is plotted in the space 2, so that the three-dimensional position of the products P1 to P6 can be specified in this spatial coordinate system 3, which has its coordinate axes XR, YR and ZR. The digital representation of these three-dimensional positions is referred to below as object position data for the respective product P1-P6.

[0079] In the space 2, only the two shelf bottoms 7A and 7B of the shelf 7 are visible, and three electronic shelf labels 4A-4C or 4D-4F are fastened at the front edge of the shelf 7 corresponding to or adjacent to the products P1-P6, the electronic shelf labels are logically associated with the products and the electronic shelf labels display product and / or price information for the products respectively using their screens 5. Each shelf label 4A-4F also has a light-emitting diode 6 positioned at the front adjacent to the screen 5, which is configured to emit a light signal, wherein when the light signal is emitted, the light signal is encoded according to a separate identification information, and the identification information is used to unambiguously identify the corresponding product P1-P6. In the present case, a binary code constituting the identification information is transmitted, wherein the current flowing through the light-emitting diode 6 is either switched on or off according to the corresponding bit value. The transmission can also be embedded in the defined (one or more) start and stop bits, etc.

[0080] The system 1 furthermore has a server 8 which stores a logical connection between each product P1-P6 and the shelf label 4A-4F assigned to the product. The shelf labels 4A-4F receive their respective product and / or price information to be displayed from the server 8, wherein the product and / or price information is transmitted to the respective shelf labels 4A-4F by means of a radio shelf label access point 9. Since the server 8 thus knows the identity of each shelf label 4A-4F and the product P1-P6 assigned to the respective shelf label 4A-4F, it is in principle sufficient that the light signal transmits only the identification information of the respective shelf label 4A-4F as identification information, so that the respective product P1-P6 concerned can be unambiguously identified at the server 8 based on the identification information.

[0081] The system 1 further comprises a camera 10 which is freely movable in the room 2, so that in principle it can be moved into all areas of the room 2 in order to detect the respective existing scene there by means of a digital scene image. The scene detected by means of the camera 10 can be described by means of a camera coordinate system 11 with three-dimensional coordinates XK, YK, ZK, wherein the camera coordinate system 11 is located in the center of the image-capturing pixel of the CCD sensor of the camera 10, i.e. has its origin there, and wherein the plane spanned by the coordinates XK and ZK extends in the image-capturing plane of the CCD sensor and has a granularity according to the pixels of the CCD sensor. In the pixel matrix of the CCD sensor, each pixel can be described by a two-dimensional pixel coordinate system, which has axes XKP and ZKP, wherein the axis XKP extends and is oriented according to the axis XK and the axis ZKP extends and is oriented according to the axis ZK. Due to the simpler discussion, it should be assumed here that the pixels of the digital scene image, i.e. the result of the digitization of the imaging of the scene by means of the CCD sensor, can also be addressed according to the two-dimensional pixel coordinate system.

[0082] Furthermore, it should be understood that the third coordinate YK of the camera coordinate system 11 points in the direction of the scene to be detected by the optics (or lens - not shown) of the camera 10. The lens essentially defines the detection area E of the camera. In the presence of digital zoom, this detection area can of course also be related to the setting of the digital zoom.

[0083] The spatial orientation and position of the camera coordinate system 11 in the space 2 , ie in the spatial coordinate system 3 , is therefore dependent on the corresponding position and orientation of the camera 10 in the space 2 .

[0084] In the present case, it should be assumed that the base surface (ground) of the room 2 is coincident with a plane spanned by the coordinate axes XR and YR. Due to the simpler discussion, the camera is also oriented parallel to the base surface in terms of its inclination, which means that the plane spanned by the coordinate axes XK and YK of the camera coordinate system 11 runs parallel to the base surface. The camera is therefore oriented without inclination. If the inclination could be taken into account, inclination data representing the determined inclination would be generated by means of the sensor of the camera 10.

[0085] In the present case, it should also be assumed that the reference orientation in the space 2 is given by the direction of the coordinate axis XR of the spatial coordinate system 3. The current orientation of the camera 10 is therefore specified relative to this direction. This can be done with the aid of an electronic compass of the camera 10, which is set or programmed according to this direction as a reference orientation. An electronic magnetic compass can also be used, with the aid of which the orientation determined with respect to the north can be converted relative to the reference orientation. Independently of the manner in which the current orientation is determined, the orientation data obtained in this way are provided by the camera 10 for further processing.

[0086] The system 1 further comprises a radio camera access point 12, which is connected to the server 8 and by means of which orientation data are transmitted from the camera 10 to the server 8. For this purpose, the camera 10 comprises a corresponding camera data radio communication stage 12A (not shown in detail), of which only a first antenna configuration is shown. The camera data radio communication stage 12A is used to transmit the orientation data and, if necessary, the inclination data (if the inclination data should be taken into account), and also for transmitting other image processing data, which can be transmitted as a result of image processing by means of a computer of the camera 10, which is referred to below as a camera computer (not shown).

[0087] In the present case, the position of the camera 10 in the space 2 is determined by means of UWB radio and this position is assigned in a good approximation to the origin of the camera coordinate system 11. For this purpose, a UWB radio system 13 is provided, the position of which is clearly defined in the space 2 and which is coupled to the server 8 in order to transmit to the server 8 camera position data determined by means of UWB radio, which camera position data describe the position of the camera 10 in the space 2 (specifically relative to the position of the UWB radio system 13, from which position can in turn be converted to a position via the spatial coordinate system 3). Of course, the origin of the spatial coordinate system 3 can also be located at the location of the UWB radio system in order to avoid the conversion in question. For the purpose of UWB radio communication, the camera 10 has a corresponding camera UWB radio communication stage 13A (not shown in detail), of which only the second antenna configuration is shown.

[0088] The method implemented with the aid of the system 1 is discussed in detail below. It should be assumed that the camera 10 is located at the location S1 along its path S and is there directed with its detection area E toward the shelf 7 and creates a series of still image recordings (hereinafter referred to as still image series) of the scene present in the detection area E. Each still image constitutes a digital scene image, the pixels of which are essentially predetermined by the pixel matrix of the CCD sensor of the camera 10, as discussed above.

[0089] In the still image series represented by the image data, in addition to the shelf labels 4A-4F, in particular the coded light signal of each shelf label 4A-4F at the corresponding pixel coordinates XKP and ZKP is now contained.

[0090] In a manner reduced to shelf labels 4A-4F and corresponding light signals, Figure 2 The imaging of the scene onto the plane of the pixel matrix of the CCD sensor of the camera 10 is now visualized, and the brightness present at the imaging location for the corresponding light signal is indicated by the symbols "circle" and "star" for five different time points t1 to t5. In this illustration, a "circle" symbolically represents darkness, i.e., the light-emitting diode 6 emitting the corresponding coded light signal is just turned off. In addition, in this illustration, in contrast to a "circle", a "star" symbolically represents brightness, i.e., the light-emitting diode emitting the corresponding coded light signal 6 is just turned on.

[0091] With the aid of the computer of the camera, it is now determined, on the one hand, by identifying the coding of the corresponding light signal: it is the light signal to be considered in the current context and there are no other light signals that are irrelevant to the purpose of the present invention, and for such a light signal to be considered, the position in the pixel matrix is ​​determined and the corresponding identification information is determined. These determination data on the pixel level are stored in between. At this point in time, the determined identification information can already be used for identifying the associated product P1-P6.

[0092] In a further method step, it is determined how far apart the light signals are in the real scene. For this purpose, a measure is determined which is derived from the knowledge of the real dimensions of the shelf labels 4A-4F present in the digital image (e.g. the length of the side of the front border). Since the light signals are indeed located within the visible front part or front border of the respective shelf labels 4A-4F, the position of the respective shelf labels 4A-4F can be determined without any problems in a computer-aided manner in the digital scene image. The real dimensions of these shelf labels 4A-4F used as reference objects (i.e. the actual length of the section of the front border) are also known to the computer of the camera. By pattern recognition, the respective front border is determined in the digital image, and the number of associated pixels is determined along this border (along the longer and / or shorter side). Since the physical pixel distance of the CCD sensor is known, it is now possible to calculate a measure for converting the counted pixels (i.e. the distance or length in the pixel system of the digital scene image) into real length units in the scene (e.g. millimeters). It is thus possible to determine the relative distances of the light signals in the real scene, i.e. to describe them in the XK-ZK plane of the camera coordinate system 11. In principle, the positions of the shelf labels 4A- 4F and subsequently the positions of the associated products P1 - P6 are thus also already defined in this XK-ZK plane.

[0093] In a further method step, the distance from the camera 10 to the scene is now determined in order to supplement the two-dimensional positioning in the camera coordinate system 11 with a third dimension. Once the physical parameters of both the CCD sensor and the lens are known, i.e., the length of the object imaged on the CCD sensor can be determined at the CCD sensor in, for example, millimeters, and once the dimensions of the corresponding reference object in the real scene are known, the distance between the camera and the real scene can be easily calculated by applying the lens equation with the aid of the computer of the camera 10, which stores the necessary information so that the position of the light signal can be specified in the camera coordinate system 11 along the coordinates XK, YK and ZK, for example in the metric scale (e.g. millimeters).

[0094] The object position data thus determined for each light signal (in a broader sense and thus of course for each shelf label 4A-4F) in the camera coordinate system 11 (which are again present here in the coordinate specifications XK, YK and ZK) are then transmitted from the camera 10 to the server 8 together with the corresponding identification information. Together with this, the orientation data determined at the location S1 are also transmitted to the server 8. The camera position data are also present at the server 8, which were determined for the location S1 at the time of recording the image sequence by means of UWB radio communication. The orientation data and the camera position data together form supplementary data, with the help of which the position specifications of the light signal valid for the camera coordinate system 11 are converted into the spatial coordinate system 3 at the server 8 by means of a conversion (coordinate transformation taking into account the fact that the position of the UWB radio system 13 does not coincide with the origin of the spatial coordinate system 3). The object position data then define the respective position of the light signal with the coordinate specifications XR, YR and ZR.

[0095] As from Figure 2 As can be seen, the camera 10 can in principle be freely movable. Thus, depending on the implementation, the camera can be mounted, for example, at the glasses or at the shopping cart 14, so that Figure 3 As shown.

[0096] Specifically, Figure 3A shelf aisle is shown, which is flanked on both sides by shelves 7 and in which a shopping cart 14 carrying a camera 10 moves along a path S at a high pole 15. For reasons of clarity, only the first six shelf tags 4A-4F and the first six products P1-P6 are provided with reference numerals here as well, wherein the products P1-P6 are merely indicated by a dashed box which illustrates the position of the products P1-P6 in the shelf 7. In the present case, it is assumed that the camera 10 is oriented toward the left side of the cart 14, i.e., the left shelf 7 is located in the detection area E of the camera 10, wherein the length of the shelf 7 exceeds the width of the detection area E. Thus, along the path S, the camera 10 detects the respective scene located in its detection area E several times, for example, first at position S1 and second at position S2. The series of still images generated at each location S1 or S2 is processed by the computer of the camera 10 in a manner similar to the previously discussed and subsequently transmitted to the server 8, which performs the positioning of the light signal, i.e., the final product, in the spatial coordinate system by means of this conversion. As can be seen in the present example, if the series of images detected from adjacent positions P1 and P2 have an overlap of shelf areas, this can be used to improve the result of the determination of the object position data, since the result is indeed generated multiple times in the overlapping area for the same light signal. The same also applies to multiple detections at completely different times with the same camera 10 or with different cameras 10 moving with different shopping carts 14. A large number of moving shopping carts 14 also ensures in larger stores that the entire store is mapped with the help of customers, i.e. object position data for all products are detected over time.

[0097] In order to detect Figure 3 The right-hand shelf 7 can be seen in FIG. 1 , the shopping cart 14 must be turned over and moved along the aisle again, or there are two cameras 10 , wherein the first camera 10 is oriented to the left and the second camera 10 is oriented to the right.

[0098] All of the individual object position data gradually existing on the server 8 for the respective product P1, P2, etc. in the spatial coordinate system 3 are finally used on the server 8 to create a digital three-dimensional map of the product position, which is also referred to as a floor plan in the jargon. The construction or updating of this three-dimensional map of the product P1, P2, etc. can be carried out after a specific detection period has elapsed or gradually, that is, when new object position data are available.

[0099] Alternatively it should also be mentioned that the freely movable device (i.e. glasses or shopping cart) can have a computer which receives the raw data from the camera 10 and performs the processing of the raw data described in conjunction with the camera 10. In this case, the freely movable device can also have a camera data radio communication stage 12A and a camera UWB radio communication stage 13A.

[0100] According to another embodiment, a mobile phone or tablet can also be used as a freely movable device carrying a camera. Modern devices of this type are usually already equipped with an excellent camera system, including a sensor for detecting the distance between the camera and the object detected by the camera (usually implemented as a time-of-flight sensor). The modern devices also have an efficient computer, which can implement the previously discussed computing tasks without problems (especially in real time) using appropriate programming. In the case of such devices, so-called applications are used, i.e. software, which, once executed on the computer of the device, provides the functionality discussed in the context of the present invention. The position determination of these devices in the space of the business premises or warehouse can also be carried out using radio devices integrated in the corresponding device (such as WLAN or ) can be carried out, for example, without problems by means of triangulation or also by means of a UWB radio, provided that this functionality is also implemented at the corresponding device. If such a device is not already equipped with an integrated UWB radio at the factory, it can be retrofitted with a UWB radio (for example in the form of a UWB radio adapter or a USB-capable UWB radio) which can be plugged into, for example, a USB port of the device. In the context of the present invention, such a device is mainly used by personnel in a store or warehouse, where the device is held in the direction of the shelves, carried past the shelves or along the shelves, and in this process detects the corresponding scene. The use of such a device has therefore also proven to be particularly advantageous, since such a device has an integrated screen, with the aid of which the personnel can always visually follow and evaluate the detected scene together on the spot, and can, if necessary, adapt the orientation or direction, and, if necessary, also the zoom setting or exposure, etc., in order to ensure an optimized detection in a single working channel. If the scene detected by means of the camera is not to be subjected to a visual immediate check by a person, then it is also possible to identify by automatic image or video evaluation (e.g. at a server) in which areas of the business premises or warehouse it is necessary to detect the scene again if necessary in order to obtain a complete picture of the situation existing there. However, since the device itself also has an effective computer, it is also possible to automatically check with the corresponding device during the detection whether the digital scene image is suitable for further processing or whether the scene should be detected again. This can be communicated to the person by means of an audio module integrated in the device, for example by means of instructions "spoken" by the device or also in the form of a dialogue, for example acoustically, so that the person is informed of clear and unambiguous action instructions, such as holding the device higher or lower or changing the inclination or orientation in a predetermined direction, etc.

[0101] Finally, it is pointed out again that the drawings described in detail above are only embodiments, which can be modified in various ways by a person skilled in the art without departing from the scope of the present invention. For the sake of completeness, it is also pointed out that the use of the indefinite article "a" or "an" does not exclude that the relevant feature may also be present multiple times.

Claims

1. A method for determining the position of at least one object (P1-P6) in a space (2), in particular at least one product (P1-P6) positioned in a shelf (7), wherein the method comprises: - emitting a light signal from a light emitting device (4A-4F) located adjacent to said object (P1-P6), said light signal being encoded with identification information for identifying said object (P1-P6), and - detecting a scene by means of at least one freely movable camera (10), wherein a digital scene image generated by means of the camera (10) represents the scene, and - generating object position data based on the computerized determined position of the light signal occurring in the digital scene image relative to the camera (10) and the identification information emitted by the light signal and knowing the automatically determined orientation and position of the camera (10) in the space (2) when capturing the scene, The object position data represents the position of the objects (P1-P6) in the scene relative to the space (2).

2. The method according to claim 1, wherein the method is performed with the aid of a plurality of cameras (10) that are moved independently of one another.

3. The method according to claim 1, wherein the at least one freely movable camera (10) is arranged on the shopping cart (14) or on the spectacles.

4. The method according to claim 3, wherein a plurality of cameras (10) are provided, which are moved jointly with one another, ie in groups, and generate individual digital scene images from different detection positions and / or in different detection orientations.

5. A method according to any one of the preceding claims, wherein a control signal transmitter that moves together with the at least one freely movable camera (10) sends a control signal, in particular a radio-based control signal, and a light-emitting device (4A-4F) that is constructed to receive the control signal and is located in a receiving area of ​​the control signal only emits a light signal when receiving the control signal.

6. The method according to claim 1 , wherein determining the position relative to the camera (10) comprises automatically determining a dimension that can be derived from the digital scene image, wherein the dimension is determined with knowledge of the real dimensions of a reference object (4A-4F) identified in the digital scene image, wherein the reference object is in particular designed as an electronic shelf label (4A-4F) and is used for converting between position specifications or dimensions determined in the digital scene image and position specifications or dimensions in the scene detected by means of the camera (10).

7. The method according to any of the preceding claims, wherein determining the position relative to the camera (10) comprises automatically determining the distance between the camera (10) and a scene detected by means of the camera.

8. A method according to any one of the preceding claims, wherein determining the identification information comprises analyzing the temporal variation of an optical signal parameter with knowledge of a coding scheme applied when transmitting the optical signal.

9. A method according to any one of the preceding claims, wherein the method step of emitting the light signal is performed substantially simultaneously by a plurality of light-emitting devices (4A-4F), and wherein the method step of determining in a computerized manner the position of the light signal appearing in the digital scene image relative to the camera (10) and the method step of utilizing the identification information emitted by the light signal is performed for all the light signals appearing in the digital scene image.

10. The method according to any of the preceding claims, wherein for generating the object position data, supplementary data are taken into account, the supplementary data having at least one of the following types of data, namely: - orientation data describing the orientation of the camera (10) relative to a reference orientation (2) in space (2); - inclination data describing the inclination of the camera (10) relative to a reference plane in space (2); - Position data describing the position of the camera (10) relative to a reference point in space (2).

11. A method according to any of the preceding claims, wherein the data processing device generates a data structure from the object position data determined for each object (P1-P6), wherein the data structure describes the three-dimensional position of each object (P1-P6) in the space (2).

12. A freely movable device (14), in particular a shopping cart or glasses or a mobile phone or a tablet, for determining the position of at least one object (P1-P6) in a space (2), in particular at least one product (P1-P6) located in a shelf (7), wherein the position is displayed by a light signal emitted by a light emitting device (4A-4F) located adjacent to the object (P1-P6), the light signal encoding identification information by means of which the object (P1-P6) can be identified, - wherein the device carries at least one camera (10) which is designed to generate a digital scene image of a scene captured by the camera, - and wherein the device, in particular the camera (10), is designed to determine in a computerized manner the position of a light signal occurring in the digital scene image relative to the camera (10) and the identification information emitted by the light signal, - and wherein the device, in particular the camera (10), is designed to at least assist in the automated determination of the orientation and position of the camera (10) in the room (2) when detecting a scene.

13. A lighting device (4A-4F), in particular a shelf label (4A-4F), which is designed to display product and / or price information, or a product separator, which is designed to separate different products (P1-P6), wherein the lighting device (4A-4F) has: a storage stage for storing identification information by means of which objects (P1-P6), in particular products (P1-P6) located in a shelf (7), can be identified, in the vicinity of which the lighting device (4A-4F) is located, and - a light signal generating stage (6) which is designed to encode a light signal according to the identification information and to emit the encoded light signal.

14. A system (1) for determining the position of at least one object (P1-P6) in a space (2), in particular at least one product (P1-P6) positioned in a shelf (7), wherein the system (1) comprises: - at least one lighting device (4A-4F) according to claim 13, and - At least one freely movable device (14) according to claim 12.

Images