SENSOR SYSTEM AND METHOD FOR OPERATING A SENSOR SYSTEM

Description

[0001] The invention relates to a sensor system and a method for operating a sensor system.

[0002] Such sensor systems include a camera system that can detect objects inside a building.

[0003] Such sensor systems can be used, for example, to monitor the interiors of buildings. The camera system is then stationary in an interior space, positioned to detect people entering the building. The sensor system can then trigger an alarm that generates a notification when someone enters the building.

[0004] It is also known that sensor systems combined with camera systems are used in department stores for theft prevention. Shoplifters can be identified using image sequences recorded by the camera system.

[0005] US Patent 2021 / 0049772 A1 describes an automated retail store in which the movements of individuals are tracked by a camera system. This tracking is anonymous. Individuals are distinguished by analyzing images generated by the camera system, from which distinguishing characteristics are derived. For access control, a person can carry an RFID tag, a credit card, or a smartphone.

[0006] US patent 2021 / 0158675 A1 concerns a people tracking system with a camera system that records people's movements in real time. Mobile devices are also provided, allowing people to receive information from a server.

[0007] US 2019 / 0043281 A1 concerns a system for tracking individuals in a theme park. Guests of the theme park are authenticated using electronic tickets. Tracking is carried out using RFID tags.

[0008] The invention is based on the objective of providing a sensor system with high functionality.

[0009] The features of the independent claims are provided to solve this problem. Advantageous embodiments and expedient further developments of the invention are described in the dependent claims.

[0010] The invention relates to a sensor system with a camera system arranged in the interior of a building, designed to identify objects or persons within the interior. A control and evaluation unit evaluates sensor signals from the camera system. At least one terminal device is assigned to each user. The control and evaluation unit assigns a unique user ID to the terminal device. After successful user authentication, a temporary user ID is assigned to each terminal device, valid only for the duration of a visitor session, i.e., the user's stay in the interior, and is then deleted. The control and evaluation unit performs object tracking based on signals from the camera system and the user ID identifying the terminal device. Output data is generated based on this tracking.

[0011] The invention also relates to a corresponding method.

[0012] The sensor system according to the invention enables object tracking within the interior of a building. A key feature of the system is the use of a camera system in combination with a user ID for a user's terminal device. Sensor signals captured by the camera system are linked to the terminal device's user ID in a central control and evaluation unit. Thus, the sensor signals from the camera system, which are related to the terminal device with the user ID, provide user-specific information that is evaluated in the control and evaluation unit to generate corresponding output signals.

[0013] The output signals not only provide information about user-specific use of the interior space. Rather, particularly through object tracking of a large number of users, output signals can be determined that can be used to optimize interior design based on user behavior.

[0014] A particularly advantageous feature of the control and evaluation unit is the anonymized categorization of users based on user IDs. Output data is generated for these user categories.

[0015] This allows for the generation of baseline data for specific user categories while maintaining the anonymity of those tracking indoor objects. This data, particularly through statistical analysis of the collected data, provides information about indoor usage and furnishings for specific groups of people, categorized, for example, by age or gender. This information can be used, in particular, by operators of commercially used indoor spaces to optimize these spaces with a focus on customer needs.

[0016] Finally, output signals generated in the control and evaluation unit can be transmitted to a user's terminal device and displayed there. These output signals can contain information about the interior space or its use, which is preferably displayed to the user depending on their location within the space. The output data is advantageously specific to the individual user.

[0017] A particularly advantageous application of the sensor system according to the invention is given by the fact that the interior is a sales area with a display of products, whereby the entire sales area is recorded by the camera system.

[0018] Retail spaces can generally be found in shopping centers. Particularly advantageous applications are retail stores.

[0019] In such applications, the purchasing behavior of anonymized user categories can be recorded in the control and evaluation unit.

[0020] User IDs on end devices are anonymously assigned to specific groups of people. Typical groups include children, adolescents, and adults, men and women, or people of specific age groups.

[0021] For all user IDs assigned to a group of people, the sensor signals of the camera system are evaluated, which determines which locations or positions in the interior are visited by the people, when and how often, whereby the display of products in the interior is known depending on the location and the removal of products from shelves or other recordings that are present in the interior is also monitored with the camera system.

[0022] In this way, the purchasing behavior of each group of people can be determined over specific periods of time.

[0023] This purchasing behavior of specific groups of people can be made available to, for example, store operators. Based on this information, the display of products in the respective store can be adjusted. The procurement of specific products can also be optimized with regard to the identified purchasing behavior.

[0024] Another advantageous application is that a user can enter a desired product on their terminal device, whereupon the user is shown the way to this product depending on their location.

[0025] This allows users to find the products they want much faster. Time-consuming searches for staff and questioning them are therefore eliminated. This significantly increases customer satisfaction in stores without requiring additional staff.

[0026] Particularly advantageous is the display of the user's surroundings in the sales area, depending on their location.

[0027] The user then sees a floor plan of the interior, i.e. the sales area, displayed on their device, which allows them to orient themselves easily and quickly.

[0028] Additionally, products available in the user's vicinity are displayed.

[0029] This gives the user a quick overview of the products available in the sales area. Since these products are displayed in the context of the surrounding environment, showing their exact locations within the sales area, they are easy for the user to find.

[0030] According to an advantageous embodiment, the camera system forms a triangulation system.

[0031] The camera system generally acquires image information that allows objects and people to be recognized. It also serves to determine the position of objects and people indoors, enabling time- and location-resolved object tracking. Triangulation provides particularly reliable and accurate position determination, with a typical measurement accuracy of 10 cm. For triangulation measurements, the camera module comprises several fixed camera modules, which are thus in a fixed, known spatial relationship to one another.

[0032] According to a particularly advantageous embodiment, the control and evaluation unit is a cloud-based computer system.

[0033] By moving the control and evaluation unit to a cloud, the evaluation of sensor signals from the camera system and the execution of object tracking can be carried out completely independently of the camera system's installation location. Furthermore, this allows for easy expansion of the sensor system to multiple camera systems located in different interior spaces of buildings.

[0034] Furthermore, it is advantageous that no local hardware needs to be installed for the training of the control and evaluation unit, which would mean increased design and maintenance costs.

[0035] Instead, the components of the sensor system can use the internet to connect to the cloud, which does not require any additional hardware.

[0036] According to an advantageous embodiment, the terminal device that a user carries is a mobile terminal device, such as a smartphone.

[0037] In this case, an app is installed on the mobile device, which enables communication with the control and evaluation unit.

[0038] The user enters commands on the mobile device's display to communicate with the control and evaluation unit. Information generated specifically for the user by the control and evaluation unit is then displayed on the user's screen.

[0039] According to an alternative design, the terminal device features data glasses.

[0040] The device thus forms an augmented reality unit. The smart glasses have a processor that handles communication with the control and evaluation unit. Input commands can be given via gestures, voice commands, and the like.

[0041] Information from the control and evaluation unit can be displayed as graphical elements in the environment image that the user sees through the data glasses.

[0042] To ensure the unambiguous identification of a user, their device is assigned a user ID, which is transmitted to the control and evaluation unit. Authentication data such as codes are generally suitable as user IDs. Face ID, i.e., facial recognition of the user, is particularly advantageous.

[0043] For reasons of data security and privacy, it is planned that a temporary user identifier will be assigned to each end device, which is only valid for the duration of a visitor session.

[0044] Once the user has left the interior, the user ID is deleted in the control and evaluation unit.

[0045] The invention will be explained below with reference to the drawings. The drawings show: Figure 1: Schematic representation of an embodiment of the sensor system according to the invention. Figure 2: Example of an interior space with a camera system of the sensor system according to the invention installed there.

[0046] Figure 1 Figure 1 shows, in a highly schematic form, an embodiment of the sensor system according to the invention.

[0047] Sensor system 1 comprises a camera system 2, which is installed in an interior space 3 of a building. The interior space 3 of the building can be a sales area of ​​a retail store, as in Figure 2 The diagram is highly schematic. This sales area contains, as is typical, shelves (4) or other storage areas for product presentation, as well as a cash register system (5).

[0048] The camera system 5 comprises several camera modules 6, which are permanently installed at defined intervals from one another on interior walls or on the ceiling of the interior space 3. Each camera module 6 has an image sensor with associated optics, as is known. The camera module 6 may also include a lighting unit. Finally, the camera module 6 has an evaluation unit in which received signals generated by the image sensor are evaluated, thereby generating image information. Advantageously, the distances between the camera modules 6 are stored in the evaluation units.

[0049] The camera modules 6 of the camera system 2 form a triangulation system with which not only image information of objects or persons can be obtained, but also their positions in the interior 3 can be determined.

[0050] The camera system 2 with its camera modules 6 is connected via the Internet to a cloud, i.e. a cloud-based computer system, which forms a control and evaluation unit 7 of the sensor system 1.

[0051] The control and evaluation unit 7 controls the components of the sensor system 1 and evaluates sensor signals generated in the camera system 2 or quantities derived therefrom.

[0052] Individuals located in interior space 3 can be identified using a device 8 that they are carrying. Figure 1 The figure depicts a person who, as customer K, gains access to interior space 3.

[0053] The terminal device 8 can, in principle, consist of smart glasses that form an augmented reality system. The smart glasses contain a microprocessor that establishes the connection to the control and evaluation unit 7.

[0054] In this case, the terminal device 8 is a mobile device, in particular a smartphone. An app is installed on the mobile device, through which the connection between the mobile device and the control and evaluation unit 7 is established.

[0055] Upon entering interior space 3, a person is authenticated using authentication data, which may consist of codes or a face ID. After successful authentication, the control and evaluation unit 7 assigns a user ID to the person's mobile device. In this case, a temporary user ID is issued, valid only for the duration of the person's stay in interior space 3. The user ID is then deleted.

[0056] In the control and evaluation unit 7, object tracking is carried out based on the sensor signals of the camera system 2 and the user identifiers of the persons located in the interior 3, i.e., the location of the persons is recorded depending on the time.

[0057] In control and evaluation unit 7, the measurement results obtained are analyzed to perform anonymous user categorization. The measurement data collected for individual persons are assigned to different categories. These categories can be male / female, adult / adolescent / child, or others.

[0058] For each user category, the purchasing behavior is then determined based on the measured values. The output signals generated in this way are stored in the control and evaluation unit 7 and can be accessed by authorized persons, such as the retail store operator. Using this information, the operator can, for example, optimize their product portfolio or the product display in the store 3.

[0059] Furthermore, in the control and evaluation unit 7, a personalized evaluation of measurement results for a specific person is possible, whereby the control and evaluation unit 7 can generate information as output signals depending on these measurement results, which are displayed to the person on their terminal device 8.

[0060] For example, a person can use their terminal device 8 to display a desired product. The control and evaluation unit 7 then displays the route to the product, depending on the terminal device 8's current location. Preferably, the display shows the surroundings of the interior space 3 on the terminal device 8, possibly including other products displayed there. Reference symbol list

[0061] (1) Sensor system (2) Camera system (3) Interior (4) Shelf (5) Cash register system (6) Camera module (7) Control and evaluation unit (8) Terminal device (K)Customer

Claims

1. Sensor system (1) comprising a camera system (2) arranged in an interior space (3) of a building, which is designed to detect objects or persons in the interior space (3), and comprising a control and evaluation unit (7), in which sensor signals from the camera system (2) are evaluated, wherein at least one terminal device (8) associated with a user is present, wherein the terminal device (8) is assigned a unique user identifier by the control and evaluation unit (7), characterised in that, following successful authentication of the user, a temporary user ID is assigned to the or each terminal device (8), which is valid only for the duration of a visitor session, i.e. the duration of the user's stay in the interior, and is subsequently deleted, that object tracking is carried out in the control and evaluation unit (7) on the basis of signals from the camera system (2) and the user ID identifying the terminal device (8), and output data is generated depending thereon.

2. Sensor system (1) according to claim 1, characterised in that the camera system (2) forms a triangulation system.

3. Sensor system (1) according to one of claims 1 or 2, characterised in that the control and evaluation unit (7) is a cloud-based computer system.

4. Sensor system (1) according to any one of claims 1 to 3, characterised in that the terminal device (8) is a mobile terminal device.

5. A sensor system (1) according to any one of claims 1 to 4, characterised in that an app is installed on the mobile terminal, via which communication with the control and evaluation unit (7) takes place.

6. A sensor system (1) according to any one of claims 1 to 5, characterised in that the terminal device (8) comprises smart glasses.

7. A sensor system (1) according to any one of claims 1 to 6, characterised in that the user identifier is a code or a Face ID.

8. A sensor system (1) according to any one of claims 1 to 7, characterised in that an anonymised categorisation of users is carried out in the control and evaluation unit (7) on the basis of user identifiers, and that output data is generated for user categories, or that individual output data is generated for a user under their user identifier in the control and evaluation unit (7) and transmitted to that user's terminal device (8).

9. Sensor system (1) according to any one of claims 1 to 8, characterised in that a multiple arrangement of camera systems (2) is present in various interior spaces of buildings, wherein the sensor signals of the camera systems (2) are evaluated in a common control and evaluation unit (7).

10. Sensor system (1) according to any one of claims 1 to 9, characterised in that the interior (3) is a sales area with a display of products, wherein the entire sales area is captured by the camera system (2).

11. Sensor system (1) according to claim 10, characterised in that the purchasing behaviour of anonymised user categories is recorded in the control and evaluation unit (7).

12. Sensor system (1) according to any one of claims 10 or 11, characterised in that a user can enter a desired product on the terminal device (8), whereupon the route to this product is displayed to the user depending on their location.

13. Sensor system (1) according to claim 12, characterised in that the user is shown their surroundings in the sales area depending on their location, and / or that products present in the surroundings are shown to the user.

14. Method for operating a sensor system (1) comprising a camera system (2) arranged in an interior (3) of a building, which is designed to detect objects or persons in the interior (3), and comprising a control and evaluation unit (7), in which sensor signals from the camera system (2) are evaluated, wherein at least one terminal device (8) associated with a user is present, wherein the terminal device (8) is assigned a unique user identifier by the control and evaluation unit (7), characterised in that, following successful authentication of the user, a temporary user identifier is assigned to the or each terminal device (8), which is valid only for the duration of a visitor session, i.e. the duration of the user's stay in the interior, and is subsequently deleted, that object tracking is carried out in the control and evaluation unit (7) on the basis of signals from the camera system (2) and the user ID identifying the terminal device (8), and output data is generated depending on this.