System, camera device and method for displaying sensor data together with an image

Abstract

The present invention relates to a system, a camera device and a method for displaying sensor data together with an image. The method is performed by a camera device. The camera device includes an image capture unit, a communication interface and a processing circuit. The image capture unit is configured with a variable zoom level. The method includes capturing an image depicting a scene by the image capture unit. The method includes communicating with a user interface for displaying an image and with a sensor device through a communication interface to obtain collected sensor data from the sensor device to be displayed on the user interface together with the image. The method includes configuring the sensor device to provide the collected sensor data at a higher frequency of occurrence by the processing circuit and via the communication interface when the image capture unit captures and depicts an image of the sensor device at a zoom level above a zoom threshold compared to when the image capture unit captures the image at a zoom level below a zoom threshold.

Description

Technical Field

[0001] Embodiments presented herein relate to systems, camera devices, methods, computer programs, and computer program products for displaying collected sensor data with an image. Background Art

[0002] In general, a sensor device is a device, module, machine or subsystem that aims to detect events or changes in its environment and send the information to other electronic devices such as computer processors for further processing or for display to the user at a user interface. In order to do this, the sensor device consumes electrical energy. There may be different ways to provide electrical energy to the sensor device. In some deployments, it may be beneficial to use a user sensor device that consumes as little power as possible. This may be the case in a deployment with multiple sensor devices, where the total power consumption therefore needs to be limited, and / or in a deployment with battery-powered sensors, or otherwise only with access to very limited electrical energy resources, but this should still have a long life. Further, even if it is not necessary due to limited electrical energy resources, it may be desirable to reduce power consumption from an environmental or cost perspective.

[0003] As a non-limiting but illustrative example, assume that the sensor device is a Z-Wave type sensor device. Therefore, the Z-Wave device is battery-powered, but the energy efficiency is high. This is because the Z-Wave device is configured with a sleep state, and when it has entered the sleep state, the Z-Wave device does not accept control from an external controller. After a period of time, the Z-Wave device wakes up from sleep, exchanges commands with the controller, and then re-enters the sleep state. The Z-Wave device can be configured with a minimum value, a maximum value, and a default value for the duration of the sleep state. Similarly, the wake-up time can be set within a range from a minimum value to a maximum value.

[0004] When a user accesses a Z-Wave device, the user expects an immediate response from the device. For this purpose, the sleep time needs to be set short (i.e., the duration of the sleep state is short). However, if the sleep time is set too short, the power consumption of the device will increase, possibly exhausting the power of the sensor device faster than expected. On the one hand, the sleep time should be short for the benefit of the user, but on the other hand, in order to extend the operating life of the sensor device, the sleep time should be long. This leads to a conflict of interest in how the sleep time should be selected.

[0005] Similar considerations exist for other types of sensor devices besides Z-Wave devices regarding the need for a user to access sensor data versus conserving power resources of the sensor device. Thus, selecting a sleep time or similar parameter to achieve a long operating life for the sensor device while still allowing sensor data to be provided to the user as needed can be cumbersome. Summary of the invention

[0006] The purpose of the embodiments herein is to solve the above-mentioned problems.

[0007] In general, according to the inventive concepts disclosed herein, a sensor device is monitored by an image capture unit, and the rate at which the sensor device provides sensor data is determined as a function of the zoom level used by the image capture unit. This enables sensor data collected by the sensor device to be variably displayed on a user interface.

[0008] According to a first aspect, the inventive concept is defined by a system for displaying collected sensor data with an image. The system includes a camera device. The camera device includes an image capture unit configured to capture an image depicting a scene. The image capture unit is configured with a variable zoom level. The camera device is configured to communicate with a user interface for displaying an image. The system includes a sensor device. The sensor device is configured to collect sensor data and provide the collected sensor data to the camera device with a variable frequency of occurrence to be displayed on the user interface with the image. When the camera device acquires an image captured at a zoom level above a zoom threshold and depicting the sensor device from the image capture unit, the sensor device is configured by the camera device to provide the collected sensor data at a higher frequency of occurrence than when the camera device acquires an image captured at a zoom level below a zoom threshold from the image capture unit.

[0009] According to a second aspect, the inventive concept is defined by a camera device for displaying collected sensor data with an image. The camera device includes an image capture unit. The image capture unit is configured to capture an image depicting a scene. The image capture unit is configured with a variable zoom level. The camera device includes a communication interface. The communication interface is configured to communicate with a user interface for displaying an image and with a sensor device to obtain the collected sensor data from the sensor device to be displayed on the user interface with the image. The camera device includes a processing circuit. The processing circuit is configured to configure the sensor device via the communication interface to provide the collected sensor data at a higher frequency of occurrence when the image capture unit captures and depicts an image of the sensor device at a zoom level above a zoom threshold compared to when the image capture unit captures the image at a zoom level below a zoom threshold.

[0010] According to a third aspect, the inventive concept is defined by a method for displaying collected sensor data with an image. The method is performed by a camera device. The camera device includes an image capture unit, a communication interface, and a processing circuit. The image capture unit is configured with a variable zoom level. The method includes capturing an image depicting a scene by the image capture unit. The method includes communicating with a user interface for displaying an image and with a sensor device through a communication interface to obtain the collected sensor data from the sensor device to be displayed on the user interface together with the image. The method includes configuring the sensor device to provide the collected sensor data at a higher frequency of occurrence by the processing circuit and via the communication interface when the image capture unit captures and depicts an image of the sensor device at a zoom level above a zoom threshold compared to when the image capture unit captures the image at a zoom level below a zoom threshold.

[0011] According to a fourth aspect, the inventive concept is defined by a computer program for displaying collected sensor data together with an image, the computer program comprising computer program code which, when run on a camera device, causes the camera device to perform the method according to the third aspect.

[0012] According to a fifth aspect, the inventive concept is defined by a computer program product comprising a computer program according to the fourth aspect and a computer readable storage medium having the computer program stored thereon. The computer readable storage medium may be a non-transitory computer readable storage medium.

[0013] Advantageously, the inventive concept enables the selection of a sleep time or similar parameter such that a long operating life of the sensor device can be achieved while allowing the sensor data to be provided to the user as desired.

[0014] Advantageously, the inventive concept enables a balance to be struck between the user's need to access sensor data and the conservation of electrical energy resources of the sensor device.

[0015] Advantageously, the inventive concept reduces power consumption of the sensor device without reducing information provided to a user interacting with the user interface. This is because the user is more likely to be interested in the sensor data of the sensor device if the camera is zoomed in to depict an area including the sensor device.

[0016] It should be noted that any feature of the first, second, third, fourth and fifth aspects may be applied to any other aspect, as appropriate. Likewise, any advantage of the first aspect may also be applicable to the second, third, fourth and / or fifth aspects, respectively, and vice versa. Other objects, features and advantages of the attached embodiments will become apparent from the following detailed disclosure, from the attached dependent claims and from the drawings.

[0017] Generally, all terms used in the claims should be interpreted according to their ordinary meaning in the technical field, unless otherwise explicitly defined herein. Unless otherwise explicitly stated, all references to "a / an / the element, device, part, means, module, step, etc." should be openly interpreted as referring to at least one instance of the element, device, part, means, module, step, etc. Unless explicitly stated otherwise, the steps of any method disclosed herein do not have to be performed in the exact order disclosed. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The inventive concept will now be described by way of example with reference to the accompanying drawings, in which:

[0019] Figure 1 is a schematic diagram illustrating a system according to an embodiment;

[0020] Figure 2 and Figure 3 schematically illustrates a user interface according to an embodiment;

[0021] Figure 4 , Figure 5 , Figure 6 , Figure 7 and Figure 8 is a flow chart of a method according to an embodiment;

[0022] Fig. 9 is a schematic diagram showing functional units of a camera device according to an embodiment; and

[0023] Fig.10 An example of a computer program product including a computer-readable storage medium according to an embodiment is shown. DETAILED DESCRIPTION

[0024] The inventive concept will now be described more fully below with reference to the accompanying drawings, in which certain embodiments of the inventive concept are shown. However, the inventive concept can be implemented in many different forms and should not be construed as being limited to the embodiments set forth herein; rather, these embodiments are provided by way of example so that the disclosure will be thorough and complete and will fully convey the scope of the inventive concept to those skilled in the art. Throughout the description, the same reference numerals refer to the same elements. Any steps illustrated by dashed lines should be considered optional.

[0025] As discussed above, selecting a sleep time or similar parameter to achieve a long operational life of the sensor device while still allowing sensor data to be provided to the user as desired can be cumbersome.

[0026] Therefore, embodiments disclosed herein relate to mechanisms for variably displaying collected sensor data. To achieve such mechanisms, a system 100, a camera device 110, a method performed by the camera device 110, and a computer program product including code, for example in the form of a computer program, which, when executed on the camera device 110, causes the camera device 110 to perform the method are provided.

[0027] As an example, consider a deployment in which sensor data collected from a sensor device is overlaid or otherwise laid over an image of the sensor device and displayed on a user interface (e.g., based on a graphical user interface). Figure 1 in the settings). Figure 1 1 is a schematic diagram illustrating a system 100 to which embodiments presented herein may be applied. System 100 includes a camera device 110. Camera device 110 includes an image capture unit 120. Image capture unit 120 is configured to capture a digital image depicting a scene 160. Image capture unit 120 is configured with a variable zoom level. That is, image capture unit 120 is capable of capturing images using a variable zoom level. Different examples of how to change the zoom level will be disclosed below. Figure 1, the field of view of the image capture unit 120 corresponding to when the zoom level is maximally zoomed out (i.e., the widest field of view that the image capture unit 120 can achieve) is schematically illustrated at reference numeral 150a. The field of view of the image capture unit 120 corresponding to when the zoom level is maximally zoomed in (i.e., the narrowest field of view of the image capture unit 120) is schematically illustrated at reference numeral 150b. The fields of view 150a, 150b are illustrated for the same pointing direction of the image capture unit 120. In this regard, there may be different ways to change the zoom level. In one non-limiting example, the zoom level is variable by optical zooming of an optical lens system that is part of the image capture unit 120 or attached to the image capture unit 120 and controlled by the camera device 110. In one non-limiting example, the zoom level is variable by digital zooming, wherein the zoom level is thus implemented in software that is provided in the image capture unit 120 or the camera device 110. In some examples, the image capture unit 120 is capable of (digital) pan, tilt and zoom (PTZ) and, therefore, may be considered a (digital) PTZ image capture unit. The camera device 110 is configured to communicate with a user interface 130 for displaying images. Further, the camera device 110 is configured to encode images according to any known video coding standard (e.g., H.264 or H.265). In this regard, the encoding may be performed directly in conjunction with the image capture unit 120 capturing the image or at another entity such as a computing server 114, and then at least temporarily stored in the data store 116. The computing server 114 and the data store 116 are therefore considered to be part of the camera device 110. The computing server 114 and the data store 116 are operably connected to the image capture unit 120 via a network 112. The network 112 may be wired, wireless, or partially wired and partially wireless.

[0028] In some embodiments, the camera device 110 includes a communication interface 920 configured to communicate with a user interface 130 for displaying images and with the sensor devices 170a, 170b for obtaining the collected sensor data from the sensor devices 170a, 170b for display on the user interface 130 together with the images. In the case where the images have been encoded, then corresponding decoding is performed before the images are displayed at the user interface 130. In some aspects, a video management system (VMS) may be provided in conjunction with the user interface 130 and configured to decode the received encoded image sequence. Fig. 9Other aspects of the communication interface 920 are disclosed. An example connection between the camera device 110 and the user interface 130 is illustrated at reference numeral 140. The connection 140 may be wired, wireless, or partially wired and partially wireless. A user 180 may interact with the user interface 130. It should be understood that the user interface 130 is at least partially a visual user interface in that it is configured to display images captured by the image capture unit 120. Figure 2 Other aspects of the user interface 130 are disclosed. Figure 3 Other aspects with which user 180 may interact with interface 130 are disclosed.

[0029] The system 100 further includes at least one sensor device 170a, 170b. Each sensor device 170a, 170b is configured to collect sensor data and provide the collected sensor data to the camera device 110 for display on the user interface 130 together with the image. When the sensor data is displayed at the user interface 130 (e.g., at the location where the sensor device is depicted in the image), the sensor data can thus be added to the acquired image as an overlay. The overlay can be added directly to the image sequence or sent separately as metadata and added to the image only when displayed at the user interface 130 or at least after the image is decoded. The overlay can thus be encoded together with the image and / or separately from the image.

[0030] As part of developing the inventive concept, the inventors of the embodiments disclosed herein have recognized that when a user 180 is monitoring the user interface 130 with a wide field of view captured image, the user 180 may be interested in sensor data of nearby sensor devices rather than sensor data of distant sensor devices. Conversely, if the user 180 has zoomed in toward a sensor device (near or far), the user 180 may be interested in sensor data of the sensor device depicted at the zoomed in position rather than sensor data from the sensor device that is not zoomed in. In the latter case, if sensor data obtained from a sensor device that the user 180 is not interested in is displayed, the zoomed out view on the user interface 130 becomes cluttered and difficult to interpret. Further, there is a risk that the power of the sensor device is drained for no purpose.

[0031] In general, therefore, the inventive concept disclosed herein is based on providing an effective system 100 of sensor devices 170a, 170b and camera device 110 that solves the above-mentioned problems, so that sensor data can be effectively superimposed on the captured image. According to the inventive concept, this is achieved by the sensor devices 170a, 170b being monitored by the image capture unit 120, and the rate at which the sensor devices 170a, 170b provide sensor data to the camera device 110 is determined as a function of the zoom level used by the image capture unit 120.

[0032] Specifically, when camera device 110 acquires from image capture unit 120 an image captured at a zoom level above the zoom threshold and depicting sensor devices 170a, 170b, each sensor device 170a, 170b is configured by camera device 110 to provide collected sensor data at a higher frequency of occurrence than when camera device 110 acquires from image capture unit 120 an image captured at a zoom level below the zoom threshold. In some embodiments, camera device 110 includes processing circuitry 910 configured to configure sensor devices 170a, 170b accordingly via communication interface 920. Reference will be made to Fig. 9 Other aspects of the processing circuitry 910 are disclosed. Each sensor device 170a, 170b is thus capable of collecting sensor data and providing the collected sensor data to the camera device 110 at a variable frequency of occurrence for display on the user interface 130 along with the image.

[0033] In other words, the frequency with which the sensor devices 170a, 170b provide the collected sensor data to the camera device 110 therefore depends on the current zoom level used by the image capture unit 120. This dependency is achieved through a zoom threshold. When the image capture unit 120 uses a zoom level that is higher than (i.e., toward the telephoto end) the zoom threshold, this indicates that the depicted sensor devices 170a, 170b are zoomed in. Therefore, the update frequency is set to a higher value and the sensor data is added as an overlay. Conversely, when the image capture unit 120 uses a zoom level that is lower than (i.e., toward the wide-angle end) the zoom threshold, this indicates that the depicted sensor devices 170a, 170b are zoomed out. Therefore, the update frequency is set to a lower value and the sensor data, which is an optional feature, is not added as an overlay.

[0034] exist Figure 1 An example system 100 is shown in which there are two sensor devices 170a, 170b. The embodiments disclosed herein are not limited to any specific number of sensor devices 170a, 170b. However, in some embodiments, it is envisioned that there are at least two sensor devices 170a, 170b, each of which is associated with its own zoom threshold. That is, in some embodiments, the system 100 includes at least two sensor devices 170a, 170b, wherein each sensor device 170a, 170b is configured to collect sensor data and provide the collected sensor data at a respective variable frequency of occurrence to be displayed on the user interface 130 together with the image. Each of the at least two sensor devices 170a, 170b can then be associated with its own zoom threshold. It will be disclosed below how each sensor device 170a, 170b can be configured with or associated with its own zoom threshold.

[0035] Figure 2 The diagram schematically illustrates the user interface 130 according to an embodiment when displaying an image of a scene 160 captured by the image capture unit 120. Figure 2 , the user interface 130 is illustrated as a screen that may be a touch screen that allows a user to interact with the user interface 130. However, the manner in which a user interacts with the user interface 130 may also be different. For example, the user interface 130 may be configured to allow a user to move a cursor on the user interface 130. In this regard, the user interface 130 may be provided with or operably connected to a navigation tool such as a mouse, keyboard, joystick, etc. to allow the user to move a cursor on the user interface 130. As described above, the user interface 130 is operably connected to the camera device 110 via a connection 140. As will be further disclosed below, this allows the system 100 to detect a user click on the user interface 130 (either through a touch input on the touch screen or through an indication from a navigation tool). Further, the user interface 130 may be operably connected to a computer, terminal, and / or a communication network.

[0036] If image capture unit 120 is maximally zoomed out (i.e., corresponding to field of view 150a), a view of scene 160 is shown at reference numeral 132a. If image capture unit 120 is maximally zoomed in toward the center (i.e., corresponding to field of view 150b), a view of scene 160 is shown at reference numeral 132b. Sensor device 170b is closer to image capture unit 120 than sensor device 170a, and therefore, when sensor device 170b is displayed on user interface 130, sensor device 170b will appear to have a larger size than sensor device 170a. Further, because sensor device 170b is outside the narrowest field of view of image capture unit 120, sensor device 170b is not displayed when the zoom level is maximally zoomed in.

[0037] Now we will continue to refer to Figure 1 and Figure 2 Embodiments related to further details of the system 100 (and in particular the camera device 110 and the sensor devices 170a, 170b) are disclosed. Reference will also be made to Figure 3 , and especially Figure 3 (a) In Figure 3 In the illustrative example of , each sensor device 170a, 170b represents a temperature sensor, and therefore, the sensor data represents the temperature readings of the sensor devices 170a, 170b. That is, Figure 3The sensor devices 170a, 170b in represent types of weather sensors. However, as will be appreciated by those skilled in the art, there may also be other types of sensor devices with corresponding types of sensor data (such as, but not limited to, light, voltage, current, pressure, humidity, air pollution, fire, gas, etc.), and the embodiments disclosed herein are not limited to any particular type of sensor device or sensor data. In some embodiments, the sensor devices 170a, 170b are battery powered Z-wave devices. Other embodiments include sensor devices that communicate via Bluetooth, ZigBee, or any other suitable communication protocol for applications.

[0038] In some aspects, sensor data is displayed as overlays 172a, 172b only when the zoom level is above a zoom threshold of the corresponding sensor device 170a, 170b. Specifically, in some embodiments, the camera device 110 is configured to provide the sensor data as an overlay 172a, 172b to the image when the camera device 110 acquires an image that is captured at a zoom level above the zoom threshold and depicts the sensor devices 170a, 170b. That is, the first condition is that the zoom level is above the zoom threshold of the sensor devices 170a, 170b, and the second condition is that the sensor devices 170a, 170b are depicted in the image (and therefore shown on the user interface 130). Figure 3 In the example of (a), for illustrative purposes, it is assumed that the zoom level is above the zoom threshold of sensor device 170a but below the zoom threshold of sensor device 170b. Therefore, the sensor data is shown as an overlay 172a of sensor device 170a (indicating a temperature value of 24.6°C).

[0039] In some aspects, when the zoom level is below a zoom threshold of the corresponding sensor device 170a, 170b, a representation of the sensor device 170a, 170b is displayed. Specifically, in some embodiments, the camera device 110 is configured to display an illustration 174 representing the sensor device 170a, 170b as an overlay 172a, 172b of the image on the user interface 130 when the camera device 110 acquires an image captured and depicting the sensor device 170a, 170b at a zoom level below the zoom threshold. Non-limiting examples of illustrations include icons, symbols, etc. Figure 3 In the example of (a), an illustration 174 in the form of an icon of sensor 170b is displayed because, as disclosed above, it is assumed that the zoom level is below the zoom threshold of sensor device 170b (and sensor device 170b is shown on user interface 130).

[0040] As disclosed above, the system 100 may be configured to detect a user click on the user interface 130. Assume that the click is made on a location on the user interface 130. Therefore, the stored image locations of the sensor devices 170a, 170b may be compared to determine which sensor device 170a, 170b (if any) the user selected. That is, the sensor device 170a, 170b has a location in the scene 160 in the image. Since the location is in the image (as opposed to the actual location of the sensor device 170a, 170b), the location may be referred to as an image location. When the sensor device 170a, 170b is displayed on the user interface 130, the image location of the sensor device 170a, 170b in the scene 160 has a mapping to a location on the user interface 130. In this regard, there may be one mapping for each zoom level, pan direction, and tilt direction of the image capture unit 120 (depending on whether the image capture unit 120 is capable of panning and / or tilting). In some embodiments, camera device 110 is configured to determine that the user input identifies representation 174 of sensor device 170a, 170b by verifying that the location of the user input on user interface 130 corresponds to the image location of sensor device 170a, 170b on user interface 130. Thus, this allows system 100 to determine whether the user "clicked" on one of sensor devices 170a, 170b displayed on user interface 130. Figure 3 (a) shows an example of a user 180's hand 182 clicking on the depicted sensor device 170b. Figure 3 (b) and Figure 3 (c) further discloses the action taken by the system 100 when input representing a user click on the depicted sensor device 170b has been received.

[0041] In some aspects, a click by user 180 on the representation 174 of sensor device 170b causes sensor data of the sensor device 170b to be obtained and displayed. Specifically, in some embodiments, user interface 130 is configured to receive user input, wherein the user input identifies the representation 174 of sensor device 170a, 170b, and wherein camera device 110 is configured to display the sensor data as an overlay 172a, 172b of the image on user interface 130 in response thereto. This is Figure 3 (a) and Figure 3 As shown in (b), assume that the user 180's hand 182 clicks Figure 3 (a) sensor device 170b, and as indicated by arrow 190a, from Figure 3 (a) to Figure 3 (b) is converted. Figure 3In (b), sensor data obtained by sensor device 170b is shown as overlay 172b of sensor device 170b (indicating a temperature value of 20.1° C.) Thus, in response to the user input, sensor data is displayed for sensor device 170b, although the zoom level is below the zoom threshold of sensor device 170b.

[0042] In some aspects, a click by user 180 on the representation 174 of sensor device 170 b causes image capture unit 120 to zoom in on sensor device 170 b to at least a zoom threshold for displaying sensor data of sensor device 170 b. Specifically, in some embodiments, user interface 130 is configured to receive user input, wherein the user input identifies the representation 174 of sensor device 170 a, 170 b, and wherein camera device 110 is configured to cause image capture unit 120 to zoom in to the zoom threshold in response thereto. This Figure 3 (a) and Figure 3 As shown in (c), assume that the user 180's hand 182 clicks Figure 3 (a) of the sensor device 170b, and as indicated by arrow 190b, from Figure 3 (a) to Figure 3 (c) is converted. Figure 3 In (c), the image capture unit 120 has zoomed in towards the sensor device 170b to the zoom threshold of the sensor device 170b. As a result, the sensor data 172b of the sensor device 170b is shown as an overlay 172b of the sensor device 170b (indicating a temperature value of 20.1°C).

[0043] Aspects of how the camera device 110 may be configured with the sensor devices 170a, 170b will now be disclosed.

[0044] In some aspects, a calibration process is performed to determine the zoom threshold for each sensor device 170a, 170b. That is, in some embodiments, the camera device 110 is configured to establish the zoom threshold during the calibration process. An example of such a calibration process is disclosed below.

[0045] In some aspects, whether a given sensor device 170a, 170b is present in a given field of view of the image capture unit 120 is listed by reference to the stored position information obtained during the calibration process. Specifically, in some embodiments, the image capture unit 120 has a geometric relationship with the sensor devices 170a, 170b established during the calibration process. The geometric relationship includes the zoom level-dependent position information of the sensor devices 170a, 170b. The zoom level-dependent position information specifies whether the sensor devices 170a, 170b are visible in an image captured using a given zoom level. Therefore, it can be confirmed whether a given sensor device 170a, 170b is present in a given field of view of the image capture unit 120 without the need to perform object recognition or other types of image analysis on the images captured by the image capture unit 120.

[0046] As disclosed above, the sensor devices 170a, 170b are configured by the camera device 110 to provide the collected sensor data at a frequency of occurrence depending on whether the zoom level of the image capture unit 120 is above or below the zoom threshold. Further, in this regard, the camera device 110 can be configured to use an update frequency value to configure the sensor devices 170a, 170b to provide the sensor data to the camera device 110. Specifically, in some embodiments, the camera device 110 is configured to indicate to the sensor devices 170a, 170b: at which frequency of occurrence the sensor devices 170a, 170b provide the sensor data. In some aspects, the update frequency value of the sensor devices 170a, 170b is switched between at least two predetermined values. Specifically, in some embodiments, the frequency of occurrence is variable by being selectable between a first predetermined value and a second predetermined value, and the first predetermined value produces a higher frequency of occurrence than the second predetermined value. That is, the frequency of occurrence can be switched so that it is selectably equal to the first predetermined value or the second predetermined value, that is, so that the frequency of occurrence is selectably set to a high frequency of occurrence or a low frequency of occurrence. There may be different ways of selecting the predetermined values. In some embodiments, the first predetermined value is given by the wake-up time configuration of the sensor devices 170a, 170b, and the second predetermined value is given by the sleep time configuration of the sensor devices 170a, 170b. This may be the case when the sensor devices 170a, 170b are Z-wave devices, wherein the wake-up time configuration is defined by the value of the wake-up state parameter, and the sleep time configuration is defined by the value of the sleep state parameter of the Z-wave device. In general, the wake-up state parameter is a parameter that takes a relevant value when the Z-wave is in the wake-up state, and the sleep state parameter is a parameter that takes a relevant value when the Z-wave is in the sleep state. In this regard, the value of the wake-up state parameter defines the frequency or rate at which sensor data is collected and provided to the camera device 110 by the sensor devices 170a, 170b when the Z-wave device is in the wake-up state. Accordingly, the value of the sleep state parameter defines the frequency or rate at which sensor data is collected and provided to the camera device 110 by the sensor devices 170a, 170b when the Z-wave device is in the sleep state. In other words, when the occurrence frequency is set to the first predetermined value, the sensor devices 170a, 170b collect sensor data and provide the collected sensor data to the camera device 110 at a rate given by the wake-up time configuration, which in turn may be defined by the value of the wake-up state parameter. Correspondingly, when the occurrence frequency is set to the second predetermined value, the sensor devices 170a, 170b collect sensor data and provide the collected sensor data to the camera device 110 at a rate given by the sleep time configuration, which in turn may be defined by the value of the sleep state parameter.In some examples, when the frequency of occurrence is set to the second predetermined value, sensor data is not collected, and therefore, sensor data is not provided. This is because the Z-wave device is then in its sleep state. Further, in this regard, as described above, when the Z-wave device enters the sleep state, the Z-Wave device may not accept control from the external controller until it wakes up again next time. Therefore, it may be the case that when the sensor devices 170a, 170b are Z-wave devices, the update of the frequency of occurrence value of the sensor devices 170a, 170b (i.e., the frequency of occurrence either changes from a high value to a low value or from a low value to a high value) will only take effect when the Z-wave device is not in the sleep state. Therefore, although the camera device 110 obtains an image captured and depicting the sensor devices 170a, 170b at a zoom level higher than the zoom threshold from the image capture unit 120, the sensor devices 170a, 170b may still temporarily provide collected sensor data at a lower frequency of occurrence until exiting the sleep state.

[0047] Generally speaking, any of the above-disclosed aspects, embodiments, and examples of the camera device 110 are also applicable to the methods disclosed below.

[0048] Figure 4 1 is a flow chart illustrating an embodiment of a method for variably displaying collected sensor data. The method is performed by the camera device 110. The method is advantageously provided as a computer program 1020. As disclosed above, the camera device 110 includes an image capture unit 120, a communication interface 920 and a processing circuit 910, and the image capture unit 120 is configured with a variable zoom level.

[0049] S102 : The image capturing unit 120 captures an image depicting the scene 160 .

[0050] S104 : The communication interface 920 communicates with the user interface 130 for displaying an image and with the sensor devices 170 a , 170 b , so as to obtain the collected sensor data from the sensor devices 170 a , 170 b to be displayed on the user interface 130 together with the image.

[0051] S106: When the image capture unit 120 captures and depicts an image of the sensor devices 170a, 170b at a zoom level higher than the zoom threshold, the processing circuit 910 configures the sensor devices 170a, 170b via the communication interface 920 to provide collected sensor data at a higher frequency of occurrence than when the image capture unit 120 captures the image at a zoom level lower than the zoom threshold.

[0052] Embodiments related to further details of variably displaying collected sensor data as performed by camera device 110 will now be disclosed.

[0053] Figure 5 1 is a flow chart illustrating an embodiment of a method for establishing a relationship between an image capture unit 120 and sensor devices 170a, 170b in a global or common coordinate system. The relationship can be used to understand whether the image capture unit 120 is pointing or oriented to a given sensor device 170a, 170b (for a given zoom level). The relative position of each sensor device 170a, 170b relative to the image capture unit 120 is registered and stored, for example, in the storage medium 930 of the camera device 110 (step S202). The zoom level is then moved to the wide-angle end, and a user input is received that defines the farthest distance for displaying sensor data for each sensor device 170a, 170b. The determined distance for each sensor device 170a, 170b is stored, for example, in the storage medium 930 of the camera device 110 as a corresponding first set value for one of each sensor device 170a, 170b (step S204). The corresponding physical distance between the image capture unit 120 and each sensor device 170a, 170b (one of each sensor device 170a, 170b) is calculated or estimated from the stored device position. The apparent distance value when the zoom level is at the wide-angle end is calculated based on the field of view information of the image capture unit 120 (step S206). If the calculated value is greater than the first set value, the zoom level at which the apparent distance value is equal to the first set value is calculated and stored in, for example, the storage medium 930 of the camera device 110 as the zoom threshold of the sensor devices 170a, 170b (step S208). Through the above processing, the zoom threshold of each sensor device 170a, 170b is calculated and stored.

[0054] Figure 6 is a flow chart illustrating an embodiment of a method for variably displaying collected sensor data based on at least some of the embodiments disclosed above. The camera device 110 obtains the current zoom level used by the image capture unit 120 (step S302). By referring to the stored position information (e.g., based on Figure 5The method then proceeds to step S306 and then, based on the position information established, lists any sensor devices 170a, 170b present in the corresponding field of view of the image capture unit 120 (step S304). A zoom threshold is obtained for the listed sensor devices 170a, 170b (step S306). The method branches depending on whether the current zoom value is above or below the zoom threshold. Therefore, it is checked whether the current zoom value is above or below the zoom threshold (step S308). If the current zoom value is above the zoom threshold, the update frequency value of the sensor devices 170a, 170b is set to a high value (corresponding to the set short sleep time) (step S310). Sensor data is returned from the sensor devices 170a, 170b displayed on the user interface 130 (step S312). If the current zoom value is below the zoom threshold, the update frequency value of the sensor devices 170a, 170b is set to a low value (corresponding to the set long sleep time) (step S314). Representations (such as icons) indicative of the sensor devices are displayed as overlays at the locations of the sensor devices 170a, 170b on the image displayed at the user interface 130 (step S316).

[0055] Figure 7 The diagram shows the Figure 3 (a) and Figure 3 (b) is a flow chart of an embodiment of a method for variably displaying collected sensor data. As a start, assume that for a given sensor device 170b under consideration, the zoom level currently used by the image capture unit 120 is below the zoom threshold. Thus, the situation is as follows Figure 3 (a). A representation 174 (such as an icon) indicating sensor device 170b is displayed as an overlay at the position of sensor device 170b on the image displayed at user interface 130. Assume that user 180 interacts with user interface 130 to identify sensor device 170b (for example, by user's hand 182 performing a touch action on user interface 130 or by the user using a navigation tool to select sensor device 170b) (step S402). The identified position on user interface 130 is compared with the stored image positions of sensor devices 170a, 170b, and a decision is made to select sensor device 170b (step S404). The update frequency value of sensor device 170b thus selected is set to a high value (corresponding to the short sleep time that is set) (step S406). Sensor data collected by sensor device 170b is obtained (step S408). The sensor data thus obtained is displayed as an overlay 172b at the position of sensor device 170b on user interface 130 (step S410). This corresponds to the updating of sensor device 170b from the image according to arrow 190a. Figure 3 (a) to Figure 3 (b) Conversion.

[0056] Figure 8The diagram shows the Figure 3 (a) and Figure 3 (c) is a flow chart of an embodiment of a method for variably displaying collected sensor data. Figure 8 In , as a start, it is assumed that for a given sensor device 170b under consideration, the zoom level currently used by the image capture unit 120 is below the zoom threshold. Figure 3 (a). A representation 174 (such as an icon) indicating sensor device 170b is displayed as an overlay at the location of sensor device 170b on the image displayed at user interface 130. Assume that user 180 interacts with user interface 130 to identify sensor device 170b (e.g., by user's hand 182 performing a touch action on user interface 130 or by the user selecting sensor device 170b using a navigation tool) (step S502). The identified location on user interface 130 is compared with the stored locations of sensor devices 170a, 170b, and a decision is made to select sensor device 170b (step S504). It is checked whether the current zoom level of sensor device 170b is above or below the zoom threshold of sensor device 170b (step S506). If the current zoom level is below the zoom threshold, camera device 110 instructs image capture unit 120 to zoom in to at least the zoom threshold of sensor device 170b (step S508). As a result, the update frequency value of the sensor device 170b thus selected is set to a high value (corresponding to the set short sleep time) (step S510). The sensor data collected by the sensor device 170b is obtained, and the sensor data thus obtained is displayed as an overlay 172b at the position of the sensor device 170b on the user interface 130 (step S512). This corresponds to the updating frequency value of the sensor device 170b from the user interface 130 according to the arrow 190b. Figure 3 (a) to Figure 3 (c) Conversion.

[0057] Fig. 9 The components of the camera device 110 according to an embodiment are schematically illustrated in terms of a number of functional units. The processing circuit 910 is a computer program product 1010 (eg, in the form of a storage medium 930) that can be executed using a computer program product 1010 (eg, Fig.10 The processing circuit 910 may be provided by any combination of one or more of a suitable central processing unit (CPU), a multiprocessor, a microcontroller, a digital signal processor (DSP), etc., which may be provided by software instructions in (i) or (ii). The processing circuit 910 may further be provided as at least one application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). The processing circuit 910 may implement the functions of the computing server 114.

[0058] Specifically, the processing circuit 910 is configured to cause the camera device 110 to perform a set of operations or steps as disclosed above. For example, the storage medium 930 may store the set of operations, and the processing circuit 910 may be configured to retrieve the set of operations from the storage medium 930 to cause the camera device 110 to perform the set of operations. The set of operations may be provided as a set of executable instructions. The storage medium 930 may implement the functionality of the data storage 116.

[0059] Thus, the processing circuit 910 is thereby configured to perform the method as disclosed herein. The storage medium 930 may also include persistent storage, which may be any one or combination of, for example, magnetic storage, optical storage, solid-state storage, or even remotely mounted storage. The camera device 110 may further include a communication interface 920 configured at least for communicating with other entities, functions, nodes, and devices of the system 100. Thus, the communication interface 920 may include one or more transmitters and receivers including analog and digital components. The processing circuit 910 controls the general operation of the camera device 110, for example, by sending data and control signals to the communication interface 920 and the storage medium 930, by receiving data and reports from the communication interface 920, and by retrieving data and instructions from the storage medium 930. Other components of the camera device 110 and related functions are omitted so as not to obscure the concepts presented herein.

[0060] Fig.10 An example of a computer program product 1010 including a computer-readable storage medium 1030 is shown. On the computer-readable storage medium 1030, a computer program 1020 may be stored, which may cause the processing circuit 910 and entities and devices operatively coupled thereto (such as a communication interface 920 and a storage medium 930) to perform methods according to the embodiments described herein. The computer program 1020 and / or the computer program product 1010 may thus provide means for performing any of the steps disclosed herein.

[0061] exist Fig.10 In the example of , the computer program product 1010 is illustrated as an optical disc, such as a CD (compact disc) or a DVD (digital versatile disc) or a Blu-ray disc. The computer program product 1010 may also be implemented as a memory such as a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM) or an electrically erasable programmable read-only memory (EEPROM), and more specifically, a non-volatile storage medium of a device in an external memory such as a USB (universal serial bus) memory or a flash memory such as a compact flash. Thus, although the computer program 1020 is schematically shown here as a track on the depicted optical disc, the computer program 1020 may be stored in any manner suitable for the computer program product 1010.

[0062] The inventive concept has mainly been described above with reference to a few embodiments. However, as readily appreciated by a person skilled in the art, other embodiments than the ones disclosed above are equally possible within the scope of the inventive concept as defined by the appended patent claims.

Claims

1. A system for displaying collected sensor data together with an image, the system comprising: a camera device comprising an image capture unit configured to capture an image depicting a scene, the image capture unit being configured with a variable zoom level, and the camera device being configured to communicate with a user interface for displaying the image; as well as a sensor device configured to collect sensor data and provide the collected sensor data to the camera device at a variable frequency of occurrence for display on the user interface along with the image; Wherein, when the camera device acquires from the image capture unit an image captured at a zoom level above the zoom threshold and depicting the sensor device, the sensor device is configured by the camera device to provide the collected sensor data at a higher frequency of occurrence than when the camera device acquires from the image capture unit an image captured at a zoom level below the zoom threshold.

2. The system according to claim 1, wherein: The camera device is configured to provide the sensor data as an overlay to an image acquired by the camera device and captured at the zoom level above the zoom threshold and depicting the sensor device.

3. The system according to claim 1, wherein: The camera device is configured to display, on the user interface, a graphical representation of the sensor device as an overlay of an image acquired by the camera device and captured at the zoom level below the zoom threshold and depicting the sensor.

4. The system according to claim 3, wherein: The user interface is configured to receive user input, wherein the user input identifies the representation of the sensor device, and wherein the camera device is configured to display the sensor data on the user interface in response thereto as an overlay of an image acquired by the camera device and captured at the zoom level below the zoom threshold and depicting the sensor.

5. The system according to claim 3, wherein: The user interface is configured to receive a user input, wherein the user input identifies the representation of the sensor device, and wherein the camera device is configured to cause the image capture unit to zoom in to the zoom threshold in response thereto.

6. The system according to claim 1, wherein: The sensor device has a position in the scene, wherein, when the sensor device is displayed on the user interface, the position of the sensor device in the scene has a mapping to a position on the user interface, and wherein the camera device is configured to determine that the user input identifies a representation of the sensor device by verifying that the position of the user input on the user interface corresponds to the position of the sensor device on the user interface.

7. The system according to claim 1, wherein: The camera device is configured to establish the zoom threshold during a calibration process.

8. The system according to claim 7, wherein: The image capture unit has a geometric relationship with the sensor device as established during the calibration process, wherein the geometric relationship includes zoom level dependent position information of the sensor device, and wherein the zoom level dependent position information specifies whether the sensor device is visible in an image captured using a given zoom level.

9. The system according to claim 1, wherein: The variable occurrence frequency is variable by being selectable between a first predetermined value and a second predetermined value, and wherein the first predetermined value produces a higher occurrence frequency than the second predetermined value.

10. The system according to claim 9, wherein: The first predetermined value is given by a wake-up time configuration of the sensor device, and the second predetermined value is given by a sleep time configuration of the sensor device.

11. The system according to claim 1, wherein: The system includes at least two sensor devices, each sensor device being configured to collect sensor data and provide the collected sensor data at respective variable frequencies of occurrence for display on the user interface together with a captured image depicting a scene, and wherein each of the at least two sensor devices is associated with its own zoom threshold.

12. The system of claim 1, wherein: The sensor device is a battery powered Z-wave device.

13. A camera device for displaying collected sensor data together with an image, the camera device comprising: an image capture unit configured to capture an image depicting a scene, the image capture unit being configured with a variable zoom level; a communication interface configured to communicate with a user interface for displaying the image and with a sensor device for obtaining collected sensor data from the sensor device for display on the user interface together with the image; as well as Processing circuitry is configured to configure the sensor device, via the communication interface, to provide the collected sensor data at a higher frequency of occurrence when the image capture unit captures and depicts an image of the sensor device at a zoom level above the zoom threshold compared to when the image capture unit captures the image at a zoom level below the zoom threshold.

14. A method for displaying collected sensor data together with an image, the method being performed by a camera device comprising an image capture unit, a communication interface, and a processing circuit, wherein: The image capture unit is configured with a variable zoom level, the method comprising: capturing, by the image capture unit, an image depicting a scene; communicating with a user interface for displaying the image and with a sensor device through the communication interface to obtain the collected sensor data from the sensor device for display on the user interface together with the image; and The sensor device is configured by the processing circuit and via the communication interface to provide the collected sensor data at a higher frequency of occurrence when the image capture unit captures and depicts an image of the sensor device at a zoom level above the zoom threshold compared to when the image capture unit captures the image at a zoom level below the zoom threshold.

15. A non-transitory computer-readable storage medium having stored thereon a computer program for displaying collected sensor data together with an image, the computer program comprising computer code, the computer code, when executed on a camera device, causing the camera device to perform a method for displaying collected sensor data together with an image, the method being performed by a camera device comprising an image capture unit, a communication interface, and a processing circuit, wherein: The image capture unit is configured with a variable zoom level, the method comprising: capturing, by the image capture unit, an image depicting a scene; communicating with a user interface for displaying the image and with a sensor device through the communication interface to obtain the collected sensor data from the sensor device for display on the user interface together with the image; and The sensor device is configured by the processing circuit and via the communication interface to provide the collected sensor data at a higher frequency of occurrence when the image capture unit captures and depicts an image of the sensor device at a zoom level above the zoom threshold compared to when the image capture unit captures the image at a zoom level below the zoom threshold.

Images