Apparatus and method for position determination in a thermal imaging system - Patents.com

Abstract

To provide an imaging system capable of interpreting images obtained during a monitoring service to generate monitoring outputs such as differential identification of the presence, location and / or activity of people, apparent adults, children and pets. [Solution] The imaging system includes a camera assembly and a server, wherein the camera assembly and the server communicate over a computer communications network to identify at least one installation measurement of the camera assembly and establish a mapping from an image coordinate system for images generated by the at least one imaging device to a real-world coordinate system, the at least one installation measurement including a mounting height of the camera assembly above a floor surface on which a user of the imaging system stands.

Description

[Technical field]

[0001] (Related Applications) This application claims the benefit of the filing date of U.S. patent application Ser. No. 15 / 607,345, filed May 26, 2017, entitled “Apparatus and Method of Location Determination in a Thermal Imaging System,” the entire disclosure of which is incorporated herein by reference.

[0002] At least some embodiments disclosed herein relate generally to thermal imaging systems, and more specifically, but not exclusively, to position determination and size measurement in such thermal imaging systems for object recognition and surveillance. [Background technology]

[0003] U.S. Patent Application Publication No. 2015 / 0377711, entitled "Apparatus and Method for Electromagnetic Radiation Sensing," discloses an apparatus for thermal imaging based on infrared (IR) radiation. Such an apparatus may be used for person detection, fire detection, gas detection, temperature measurement, environmental monitoring, energy conservation, behavioral analysis, surveillance, intelligence gathering, and human-machine interface. Such an apparatus and / or other similar apparatus may be used in the embodiments of the invention disclosed herein. The entire disclosure of U.S. Patent Application Publication No. 2015 / 0377711 is incorporated herein by reference. [Brief description of the drawings]

[0004] Embodiments are illustrated by way of example, and not limitation, in the figures of the accompanying drawings in which like reference symbols refer to similar elements and in which: [Figure 1] FIG. 1 illustrates a thermal imaging system according to one embodiment. [Diagram 2] FIG. 1 illustrates a method for measuring mounting configuration parameters of a thermal imaging camera according to one embodiment. [Diagram 3] FIG. 1 illustrates a user interface for obtaining user input to determine the mounting height of the camera according to one embodiment. [Figure 4] FIG. 2 illustrates one embodiment of a process for establishing a positional mapping of a thermal image acquired by a camera and the environment in which the camera is installed. [Diagram 5] FIG. 2 illustrates one embodiment of a process for establishing a positional mapping of a thermal image acquired by a camera and the environment in which the camera is installed. [Figure 6] FIG. 2 illustrates one embodiment of a process for establishing a positional mapping of a thermal image acquired by a camera and the environment in which the camera is installed. [Figure 7] FIG. 2 illustrates the layout of a thermal imaging camera's environment overlaid on a thermal image generated by the camera, according to one embodiment. [Figure 8] FIG. 1 illustrates an application of a position determination system according to one embodiment. [Figure 9] FIG. 1 illustrates a thermal camera assembly with an enclosure installed in a room with an occupant. [Figure 10] FIG. 1 shows a thermal camera assembly having an enclosure mounted on the edges of two orthogonal walls. [Figure 11] FIG. 11 is a rear top view of the enclosure shown in FIG. 10. [Figure 12] FIG. 13 illustrates a thermal camera assembly having alternative orientation marks. [Figure 13] FIG. 13 shows the thermal camera assembly with the base surface of the enclosure omitted. [Figure 14] FIG. 1 shows a thermal camera assembly having a replaceable battery unit at its bottom corner. [Figure 15]FIG. 1 shows a thermal camera assembly having a replaceable battery unit at its bottom corner. [Figure 16] 13A-13C show thermal camera assemblies having alternative shapes for the base surface. [Figure 17] 13A-13C show thermal camera assemblies having alternative shapes for the base surface. [Figure 18] A diagram showing the geometric relationship between the mounting position of the enclosure, the direction of the optical axis of the imaging device contained in the enclosure, the field of view of the imaging device, and the space within the room that can be captured in an image obtained from the imaging device contained within the enclosure. [Figure 19] FIG. 1 illustrates an installation process for an imaging system according to one embodiment. [Figure 20] FIG. 1 illustrates a data processing system that can be used to implement some components of embodiments of the present application. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0005] The following description and drawings are illustrative and should not be construed as limiting. Many specific details are described to provide a thorough understanding. However, in some cases, well-known or conventional details are not described to avoid obscuring the description. References to one embodiment in the present disclosure are not necessarily references to the same embodiment, and such references mean at least one.

[0006] At least some embodiments disclosed herein provide a user-friendly method for determining an installation configuration of a thermal imaging assembly in a thermal imaging system based on thermal images captured during adjustment of the thermal imaging assembly and user input provided in association with the thermal images, which train the thermal imaging system to obtain information about the environment in which the thermal imaging assembly is installed and configured to monitor. The configuration parameters and information about the environment are subsequently used to interpret the images obtained during a monitoring service to generate monitoring outputs, such as differentially identifying the presence, location, and / or activity of people, visible adults, children, and pets.

[0007] For example, the user may provide the height of a person (e.g., the user) detectable in the thermal image during the installation / adjustment of the thermal imaging system to enable the thermal imaging system to calculate the mounting height of a thermal imaging assembly. Other user inputs may include an indication of a time instance when the user is at a point of interest (POI) (e.g., a corner of a room, a door), an identification of a POI, etc., to enable the system to learn the location of the POI within an imaging coordinate system, where the POI may not be directly visible or recognizable from the thermal image.

[0008] During the installation / adjustment, the system may instruct the user to perform activities such as walking away from or towards the camera, going to a point of interest, walking along a path within the area monitored by the camera, traversing an area, etc. The user activities generate thermal images from which the system learns the geographic configuration of the monitored environment.

[0009] Based on the user input and / or the thermal images collected during the installation / adjustment, the system calculates configuration parameters such as the mounting height of the thermal imaging assembly, the ratio or mapping of size in the image to size of people / objects in the monitored area, and the identification of POIs in images captured by the thermal camera. The system registers the location, path and / or area of ​​interest as information about the environment in which the thermal imaging assembly is installed and configured to monitor.

[0010] For example, in one embodiment, a mobile application is configured to request the user to input the height of the user captured in the thermal image presented on the mobile application. Once the mobile application detects the user in the image, the application can prompt the user to input the height of the user or to perform an action, such as going to a point of interest, such as a corner of the room, a door or a window of the room. The mobile application (or a remote server) extracts position and / or size data from the thermal image of the user performing the action and associates it with the instructions and / or any input from the user to determine configuration parameters such as the mounting height of the thermal camera, the position of the point of interest in the thermal image coordinate system, a positional mapping of the thermal image coordinate system to a coordinate system aligned with the room, a size mapping of the size of an object measured in the thermal image coordinate system and the size of a real object in the room coordinate system.

[0011] FIG. 1 illustrates a thermal imaging system according to one embodiment.

[0012] In FIG. 1, the thermal imaging system includes a thermal camera assembly (101) and a server (113) that processes the thermal images captured by the thermal camera included in the thermal camera assembly (101) and provides services based on the thermal images.

[0013] In FIG. 1, the thermal camera assembly (101) transmits the thermal images to the server (113) via a wireless access point (111) and a computer network (115) (eg, a local area network and / or the Internet). A mobile device (117), such as a smartphone, tablet computer, laptop computer, or personal media player, has a mobile application installed therein for interacting with the thermal camera assembly (101) and / or the server (113) for adjusting, configuring, and / or utilizing the thermal imaging system.

[0014] In some cases, the thermal camera assembly (101) sends unedited video to the mobile device (117) and / or the server (113) (e.g., via a wireless or wired connection) without any image processing within the enclosure of the thermal camera assembly (101). A host device (e.g., the mobile device (117) or another computer in or near the room (109)) or the server (113) remote from the installation site performs image processing to provide the user interface and / or to calculate configuration parameters, as described in more detail below. In some cases, the server (113) is implemented using cloud computing third party services provided via a serverless architecture.

[0015] In FIG. 1, the thermal camera assembly (101) is mounted at a location within an environment, for example, a room (109) being monitored by the thermal camera assembly (101).

[0016] Preferably, the thermal camera assembly (101) is mounted at a vertical edge (119) where two walls (171 and 172) of the room (109) meet, at a horizontal edge (102 or 104) where a wall (e.g., 171 or 172) and the ceiling of the room (109) meet, or at a corner (174) of the room (109) where two walls (171 and 172) of the room (109) meet the ceiling of the room (109). Alternatively, the thermal camera assembly (101) may be mounted in other locations, such as at a location on the surface of a wall (171 or 172) or ceiling, or anywhere within a scene. For example, the thermal camera assembly (101) can be configured to be mounted to the ceiling of the room for top-down monitoring, and the thermal camera assembly (101) may be mounted on and / or in conjunction with a holder and / or device such as an IP camera, a passive infrared sensor (PIR), etc.

[0017] Preferably, the thermal camera assembly (101) has an enclosure or housing having a surface adapted to align with a surface of a wall (e.g., 171 or 172) and / or a ceiling of the room (109). Thus, directional alignment of the thermal camera assembly (101) relative to the vertical and horizontal directions can be easily achieved by pressing two or more mounting surfaces of the enclosure or housing of the thermal camera assembly (101) against flat surfaces of the wall (171, 172) and / or the ceiling of the room (109).

[0018] For example, in one embodiment, the exterior mounting surface of the enclosure or housing of the thermal camera assembly (101) has a pre-applied adhesive that is covered with a protective strip that can be peeled off to expose the mounting adhesive. When the enclosure or housing of the thermal camera assembly (101) is pressed against the edge (119) or corner (174) in the mounting position, the mounting surface of the enclosure or housing of the thermal camera assembly (101) is aligned and adhered to the surface of the wall and / or ceiling. The alignment of the mounting surface of the enclosure or housing with the wall and / or ceiling surface provides the alignment of the thermal camera assembly (101) with respect to the horizontal and / or vertical of the room (109).

[0019] Optionally, the enclosure or housing of the thermal camera assembly (101) is fixedly attached to the mounting location via nails, screws, or like elements.

[0020] When the thermal camera assembly (101) is mounted in the room (109) with correct horizontal and vertical alignment, the camera in the assembly (101) has a known orientation relative to the orientation of the room (109). However, the mounting height (123) (e.g., the vertical distance from the floor (127) to the thermal camera assembly (101)) is still unknown to the imaging system.

[0021] The mounting height (123) can be measured (e.g., with a tape measure) and provided to the system via a user interface, such as a graphical user interface provided by a mobile application running on the mobile device (117). Optionally, the orientation of the thermal camera assembly (101) can be determined automatically from a tilt sensor and / or other sensors (e.g., a set of accelerometers and / or a set of magnetic sensors).

[0022] Alternatively, if the thermal camera assembly (101) has two cameras mounted within their enclosures at a known distance from each other (or adjacent to the corner of a room), the server (113) can use the stereo images generated by the cameras to reference stereo images of one or more reference objects to determine the mounting height.

[0023] Alternatively, if the thermal camera assembly (101) has a distance measuring sensor that measures distance based on the time of flight (TOF) of a signal, the thermal camera assembly (101) can automatically measure its mounting height from the floor surface (127). The TOF can be measured based on an ultrasonic signal or a radio frequency signal. Alternatively, the mounting height may be measured via an air pressure sensor and / or a motion sensor. Optionally, the thermal camera assembly (101) includes sensors and / or devices such as a GPS receiver to the determined location of the thermal camera assembly (101), a magnetic sensor to determine the orientation of the thermal camera assembly (101) relative to the earth's magnetic field, light and / or audio devices to provide visual and / or acoustic feedback and / or warnings, air quality monitoring equipment, etc.

[0024] In a preferred embodiment, the imaging system determines the mounting height based on measuring the size of a reference object (131) in the thermal image and receiving an input of the actual size of the reference object.

[0025] For example, the reference object (131) of Figure 1 has an upper point (106) and a lowermost point (105) that are captured in the thermal image generated by the thermal camera assembly (101). The thermal image of the monitor area shown in the projection image plane (103) has an image (133) of the reference object (131) with a corresponding upper point (108) and a corresponding lowermost point (107). Measurement of the size of the image (133) of the reference object (131) and an input identifying the actual size of the reference object (131) can be used to calculate the mounting height (123), as further described below in connection with Figure 2.

[0026] For example, the reference object (131) can be the person installing, adjusting and / or setting up the thermal camera assembly (101) or another person in the monitored area of ​​the room (109), and the height of the person is the actual size of the reference object (131) in the calculation of the mounting height (123). Such an approach greatly simplifies the process of adjusting and / or setting up the thermal imaging system.

[0027] 2 illustrates a method for measuring mounting configuration parameters of a thermal imaging camera, according to one embodiment. For example, the method of FIG. 2 can be used to determine the mounting height of the thermal camera assembly (101) of the imaging system shown in FIG.

[0028] In Fig. 2, the camera at the mounting position (121) has a determined mounting angle (125) with respect to its housing or enclosure that is aligned with the room coordinate system. The measurement of size in the captured thermal image is performed in a predetermined projected image plane (103) or in an image coordinate system that corresponds to a predetermined mounting height (122) that has a fixed geometric relationship with respect to the image plane (103) defined by the mounting angle (125). The predetermined mounting height (122) can be considered as a reference mounting height from a reference floor (126), and the reference height (130) in the vertical direction is projected to have an image (133) in the projected image plane (103).

[0029] In FIG. 2, if the real reference object (131) has the same size as the image (133) in the imaging plane (103), the ratio of the reference height (130) to the height of the real reference object (131) is the same as the ratio of the reference mounting height (122) to the real mounting height (123) to the floor (127) on which the object (131) stands. Thus, the reference mounting height (122) can be scaled up to obtain the real mounting height (123) based on the ratio of the reference height (130) to the height of the real reference object (131). The reference height (130) can be determined from the size and position of the image (133) and the mounting angle (125) of the camera.

[0030] In one implementation, a formula is derived to calculate the reference height (130) from the mounting angle (125), the reference mounting height (122), and the size and position of the image (133). The ratio of the height of the reference height (130) to the height of the real reference object (131) can be used to scale the reference mounting height (122) to the real mounting height (123).

[0031] In another embodiment, the thermal camera assembly (101) is mounted at a reference height (122) in a reference room. References of objects (e.g., 130) of different heights are placed at the positions of the objects (130) shown in FIG. 2 to obtain images (e.g., 133) of different sizes, thereby establishing a mapping between the reference heights and the sizes of the images. If the real object (131) generates an image size at the position in the image plane (103), the mapping can be used to find the reference height (130) in the reference room. The reference mounting height (122) can then be extended to the real mounting height (123) up to the floor (127) according to the ratio between the reference height (130) found from the mapping with the size of the image (133) and the height of the real reference object (131) that generates the image (133) of the same size in the image plane (103).

[0032] In one embodiment, the camera of the thermal camera assembly (101) has a limited field of view. For example, in FIG. 2, the camera cannot capture an area closer to the edge (119) than the line (129). Therefore, if the object (131) generates an image (133) adjacent to the boundary of the image frame, the system may not be able to determine whether the image captures the entire object (131). Therefore, the mobile application running in the mobile device (117) can provide instructions to move the object (131) so that the image (133) leaves the boundary of the image frame and the image (133) does not have pixels at the boundary of the image frame. As soon as the image (133) leaves the boundary, the system captures the image (133) of the object (131) and determines the reference height (130) that will generate the same image size at that location.

[0033] In general, the mobile application generates a display of the thermal image captured by the thermal camera assembly and can provide instructions to guide the movement of the object (131) to a particular location within the room so that the object (131) appears to be standing at a particular location within the thermal image, and the size of the object (131) in the image is used together with the real-world size of the object (131) to calculate the mounting height (123).

[0034] 3 illustrates a user interface for obtaining user input and determining a mounting height for a camera, according to one embodiment. For example, the user interface of FIG. 3 can be implemented on the mobile device (117) in the thermal imaging system of FIG. 1 to calculate the mounting height using the method of FIG. 2.

[0035] In Fig. 3, after the thermal camera assembly (101) is mounted at the mounting location (121), the camera is configured to establish communication with the server (113) and / or the mobile device (117) to generate a thermal image of the monitored area of ​​the room (109) for display on the mobile device (117). The thermal image (141) presented on the mobile device (117) (e.g., using a mobile application running in the mobile device (117)) includes a thermal image (133) of an object (131) in the room (109) that has a temperature significantly different than room temperature. For example, the object (131) is a person in the room (109), such as an installer or owner of the thermal camera assembly (101), or another person in the monitored area of ​​the room (109). The user interface requests the user of the mobile device (117) to input the real-world height of the object (131) (eg, a person) identified in the thermal image (141).

[0036] Optionally, if the thermal image (141) captures the thermal images of multiple objects, the user interface allows the user of the mobile device (117) to select an object (131) and specify the height of the selected object (131).

[0037] The thermal imaging system is configured (e.g., by the mobile device (141), the server (113), or a mobile application running within the thermal camera assembly (101)) to measure the height (137) of the thermal image (133) within the image coordinate system (139).

[0038] In some cases, the thermal imaging system is configured to measure the height (137) when the thermal image (133) is at a particular position in the image coordinate system (139) (e.g., when the thermal image (133) of the object (131) stands at a particular location in the image coordinate system (139)). In such an embodiment, the user interface can provide instructions to guide the movement of the object (131) so that the thermal image (133) stands at the particular location marked in the image (141) presented on the mobile device (117).

[0039] In another example, the thermal imaging system can measure the height (137) and calculate the mounting height (123) using the height (143) provided by the user of the mobile device (117) as soon as the entire thermal image (133) of the object is captured in the image (141) (e.g., no portion of the object (131) is located within a blind spot of the thermal camera assembly (101)), without requiring the object (131) to go to a specific location.

[0040] Optionally, the mobile device (117) instructs the user (131) to move the object (131) (e.g., the user) around the room so that the height (137) of the thermal image (133) of the object (131) can be measured at multiple different locations and the mounting height (123) can be calculated from the measurements at the corresponding locations. The mounting height (123) calculations can be combined (e.g., by averaging the results or by a weighted average of the results) for improved accuracy.

[0041] After the mounting height (123) is determined, the thermal imaging system can match the coordinates in the image coordinate system (139) (of points with known or estimated height) to the coordinates in the room (109).

[0042] Figures 1 to 3 have described the measurement of the mounting height of a thermal camera (imaging based on IR radiation), the techniques can be extended to determine the mounting height of a camera imaging based on light visible to the human eye as well.

[0043] While images of the room (109) captured based on visible light can show features of the room (109) (e.g., the boundaries of the floor and the locations of points of interest, such as the doors, windows, furniture) that can be used to automatically determine the layout of the room (109), thermal images of the room (109) typically do not have enough features that can be used to identify the layout of the room (109), even for a person, especially if the thermal images have a low resolution to protect the privacy of the occupants of the room (109).

[0044] Figures 4-6 show one embodiment process for establishing a positional mapping between a thermal image acquired by a camera and the environment the camera is installed in. The process of Figures 4-6 can be used in the thermal imaging system of Figure 1, along with the method and user interface of Figures 2 and 3, to determine the mounting height of the thermal camera assembly (101).

[0045] 4 to 6, the system is aware that the object (131) is standing on the floor (127) of the room (109). One end (151) of the thermal image (133) therefore identifies the location on the floor (127) where the object (131) is standing. The mobile device (117) provides instructions (145) to move the object (131) to various points of interest in the room (109) so that the system can register corresponding locations in the image coordinate system (139) to generate a layout of the room.

[0046] For example, in FIG. 4, the user of the mobile device (117) is the reference object (131), and the mobile device (117) instructs the user to walk to a diagonal corner of the room (109) so that the location of the diagonal corner on the floor (127) can be marked at a position (151) in the image coordinate system (139).

[0047] For example, the mobile device (117) may instruct the user to provide an indication when the user is at the diagonal corner by, for example, a gesture input caused by shaking the mobile device (117), a voice confirmation provided to the mobile device (117), pressing a predetermined button on the mobile device (117), tapping a user interface element presented on a touch screen of the mobile device (117), standing at the corner for a predetermined period of time, etc.

[0048] 5 illustrates a scenario in which the user is instructed to walk to another corner of the room to register a corner position (153). The mobile device (117) overlays the location identified and registered within the image coordinate system (139) onto the image (141) (e.g., to show the progress of positioning the layout of the room (109)).

[0049] 6 illustrates a scenario for locating the door position (155) in the room (109). For example, the mobile application running on the mobile device (117) may prompt the user to navigate to a point of interest and then name the point of interest (e.g., a door) by voice input, text input, selection from a list, etc. Alternatively, the mobile application may have a list of points of interest (e.g., doors, windows, desks, chairs, TV, fireplace, stove) and ask the user to identify their locations by walking to the locations. For example, the mobile application may ask the user to walk around an area that is accessible by walking to identify areas of the floor that will have walking areas. For example, the mobile application may ask the user to go to a stove and detect the thermal image of the stove and its location, then go to a TV, open a refrigerator, etc., to detect the appearance of the corresponding items (e.g., stove, TV, refrigerator) in the monitored thermal image depending on their temperature changes as a result of the user's actions, and tag the items accordingly in the image coordinate system (139). In another example, the system may ask the user, "A hot spot of 200° C. was detected. Is it the stove?" If the user responds with an indication of "yes," the system stores information to associate the hotspot with the identity of the particular object (the "stove"), otherwise the system can issue an emergency alert for this unexpected hotspot. Optionally, the system monitors the temperature range and / or size of known hotspots (e.g., stoves) and if the detected temperature and / or size exceeds the normal range or size, the system provides feedback or an alarm (e.g., an audio warning that "the stove is starting to burn").

[0050] The locations of points of interest located using the process shown in Figures 4 to 6 can be used to construct a layout of the room (109) being monitored by the thermal camera assembly (101), as shown in Figure 7.

[0051] FIG. 7 shows the layout of a thermal imaging camera's environment superimposed on a thermal image produced by the camera, according to one embodiment.

[0052] In Figure 7, lines (135) are superimposed on the thermal image (141) (e.g., having 160x60 IR-sensitive pixels). These lines of the room layout can be constructed by connecting points of interest (e.g., corners of the room) identified using the process shown in Figures 4-6.

[0053] From the image shown in Figure 7, the mobile device (117), or the mobile application running on the server (113) or another mobile device, can determine the location of the feet of the thermal image of the person in the room. Because the image of the person appears to be standing and / or the location of the feet is within a traffic area (and / or the movement of the thermal image is consistent with the pattern of a person walking in the traffic area), the system can assume that the feet are on the floor (127). Thus, the system can calculate the coordinates of the person in the room (109) based on the location of the feet in the thermal image (141).

[0054] Figure 8 illustrates an application of a position determination system, according to one embodiment. For example, the application illustrated in Figure 8 can be provided using the room layout generated by the process illustrated in Figures 4-6 and a coordinate system mapping that is based on the orientation of the camera relative to the room and the mounting height (123) of the thermal camera assembly (101) of the system illustrated in Figure 1.

[0055] In FIG. 8, the room layout (135) and the mounting height (123) allow the system to calculate the measured floor size of the monitored activity area (eg, 10×10 square meters).

[0056] From the orientation and size of the thermal image (157) of an object (and / or movement history), the system determines that a person of a determined height is on the floor (127) at a given coordinate within the room (109). Such a determination can be utilized to initiate a report of a fall of a monitored person (e.g., an elderly person or a patient).

[0057] In Figure 8, the system determines from the orientation and size of the thermal image (159) of an object (and / or movement history) that two people are at a measured distance from the camera. Such determination can be used to report the presence and / or activity of people within a monitored environment.

[0058] Optionally, the imaging system shown in FIG. 1 is configured to automatically adjust and / or readjust the mounting height (123) and / or other configuration parameters (e.g., location of POI) based on statistical analysis of the thermal images of observed persons over a period of time.

[0059] For example, the imaging system detects the thermal images of many people who have been in the monitored area (e.g., room (109)) over a period of time, and then the system calculates a statistical distribution of the relative heights of the people detected during the period. The mounting height can be scaled to match the distribution of the heights of the detected people with a known distribution of heights of people (e.g., people in the same geographic area and / or people with ages within the expected range of people visiting the monitored area). The mounting height calculated from the height distribution can be used in place of the mounting height calculated from the input regarding the height of the user detected during the installation process, or can be used to review and / or improve the mounting height calculated during the installation process. Thus, adjustment accuracy can be improved over time based on the monitoring results of the imaging system.

[0060] In some cases, an object of known height in the monitored area (e.g., room (109)) can be detected within a certain period of service of the imaging system. For example, if there is a significant temperature difference between the room (109) and the environment outside the room (109), opening the door at some time instance will enable the thermal camera assembly (101) to generate an image in which the door opening area has a thermal image recognizable in the image coordinate system (e.g., the opening area has the temperature of the environment outside the room (109), while the wall to which the door is attached has the temperature of the room). Thus, the height of the door can be determined from the thermal image of the door opening area to adjust the mounting height of the thermal camera assembly (101) relative to a known or standard door height.

[0061] After the thermal camera assembly (101) having an enclosure as shown in Figures 9-15 is placed in an area to be monitored (e.g., room (109)) to capture the scene, a calibration / training process is configured to enable the thermal imaging system (e.g., as shown in Figure 1) to understand the actual geometric data from the captured video and recognize points of interest (POIs) that may not be detectable within the capture band of the imaging device. For example, when thermal imaging in the infrared band is looking at a scene with a uniform room temperature, the POIs are not detectable due to the lack of clear temperature contrast or temperature difference.

[0062] When the enclosures shown in Figures 9 to 15 are used, a set of configuration parameters are known (from the factory configuration) without requiring measurements from the installation site. Such configuration parameters may include image size and aspect ratio, imaging lens parameters (e.g., the field of view), the mounting angle in volumetric space relative to some reference point / plane / coordinate system of the scene (e.g., room (109)). The mounting height (123) can be determined using the method shown in Figures 1-3 or 18.

[0063] The mounting height can be determined using other technical solutions using additional resources. For example, two cameras can be mounted in the enclosure at a known distance from each other and the relevance of the object can be determined by stereoscopic vision. For example, a distance measurement sensor, e.g., a TOF sensor, can be included in the thermal camera assembly (101) to automatically measure its mounting height from the floor surface (127). For example, the installer can be instructed to measure the mounting height (123) using a tape measure and input the measurement via a user interface provided by the mobile device (117) configured for the adjustment of the thermal imaging system.

[0064] Once the configuration parameters are known, the thermal imaging system has a mapping from the image coordinate system (139) and the room coordinate system for real-world geometric understanding from the captured video.

[0065] For example, the mobile device (117) is configured to establish a communication connection with the thermal camera assembly (101) and / or the server (113) (e.g., via a wireless local area network access point (111)) for the adjustment of the thermal camera assembly (101) installed in the room (109). The mobile device (117) provides a user interface to instruct the user to move around the room and to stand on the floor (127) so that the user is fully visible in the thermal image captured by the thermal camera assembly (101). If desired, the user interface may instruct the user to go to one or more suitable locations where the user is fully visible in the thermal image. The mobile device (117) receives the height of the user (e.g., via a user interface or from a data source having the user's height).

[0066] Typically, the system measures the size of the thermal image (133) from a complete thermal image (133) of the user (131) standing vertically on the floor surface (127) and calculates the mounting height (123) of the thermal camera assembly (101) to match the real height of the user (131) projected at the position and the size of the thermal image (133) of the user (131) in the image coordinate system (139). To calculate the mounting height (123), it is not necessary to know the exact position and / or distance of the user (131) relative to the thermal camera assembly (101) or the edge (119) on which the thermal camera assembly (101) rests.

[0067] In some cases, the user or object (131) having a known height may be occluded by other objects that may erroneously display the user or object (131) in the thermal image at a different height than its actual height (e.g., especially in low-resolution infrared images). For example, a hot object or other object of the same temperature may appear to be above the user or object (131) in the image generated by the thermal camera assembly (101), which makes the user or object (131) appear taller in the thermal image. By instructing the user (131) to walk around the monitored area, the system can detect the thermal image of the user (131) in relation to the thermal images of other objects that have temperatures significantly different from the temperature of the room, and can thereby identify suitable locations for measuring the size of the thermal image (133) of the user (131) for the calculation of the mounting height (123). In the preferred position, the thermal images of other objects do not interfere with the measurement of the size of the thermal image (133) of the user (131) in the image coordinate system (139). Optionally, the user interface of the mobile application running in the mobile device (117) displays the thermal images from the thermal camera assembly (101) in real time so that the user can ensure that he or she is standing in a position within a room where the thermal images of other objects do not overlap the thermal image (133) of the user (131) so as to affect an accurate measurement of the height of the thermal image (133) of the user (131) in the image coordinate system (139).

[0068] Calibration via the thermal image of a person of known height is one example. In general, any object of known height / geometry that can be detected by the imaging device of the thermal camera assembly (101) can be used as the reference object (131) in the scene to be photographed, for example, a cup of hot water, a bottle of ice water. Typically, the human body constitutes an ideal reference object in infrared light, due to its good contrast from a typical room temperature background and size similar to the object to be monitored in the scene. Therefore, the use of a human body as a reference presents simplicity and ease in installation and adjustment, which in fact does not require any know-how of technical expertise or any other object / device / assistance to perform this kind of reference making.

[0069] Preferably, communication connectivity between the thermal camera assembly (101) and a centralized remote server (113) allows for storage of calibration information of the thermal camera assembly (101) installed in the monitored area (e.g., room (109)) in the cloud to facilitate cloud computing. Such a configuration allows for a very smooth user experience and a friendly interface (e.g., implemented via a mobile application running in a mobile device (117). Some computational and / or resource intensive tasks can be performed on the mobile device (117) and / or on the server (113). Thus, the cost of the thermal camera assembly (101) can be reduced.

[0070] In a typical installation process, the mobile device (117) (or instruction manual) is configured to: 1. activate the thermal camera assembly (101) (e.g., by peeling back a protective strip between the battery contacts and the battery, or by pressing a button to turn on the device, a power through cable, etc.); 2. (e.g., by scanning a code located on or associated with the thermal camera assembly (101) that has a device unique ID that is used to establish an authenticated communications connection, such as a wireless personal area network connection (e.g., a Bluetooth connection or a near field communication connection), with a mobile application installed on the mobile device (117), e.g., a smartphone, tablet computer, or personal media player, and activate the mobile application and the connection to the thermal camera assembly (101). 3. optionally, establishing a connection to a cloud-based computing and storage system (e.g., a connection between the thermal camera assembly (101) and the server (113) using Bluetooth, ZigBee, etc., independent of the thermal camera assembly (101) or via a low power communication connection to a hub, via the Internet, to conserve energy, in order to establish a connection to the thermal camera assembly (101) and the server (113) in order to establish a connection to the cloud-based computing and storage system (e.g., a connection between the thermal camera assembly (101) and the server (113) using Bluetooth, ZigBee, etc., independent of the thermal camera assembly (101) or via a low power communication connection to a hub, via the Internet, in order to conserve energy, but where the hub is connected to the Internet); 4. optionally, configuring the mobile device (117) to perform at least some of the functions of the server (113) in storing and / or processing the adjustment information; 5.5. to mount the thermal camera assembly (101) on an edge (119) or corner (174) of a room (109), preferably at a position above head height for desired coverage of the area being monitored, in which case the enclosure of the thermal camera assembly (101) simplifies the alignment of the thermal camera assembly (101) with the orientation of the room (109), as shown in Figures 9-17; 6. to identify the approximate mounting height (123) of the thermal camera assembly (101) above the floor surface (127) of the room (109) (e.g., by instructing the user to step back until the user is sufficiently visible and then ascertaining the user's approximate height); 7. The mounting height of a subsequently added thermal camera assembly can be calculated from the heights of objects simultaneously captured by the first thermal camera assembly (101) and the subsequently added thermal camera assembly and the actual height calculated according to the mounting height of the first thermal camera assembly (101), and machine learning can be applied to associate the objects seen by different thermal camera assemblies installed to monitor the same room (109), so that additional thermal camera assemblies (e.g., installed at adjacent or opposite corners and / or edges for the improved fall detection, in which case at least two thermal camera assemblies are installed at adjacent corners or edges) can be configured and installed in a manner similar to that of the first thermal camera assembly (101) in the room (109) without having to perform an operation of identifying their mounting heights; and 8.Optionally, the user is prompted to go to one or more points of interest to identify environmental features within the room (109) that may not be detectable from the thermal image of the room (109), such as opposite corners of the room (109) to define the maximum diagonal distance of the field of view, other corners of the room (109) to help determine the ground plane of the scene, and the location of doors, tables, pillars, furniture, and other objects.

[0071] For example, the mobile application may instruct the user to walk to a location and then press a button on the mobile application (or give a voice command to the mobile application, perform a gesture by twirling or shaking the mobile device (117), or stand at the location for a few seconds) to indicate that the user is standing at the location. This tells the thermal imaging system the geometry / layout of the room (109) and the location and / or size of surrounding objects within the room (109) within the thermal image coordinate system.

[0072] For example, if a room has multiple doors, the installer can simply stand in a doorway, confirm in the mobile application that he / she is standing in a doorway, and select from a set of menu options which location the door leads to (e.g., closet, kitchen, entrance, bathroom, living room, etc.). Such information assists the thermal imaging system in determining foot traffic (e.g., to monitor a store or office space to determine where people are moving, for safety and / or security applications, etc.). This aspect is unique to thermal imaging, since the POIs, which are invisible in the infrared band, are identified by simple user interaction with a mobile application running in the mobile device (117).

[0073] Storage of input parameters, adjustment parameters, POI mapping data, etc. can be in the cloud (e.g., the server (113)), the mobile device (117) and / or the thermal camera assembly (101). The server (113) and / or the mobile device (117) can be configured to reconstruct the geometric relationship between the thermal image coordinate system and the room coordinate system based on the configuration parameters using algorithms and / or look-up tables.

[0074] When configured as described above, a thermal imaging system knows the actual geometric relationship between the thermal imaging space and the real space and / or the points of interest that are not visible to the thermal imaging system. As a result, the system can determine, for example, not only the location and height of people and objects in the field of view of the thermal camera assembly (101), but also how many people have moved within the scene.

[0075] For example, by tracking an individual throughout the landscape, the system calculates how far the individual has walked. Such information can be used to determine whether the elderly person has been sufficiently active within the elderly person's life assumptions, and the cloud-based system can calculate how much energy the subject has consumed / burned and assist the user by interacting with the user to inform the user if the activity is insufficient. For example, the system can alert the monitored elderly person and / or caregiver if the activity is excessive and there is a risk of falling / injury due to fatigue.

[0076] The above-described configuration process can be performed with minimal user requirements, i.e., the installer does not need to have any skills in the relevant technology, does not require any tools for installation, and only an interface device (e.g., a smartphone, tablet, or computer with an application or other pre-installed communication port) and user height input is used for determining the parameters.

[0077] In one aspect, the present application provides an imaging system that can automatically determine all configuration parameters for determining the geometrical relationship of the field of view via a distance measuring sensor or multiple (known) camera configurations. If such an engineering solution is not available (e.g., due to cost), a "simpler" imaging system determines the final configuration parameters (e.g., mounting height) based on input from the user, i.e., either the mounting height or the height of the user / installer / person performing the adjustment.

[0078] In another aspect, the present application provides a method for a person to become a reference (marker) for providing novel functional contextual geometric information and key points (e.g., points of interest) within the field of view of an Internet-connected thermal imaging device that has computing and storage capabilities (e.g., via cloud computing) (and interacts with the user via a user interface) under a set of instructions, without requiring the user / installer / person to have technical proficiency.

[0079] In a further aspect, the present application provides a cloud-based thermal imaging system that reconstructs and provides contextual information of a scene through simple configuration decisions.

[0080] The wirelessly connected thermal imaging system can use low-resolution thermal video to monitor occupancy and activity within a human scene without revealing information about the person's identity, and from the thermal video, the system determines the person's activity and well-being.

[0081] Optionally, the thermal camera assembly (101) includes an audio device that provides audio signals to occupants within the monitored area (e.g., room (109)). The audio signals may include voice instructions and / or prompts streamed from the server (113). In such embodiments, the user interface described above in connection with the mobile device (117) may be replaced or augmented with an audio-based interface. For example, the adjustment / installation instructions provided by the mobile device (117) may be replaced and / or augmented with voice instructions streamed from the server (113). For example, if the user (131) is in a position where the thermal image (133) of the user (131) is completely captured by a frame of an image produced by the thermal camera assembly (101), an audio prompt instructs the user to remain standing in that position while the system announces the height, and to begin walking if the announced height matches the height of the user (131). Thus, the user's height can be conveniently provided to the system via a combination of audio prompts and thermal image feedback through movement or lack of movement (and / or other gestures) that can be detected from the image produced by the thermal camera assembly (101).

[0082] Optionally, the thermal camera assembly (101) includes a microphone for receiving audio input from the user in the area monitored by the thermal camera assembly (101). Additionally, the thermal camera assembly (101) may include a light-based indicator for user interaction. In some cases, the thermal camera assembly (101) provides the user interface using a communication connection to an independent device having audio and / or visual capabilities for using the audio and / or visual capabilities. For example, an existing voice-based intelligent personal assistant (e.g., in the form of an Internet-connected mobile device (117)) can be installed in the room (109) and connected to the access point (111), and the thermal camera assembly (101) can connect to the personal assistant via the access point (111) to provide the interface for voice-based interaction, thereby eliminating the need to provide the user interface via a tablet computer (117).

[0083] The audio and / or visual devices of the thermal camera assembly (101) can be used to provide a variety of services, for example, in the case of monitoring the elderly, when the elderly cannot hear properly, some light indicators can be activated based only on the presence of a person.

[0084] The thermal camera assembly (101) may be connected to other connected devices and / or systems, such as a landline phone. For example, if a phone is placed in the living room and rings, a signal is sent to the thermal camera assembly (101) via the cloud (e.g., the server (113)), which indicates from thermal imaging that the user (131) is present in the house, and provides the user (131) with an audio and / or visual indication (e.g., light, beep, voice prompt, etc.) of the event. For example, the thermal camera assembly (101) may provide an indication of an event from an intrusion alarm system. From child monitoring to elderly care, the benefits of the system include non-intrusive person presence information and non-wearable remote notification of events. The system can provide an attractive solution where a grandmother does not need to wear or carry anything herself to be notified of some events occurring in the house. The system provides the notification based on the information regarding the grandmother's location and / or the activities of the grandmother. The notifications can be filtered and / or provided in a rational manner based on the person's activities observed by the thermal camera assembly (101). For example, if the thermal image of the grandmother is consistent with the grandmother sleeping or watching TV, certain alarms or notifications are suppressed.

[0085] 9 shows a thermal camera assembly with an enclosure installed in a room having an occupant. For example, the thermal camera assembly (101) mounted in a room (109) can be connected to the server (113) and / or the mobile device (117) to form a thermal imaging system as shown in FIG.

[0086] In FIG. 9, the thermal camera assembly (101) has an enclosure having dimensions adapted to simplify the process of aligning the orientation of the thermal camera assembly (101) with the horizontal and vertical directions of the room (109), as will be further described in relation to FIG. 10.

[0087] FIG. 10 shows a thermal camera assembly having an enclosure mounted on the edges of two orthogonal walls.

[0088] FIG. 11 shows a rear top view of the enclosure shown in FIG. 10, and FIG. 13 shows the thermal camera assembly of FIG. 10 with the base surface of the enclosure (167) removed (or made transparent) to clarify the thermal camera (175) and its optical axis (177).

[0089] 10, the enclosure of the thermal camera assembly (101) is designed to house and carry the thermal camera (175) and / or other components of the thermal camera assembly (101). The thermal camera (175) is mounted and aligned with respect to the direction / orientation of the enclosure of the thermal camera assembly (101) such that when the orientation of the enclosure is aligned with the orientation of the room (109), the thermal camera (175) has a known orientation with respect to the orientation of the room (109).

[0090] The enclosure of the thermal camera assembly (101) has at least two orthogonal mounting surfaces (162 and 163) as shown in Figure 11. The mounting surfaces (162 and 163) may be orthogonal to one another or may be substantially orthogonal to one another (e.g., having an angle between 85 degrees and 95 degrees, or between 88 degrees and 92 degrees).

[0091] It is assumed that the walls (171 and 172) of the room (109) are the two vertical surfaces within the room (109), the floor (127) and the ceiling (173) of the room (109) are the two horizontal surfaces of the room (109), the edges (119) where the walls (171 and 172) meet are in the vertical direction of the room (109) and perpendicular to the floor (127) and the ceiling surface of the room (109), and the edges where the walls (171 or 172) and the ceiling meet are in the horizontal direction.

[0092] Thus, when the thermal camera assembly (101) is pressed against the edge (119) where the two walls (171 and 172) meet, the mounting surface (162) aligns with the walls (171 and 172), respectively, which guides the thermal camera assembly (101) in an orientation that is consistent with the orientation of the room (109), where the mounting surfaces (162 and 163) are parallel to the walls (171 and 172), respectively, and the rear edge (166) of the thermal camera assembly (101) is aligned with the walls (171 and 172) and aligned with the room (109). The thermal camera (175) is mounted on a thermal wall (109) such that the thermal camera (175) is mounted on a thermal mounting surface (162) that is parallel to the edge (119) of the thermal camera assembly (101) which is parallel to the vertical of the room (109), the top surface (164) of the thermal camera assembly (101) is parallel to a horizontal plane of the room (e.g., the floor (127) and / or the ceiling (173)), the optical axis (177) of the thermal camera (175) has a predetermined orientation relative to the mounting surfaces (162 and 163) and the walls (171 and 172), and the optical axis (177) of the thermal camera (175) is in the vertical plane of the room and has a predetermined orientation relative to the vertical of the room (109).

[0093] The top surface (164) may also be configured as a mounting surface with one or more mounting elements (e.g., adhesive elements) as needed. Thus, the thermal camera assembly (101) may be pressed against an edge (119) where a wall (e.g., 171 or 172) meets the ceiling (173), or against a corner (174) where two walls (171 and 172) meet the ceiling (173). The alignment of the orientation of the thermal camera assembly (101) with the direction of the room (109) may be easily achieved by pressing the thermal camera assembly (101) against the edge (119, 102, or 104) or corner (174) where the thermal camera assembly (101) is attached.

[0094] 10 and 11 show an embodiment of marking the top surface (164) with a directional indicator (169) that can be used to avoid accidentally pressing the top surface (164) against a wall (171 or 172) and mounting the thermal camera assembly (101) at an angle.

[0095] 12 shows a thermal camera assembly (101) with an alternative directional marker (169) on the side (162) of the enclosure of the thermal camera assembly (101). The directional marker (169) includes an arrow pointing upward and the word "UP" to clarify the intended mounting orientation of the thermal camera assembly (101) along the vertical edge (119) of the room (109).

[0096] In Figures 10 and 11, the directional marker (169) includes the word "TOP" as an installation instruction to indicate that the surface (164) is the top surface for mounting the thermal camera assembly (101).

[0097] In general, the directional markers (169) can be graphic indicators (e.g., arrows) regarding the intended installation direction with or without letters or numbers as installation instructions. For example, the bottom indicator can be marked with a foot or shoe, and the top indicator can be marked with a light bulb, sun, clouds, roof, ceiling, or any symbol that intuitively indicates the correct installation position.

[0098] Preferably, at least one of the mounting surfaces (162, 163 and / or 1644) has mounting elements (e.g., adhesive elements) to simplify the process of mounting the thermal camera assembly (101). For example, the mounting element may be a double-sided adhesive film, which requires the installer to simply peel off a protective layer and place the enclosure in contact with a wall. The adhesive film provides a sufficient bond such that no further tools are required for the installation and alignment of the thermal camera assembly (101) enclosure. Alternatively, mounting can be accomplished via nails, bolts, screws, holes for wall mounting hooks, etc.

[0099] The orthogonal mounting surfaces (162 and 163) shown in FIG. 11 are adapted to enable the enclosure of the thermal camera assembly (101) to be mounted to vertical edges (119) of two substantially orthogonal walls (171 and 172) of a room (109), as shown generally in FIG.

[0100] In Figure 10, the orthogonal mounting surfaces (162 and 163) are not visible from the depicted perspective because they face walls (171 and 172), respectively. In Figure 11, the enclosure of the thermal camera assembly (101) is shown with the orthogonal mounting surfaces (162 and 163) facing the viewer of Figure 11.

[0101] This particular geometry of at least two orthogonal mounting surfaces has the advantage that when the enclosure of the thermal camera assembly (101) is mounted at a location where the walls and / or ceiling (173) of the room abut one another at a corner of the room, the walls and / or ceiling (173) of the room act to limit the number of possible orientations in which the thermal camera assembly (101) may be mounted within a room (109). The remaining variables for mounting a thermal camera assembly (101) or the like, including at least two orthogonal mounting surfaces, at a substantially orthogonal vertical edge of a room, are the possible mounting heights (123) of the thermal camera assembly (101) from the floor (127) of the room (109) and the relative orientation of the enclosure along the vertical edge of the room (e.g., upwards versus downwards).

[0102] In any case, the installation procedure of such a thermal camera assembly (101) having an enclosure with a particular geometry is simple enough to be performed by a person without any technical skills or tools. For example, the installation of the assembly (101) can be performed by attaching a double-sided adhesive tape (e.g., as an adhesive element) to one or two mounting faces (162 and 163) (and / or face (164), if necessary) of the enclosure of the thermal camera assembly (101) and bringing these faces into contact with the vertical walls (171 and 172) (and the ceiling (173), if necessary), respectively, as shown in FIG. 10. The installation can also be performed by bringing the enclosure, which is in contact with only one mounting face 2 or 3 (e.g., either of which includes an adhesive film), respectively, close to the vertical edge of the room and up to the wall (11a or 11b). The orthogonal mounting surfaces (162 and 163) form an intuitive shape for an installer to mount it to a substantially orthogonal edge of a room. Furthermore, this solution allows the enclosure of the thermal camera assembly (101) to be installed by a person with unsteady or shaky hands or arms. In fact, the solution requires very low motion or tactile sensitivity to properly mount the enclosure within a room with the orthogonal surfaces touching or proximate to the vertical edge of the room.

[0103] Such enclosure geometries may include, for example, a tetrahedral shape as seen generally in Figures 10 and 11, where the tetrahedron may optionally include a third orthogonal mounting surface (164) for contacting the ceiling (173) during installation (with or without adhesive / mounting elements). The mounting surfaces (e.g., 162, 163 and / or 164) may be planar and either solid or perforated, provided there is sufficient material to provide a mounting surface that attaches to the wall (171) and / or the wall (172) (and / or the ceiling (173) of the room (109)).

[0104] In a tetrahedral embodiment, opposite the apexes (161) of the orthogonal faces (referred to as orthogonal apexes (161)), the enclosure of the thermal camera assembly (101) may also include a room-facing base surface (165), which is shown diagrammatically in FIG.

[0105] Preferably, the base surface (165) is not transparent and the thermal camera (175) is not visible to humans within the captured field of view of the thermal camera 175 (eg, as shown in FIG. 10).

[0106] U.S. patent application Ser. No. 14 / 750,403, entitled "Apparatus and Method for Electromagnetic Radiation Sensing", filed June 25, 2015 and published as U.S. Patent Application Publication No. 2015 / 0377711, U.S. patent application Ser. No. 14 / 788,286, entitled "Micromechanical Device for Electromagnetic Radiation Sensing", filed June 30, 2015, U.S. patent application Ser. No. 14 / 810,363, entitled "Micromechanical Device for Electromagnetic Radiation Sensing", filed July 27, 2015, and / or U.S. patent application Ser. No. 15 / 188,116, entitled "Fabrication Method for Micromechanical Sensors", filed June 21, 2016, may be used as the thermal camera (175) disposed within the enclosure of the thermal camera assembly (101). However, other imaging modalities may also be used.

[0107] The imaging device may be, for example, a low-resolution thermal imaging unit having 30x20 thermal infrared pixels to capture the scene at a low frame rate (e.g., 1 frame per second, less than 9 frames per second) and transmitting such images wirelessly to a remote receiving unit (e.g., the server (113) or the mobile device (117)), with the enclosed, i.e., invisible, low-resolution thermal imaging unit and the transmitting unit being powered by a battery housed within the enclosure of the thermal camera assembly (101).

[0108] 14 and 15 show a thermal camera assembly with a replaceable battery in its bottom corner.

[0109] In Fig. 14 and Fig. 15, the thermal camera assembly (101) is shown diagrammatically with a replaceable battery unit (179) at its bottom corner. The replaceable or replaceable unit does not have to be at the bottom corner and can be located anywhere in the enclosure. In the diagrammatic embodiment of Fig. 14 and Fig. 15, the replaceable battery unit (179) can be removed from the enclosure 1 by simply pushing it once. The attachment mechanism of the replaceable battery unit (172) to the enclosure can be via a locking spring similar to a card adapter, in which case a "push in, push out" mechanism is used to attach or remove the member from its dedicated socket. The replaceable battery unit (179) can be replaced with a new battery unit or can have an interface, for example a USB interface, so that the unit can be recharged via the USB interface. If desired, a secondary battery unit can be integrated within the enclosure (not visible from the perspective views of Figures 14 and 15), i.e., when the replaceable battery unit (179) is removed from the enclosure, the power source for operation of the thermal camera (175) within the enclosure is temporarily powered by the secondary battery while the replaceable battery unit (179) is being replaced.

[0110] The base surface (165) may have the shape of an equilateral triangle (e.g., all with 60 degree angles, as shown diagrammatically in Figures 10, 11 and 13), or may be irregular, or may be spherical or flat.

[0111] In other embodiments, the base surface (165) of the enclosure of the thermal camera assembly (101) has a curved or spherical shape, as shown generally in FIG.

[0112] 16 and 17 show thermal camera assemblies having alternative base surface shapes.

[0113] The embodiment shown in Figure 16 includes three orthogonal mounting surfaces (162, 163, and 164) that are not visible from the perspective view shown. The mounting surfaces (162, 163, and 164) are planar and either solid or perforated, provided there is sufficient material to provide a mounting surface for mounting to the walls (171, 172) and / or ceiling (173) of the room (109).

[0114] In some embodiments, an enclosure having a spherical base surface (165) can have up to three orthogonal mounting surfaces and can include a shape that can be referred to as an eight-piece cut in a rounded spherical or elliptical form (e.g., a ball) (as shown in FIG. 16) or a ginkgo cut in a rounded spherical or elliptical form (as shown in FIG. 17).

[0115] In such an embodiment, the enclosure can be mounted at a substantially orthogonal ceiling corner of the room (109) (e.g., as shown in FIG. 16) if the room can be conveniently reached by the installer. Alternatively, such an embodiment can be mounted at the vertical edge (119) of the room (109) with a standoff distance between the top surface of the thermal camera assembly (101) and the ceiling (173).

[0116] In some cases, the ceiling has a strip of material (a recessed, molded or tray ceiling) between the transition from the ceiling to the vertical wall, and in such cases the enclosure 1 can be attached to the vertical edge (119) of the room (109) as shown in FIG. 17.

[0117] Typically, the enclosure of the thermal camera assembly (101) (e.g., having an overall tetrahedral shape, or a spherical / elliptical / ellipsoidal shape) has two or three orthogonal faces, one or more of which may be configured with an adhesive to act as an adhesive mounting surface for bonding to a wall and / or ceiling, as required. In some cases, only one mounting surface may be configured with an adhesive layer for attachment to a wall (171 or 172) or ceiling (173). By contacting the enclosure of the thermal camera assembly (101) to a wall adjacent to a vertical edge (119) of a room (109) or a ceiling corner (174), the thermal camera assembly (101) may be attached to the wall under a simple set of instructions in an uncomplicated installation procedure, without the requirement of any technical tools or skills.

[0118] In some embodiments, the enclosure of the thermal camera assembly (101) has only two orthogonal mounting surfaces (162) and (163) and a curved base surface (165) (e.g., having a shape consisting of a portion of a spherical or elliptical surface) that connects the two orthogonal mounting surfaces (162 and 163), as shown generally in FIG.

[0119] The shape of the enclosure shown in Figure 17 may be described as a quarter oval sliced ​​along the major axis of the oval. Orientation indicators may be provided on mounting surfaces (162 or 163) (not visible in Figure 17) to indicate the desired mounting orientation of the oval shape of the enclosure along the vertical edge (119) of the room (109). For example, the directional indicator may be a single point with a separate set of instructions explaining that the single point should be closer to the ceiling of the room to ensure proper orientation of the enclosure when mounting the enclosure to a vertical edge of the room.

[0120] In some embodiments, the enclosure of the thermal camera assembly (101) does not include three or more orthogonal mounting surfaces (each orthogonal to the remaining mounting surfaces) to enable and ensure simplicity of installation.

[0121] One advantage of this mounting procedure is that the vertical walls act to constrain the orientation of the enclosure, such that the orientation of the base surface (165) relative to the floor and walls of the room is known with a high degree of confidence, and without the need to measure the orientation after installation.

[0122] FIG. 18 shows the geometric relationship between the mounting position of the enclosure, the direction of the optical axis of the imaging device contained in the enclosure, the field of view of the imaging device, and the space within the room that can be captured in images obtained from the imaging device contained within the enclosure.

[0123] Within the thermal imaging assembly (101), the thermal camera (175) is mounted such that it has a predetermined orientation with respect to its enclosure (e.g., a desired alignment of its optical axis (177) with respect to the base surface (165)) such that when the enclosure of the thermal camera (175) is mounted in alignment with the walls (171, 172) and / or ceiling (173) of the room (109), the thermal camera (175) achieves substantial alignment with the area of ​​interest within the room (109), as shown generally in FIG. 13. This mounting of the thermal camera (175) to the enclosure, together with the alignment of the base surface (165) being restricted by placing the enclosure of the thermal imaging assembly (101) in a room corner (174) or at a room's vertical edge (119), ensures that the thermal camera (175) views the room (109) on a well-defined axis (177) relative to the walls (171 and 172) and the floor (127) of the room (109).

[0124] The desired orientation of the axis (177) of an imaging device (e.g., the thermal camera (175)) relative to the enclosure depends on many factors, such as what is most useful for the imaging device and application, to achieve desired device coverage, or to target a particular room geometry.

[0125] In one embodiment, the mounting of the imaging device (e.g., the thermal camera (175)) within the enclosure 1 is configured such that the imaging axis (177) bisects the angle between the two mounting walls (171 and 172) relative to the horizontal.

[0126] In one embodiment, the mounting of the imaging device (e.g., the thermal camera (175)) within the enclosure 1 is configured so that the imaging axis (177) is equally perpendicular or tilted to the vertical edges of the room.

[0127] In some cases, the imaging device (e.g., the thermal camera (175)) has a field of view (capture angle) of 90 degrees or more. When such imaging devices are used, fixed, symmetric orientation of the image device (e.g., the thermal camera (175)) within the enclosure in a square room can provide substantially complete room coverage or coverage of a reasonable percentage of the room.

[0128] Optionally, the enclosure houses two or more imaging devices (e.g., thermal cameras (175)). In such a case, the enclosure 1 includes a fixed mount for each imaging device (e.g., the thermal camera (175)) such that the optical axis (177) of each imaging device (e.g., the thermal camera (175)) can be fixed relative to the base surface (165). In one possible embodiment, the optical axes (177) of the imaging devices can be uniformly distributed in the horizontal and / or vertical planes.

[0129] In some cases, the optical axis (177) may have a tilt angle from the horizontal plane that is parallel to the ceiling (173) or the floor (127). The mounting of the enclosure can be performed at or above a typical person's head height, or even at a corner (174) of the ceiling of a room (109), such that the imaging device (e.g., the thermal camera (175)) has an optical axis (177) pointed at the room that includes a tilt angle with respect to the horizontal plane with the device "looking down" at the room (109). Orientation markers (169) on the enclosure serve as indicators to ensure that the enclosure is correctly oriented with the imaging device pointed at the room and toward the floor (127) of the room (109).

[0130] Generally, multiple imaging devices can be housed within the enclosure of the thermal camera assembly (101) depending on the size of the field of view of the imaging device. For example, if imaging devices with a field of view of 90 degrees or more are used for corner or edge mounting, one imaging device may be sufficient. For imaging devices that each have a limited field of view (e.g., 30 degrees), an array of imaging devices (e.g., 3×3) can be configured such that their combined fields of view cover the room.

[0131] The problem of the imaging device (e.g., the thermal camera (175)) (or multiple imaging devices) within the enclosure being visible to a person standing in front of the enclosure 1 is eliminated by the base surface (165) being visually opaque or translucent from outside the enclosure.

[0132] Such visually opaque surfaces could be infrared-transparent materials if the imaging device (e.g., the thermal camera (175)) in the enclosure 1 detects or emits in the infrared range (e.g., as in the related applications mentioned above). In some embodiments, such visually opaque but infrared-transparent surfaces can be formed from polymeric materials, such as polyethylene (PE) or polypropylene (PP). Such polymeric materials appear white or opaque in the visible range of the human eye, but are transparent in the infrared range. Other visually opaque but infrared-transparent materials include germanium (Ge) or silicon (Si). These materials appear "black" to the human eye in the visible range, and visible light cannot pass through such materials due to the lack of transmission in the visible range. In another example, the surface may be a partially transmissive mirror (one-way or two-way mirror, i.e., a visually transparent material coated with a thin metal layer), in which case a person facing the plane sees the reflective surface as the base surface (165), but the imaging device (e.g., the thermal camera (175)) can image through the partially visually transparent surface.

[0133] In at least some embodiments, the enclosure for the imaging device (e.g., the thermal camera (175)) is configured for simplified mounting procedures, with the imaging device (e.g., the thermal camera (175)) configured within the enclosure with a constant "self-aligning" viewing angle, with orientation limited by its mounting surface contacting the wall of a substantially orthogonal vertical edge of a room. Thus, if a particular orientation of the imaging device (e.g., the thermal camera (175)) within the enclosure is known, if the viewing angle (185) of the imaging device (e.g., the thermal camera (175)) is known, and if the approximate mounting height (123) is known, then a space being monitored by the imaging device can be calculated to determine whether the space contains one or more standing subjects (131) having a height (183) and positioned at a distance (181) and at an angle that is within the horizontal plane of the room (109) (e.g., relative to the walls (171 and 172)).

[0134] Alternatively, the desired mounting height (123) can be calculated from the distance (181) between the farthest subject having a height (183) and the orientation of the region of the optical axis (177) relative to the thermal camera assembly (101) and the viewing angle (185) of the imaging device (e.g., the thermal camera (175)) to form a desired space covered by the thermal imaging assembly (101). In one embodiment, the orientation of the region of the optical axis (177) relative to the enclosure and the angle of the viewing angle (185) of the imaging device (e.g., the thermal camera (175)) are predetermined by the manufacture of the enclosure.

[0135] In one embodiment, the set of installation instructions for a user / installer instructs the user to peel off a protective layer from the double-sided adhesive tape already pre-applied by default to the enclosure mounting surfaces (162 and 163) of the thermal camera assembly (101) and to mount the enclosure to a substantially orthogonal vertical edge (119) of the room (109) at a height (123) of about six feet or more above the floor level (127).

[0136] In some embodiments, the imaging device (e.g., the thermal camera (175)) in the enclosure has, for example, about 30x20 pixels with a horizontal and vertical field of view (185) slightly greater than 90 degrees. The battery-powered thermal camera (175) is activated by the user, for example, by pressing a button on the mounting surface (162), or by removing a contact-stopping tape from a battery compartment, or by remote activation via a handheld computer (e.g., 117). The thermal camera assembly (101) wirelessly streams recorded video to a receiver (e.g., using a radio transmitter for a wireless local area network, a wireless personal area network, Wi-Fi, Bluetooth, ZigBee, radio transmission, cellular communication, etc.). The low resolution of the thermal camera (175) provides privacy protection for the room's occupants. In such an embodiment, the base surface (165) can be a white, visually opaque film formed from a thin PE membrane that conceals the contents of the enclosure, specifically the imaging device (e.g., the thermal camera (175)).

[0137] The orientation of the imaging device (e.g., the thermal camera (175)) within the enclosure can be symmetrical within the horizontal plane and symmetrical with respect to the vertical plane, where the horizontal plane can be defined as a substantially plane parallel to the floor (127) of the room (109) and the vertical plane can be defined as a substantially plane parallel to one of the mounting walls of the room. For example, the orientation of the imaging device (e.g., the thermal camera (175)) within the enclosure can be such that its optical axis (177) is oriented 45 degrees downward with respect to the rear edge (166) that connects to and has equal angles with respect to the faces (162 and 163). For example, the orientation of the imaging device (e.g., the thermal camera (175)) within the enclosure can be such that its optical axis (177) is aligned in a plane that vertically bisects the enclosure (e.g., through the vertical edge connecting faces 2 and 3) and has a predetermined angle (e.g., 45 degrees) relative to the vertical edge. As shown diagrammatically in the two-dimensional cross-sectional view of Figure 18, captured images can be analyzed for the position of a subject (131) within the field of view (185), and the height (183) and width of the subject (131) with the known orientation of the camera preset at the time of manufacture, its preset field of view (185), and its approximate mounting height (123).

[0138] In FIG. 18, if the mounting height (123) of the thermal camera assembly (101) is known, for example by any of the methods described above in connection with FIGS. 1-3, and the optical axis (177) and field of view (185) are known from the design and manufacture of the enclosure of the thermal camera assembly (101), then the observable spatial location and the distance (181) between subjects or objects (131) within the field of view of the device 20 can be determined.

[0139] In FIG. 18, the dotted lines from the thermal camera assembly (101) project the pixel inversely onto the floor surface (127) and the opposite wall. The thermal radiation between adjacent dotted lines is measured by corresponding pixels in the thermal camera (175) within the thermal camera assembly (101). Thus, the space between the dotted lines represents the space imaged by the corresponding pixels.

[0140] For example, the thermal radiation projected onto the thermal camera assembly (101) between the dotted lines (188 and 189) is measured by pixel 1, and the thermal radiation projected onto the thermal camera assembly (101) between the dotted lines (187 and 188) is measured by pixel 2, and so on. The thermal intensities measured by pixel 1, pixel 2, and others constitute a vertical line (186) consisting of pixels in the thermal image. The thermal image (131) of the subject or object (131) is represented by shaded pixels (183). For a given mounting height (123) and the field of view (185), the count of the pixels (181) up to the bottom of the thermal image (133) of the object (131) corresponds to a predetermined horizontal distance (181) between the position of the subject or object (131) and the edge (119) on which the thermal camera assembly (101) is mounted. The count of the shaded pixels indicates the height (183) of the thermal image (133) of the subject or object (131) in the image coordinate system (139), which corresponds to the actual height of the subject or object (131) above the floor surface (127) of the room (109) considering the mounting height (123). Also, the geometric relationship can be used in the reverse direction to determine the mounting height (123) based on the actual height of the subject or object (131) and the count of the shaded pixels representing the height (183) of the thermal image (133) of the subject or object (131) at the position identified by the count of the pixels (181) below the shaded pixels.

[0141] In Figure 18, a one-dimensional vertical pixel column (186) shows how the subject (131) in the room (109) appears in the thermal image captured by the thermal camera assembly (101). Emission from the subject (131) causes the shaded pixels to be measured to have a significantly different temperature than other areas measured by the non-shaded pixels. The non-shaded pixels represent the portions of the room measured at room temperature, and the shaded pixels represent the elevated surface temperature of the subject (131) above the room temperature.

[0142] In FIG. 18, the vertical columns of pixels (186) are identified as "pixel 1," "pixel 2," etc., with corresponding imaging space labelled "pixel 1," "pixel 2," etc.

[0143] Assuming the subject (131) is standing vertically in the room (109), his or her height (183) and position (181) can be determined by trigonometric relationships. A similar embodiment is valid for the horizontal dimension, which allows the subject or object's position to be determined within the horizontal dimension and its width. This is valid for any object that has a different temperature than the room temperature in the case of thermal infrared imaging.

[0144] For example, hot or cold spots can be located by knowing their location and their relative size in addition to their relative temperature. A hot spot could include a dangerous item, such as an iron that has been left on because the user accidentally forgot to turn it off, potentially creating a fire hazard or safety issue, or a cold spot could include an open window that a person forgot to close, allowing very cold air to flow into the room. Many cold spots and hot spots can be detected by low-resolution thermal imaging devices, and therefore three-dimensional information of the viewed scene can be reconstructed from the images recorded by the thermal camera assembly (101).

[0145] The embodiment of Figure 18 is simplified to the case of a two-dimensional cross-section with one vertical pixel column (186) of 20 pixels representing the vertical imaging volume of the thermal camera assembly (101) in such an embodiment. The imaging device preferably has a viewing volume of a matrix of pixels, for example an array of columns equal to 30 pixels in the horizontal direction by 20 pixels in the vertical direction.

[0146] If desired, additional functionality such as decorative surfaces, lighting, Wi-Fi access points / repeaters, etc. on the visible side of the base surface (165) may be integrated within the enclosure of the thermal camera assembly (101).

[0147] Optionally, the enclosure of the thermal camera assembly (101) can have rounded vertical edges or rounded corners / edges and rounded apexes (161) for easy fitting / mounting into rounded corners of a room.

[0148] Optionally, an adapter enclosure is permanently fixedly attached to the wall, and the enclosure of the thermal camera assembly (101) housing the imaging device (e.g., the thermal camera (175)) and / or other components of the camera (e.g., a battery) is attached to the adapter enclosure so that the thermal camera assembly (101) can be easily replaced without removing the entire assembly.

[0149] If desired, any of the components of the thermal camera assembly (101) located within the enclosure, such as one or more of the imaging devices (e.g., the thermal camera (175)), the battery, wireless module, electronics boards, etc., can be designed to be replaceable or replaceable within the enclosure, but the enclosure can be permanently fixedly mounted to the wall without removing the entire assembly. For example, the battery module can be replaceable in the manner shown in Figures 14 and 15, and other replaceable modules can be similarly configured for the thermal camera (175), any wireless modules, etc.

[0150] Optionally, wedges are provided for mounting between the enclosure of the thermal camera assembly (101) and the walls (e.g., 171 and 172) and / or ceiling (173) of the room (109) when the walls and / or the ceiling are not substantially perpendicular to one another.

[0151] FIG. 19 illustrates an installation process for an imaging system according to one embodiment, such as the imaging system of FIG.

[0152] In FIG. 19, a user or installer is instructed to mount (191) a camera assembly (101) in a mounting location (e.g., along a vertical edge (119)) that is aligned with the horizontal and vertical directions of an area (e.g., a room (109)) to be monitored by the camera assembly (101), to operate the camera assembly (101) to establish communication with a remote server (113) to configure (193) an imaging system (e.g., as shown in FIG. 1), to receive instructions from the imaging system to calibrate or train (195) the imaging system in relating environmental elements of the area to elements and / or locations in images generated by the camera assembly (101), and to perform actions in accordance with the instructions and generate input (197) via features identifiable in the images generated by the camera assembly (e.g., using the user or installer (131) as a reference). If desired, the user or installer may further connect (199) a mobile device (117) to the imaging system to receive instructions and / or provide additional input for the adjustment and / or training of the imaging system, such as the height of the user or installer (131), a designated point of interest within the area to be monitored (e.g., room (109)), etc. Optionally, the user or installer may activate (193) the camera assembly (101) prior to mounting the camera assembly (101) at a mounting location. Additionally, the user or installer's connection (199) to the mobile device (117) or other device (e.g., a voice-based intelligent personal assistant) to receive instructions, provide input, and / or receive feedback is generally performed prior to any operation involving interaction with the user or installer. For example, an operation of connecting (199) the mobile device (117) or the voice-based intelligent personal assistant to the imaging system to receive the instructions using the mobile device (117) or the voice-based intelligent personal assistant may be performed prior to receiving (195) the instructions for the adjustment or training of the imaging system.

[0153] For example, the imaging system includes a camera assembly (101) having an enclosure (167) having at least two mounting surfaces (e.g., 162, 163, and / or 164) that are mutually orthogonal for alignment with at least two orthogonal surfaces (e.g., 171, 172, and / or 173) against which the camera assembly is mounted, at least one imaging device (e.g., thermal camera (175)) disposed within the enclosure (167) and having a predetermined orientation relative to the enclosure, and a communication device disposed within the enclosure (167). The imaging system further includes a server (113) disposed at a location remote from where the camera assembly (101) is mounted. The camera assembly (101) and the server (113) identify at least one installation measurement of the camera assembly through a computer communication network (115) to establish a mapping from an image coordinate system for images generated by the imaging device to a real coordinate system that is aligned with the orientation determined by the at least two orthogonal surfaces. The communication device may be a wireless communication device or a wired communication device.

[0154] For example, a user (e.g., installer or owner) of the camera assembly (101) may be instructed to mount the camera assembly (101) on a vertical edge (119) where two walls (171 and 172) meet, and the at least one installation measurement includes a mounting height (123) of the camera assembly (101) above a floor surface (127) on which the user (101) of the system is standing.

[0155] Preferably, but not necessarily, the imaging device is a thermal camera (175) that produces the image based on sensing infrared radiation. Preferably, the resolution of the thermal camera (175) is sufficiently low such that the identity of the person captured in the thermal image produced by the thermal camera (175) cannot be determined from the thermal image.

[0156] Optionally, a mobile application running within the mobile device (117) is configured to communicate with at least one of the camera assembly (101) and the server (113) in identifying the installation measurements, e.g., the mounting height (123) and / or the location of a point of interest in the image coordinate system (139), and to provide a user interface (e.g., as shown in Figures 3 to 6).

[0157] For example, the user interface is configured to receive an input identifying the height of the user (131) whose thermal image (133) is captured in an image generated by the thermal camera (175), and the mounting height (123) is calculated from the height of the thermal image (133) of the user within the image coordinate system (139) and the actual height of the user (131) received at the user interface.

[0158] The at least one installation measurement may include a location in the image coordinate system of a point of interest (e.g., a room corner, a door, or a window) in a scene, region, or space monitored by the imaging device (175). The point of interest is typically within an area of ​​the image produced by the thermal camera (175) that is not visible in the image produced by the thermal camera. The installation measurements can be used to construct a region layout (135) that defines the geometry of the monitored space.

[0159] Optionally, the user is instructed to navigate to a point of interest to mark the location of the point of interest within the image coordinate system with the location of the user's thermal image (133) at the point of interest, as shown in Figures 4-6.

[0160] In some embodiments, the imaging system includes a second camera assembly having a known mounting height (e.g., previously determined, automatically measured using a sensor, or identified by a user). In this case, the mounting height of the first camera assembly can be calculated based on the mounting height of the second camera assembly and a correlation of objects simultaneously captured in images generated by the first and second cameras. The mounting height of the first camera assembly can be adjusted such that the actual height of an object observed and calculated by the first camera matches the actual height of a corresponding object observed and calculated by the second camera.

[0161] Optionally, the camera assembly (101) includes a sensor that automatically measures a mounting height (123) between the camera assembly (101) and a floor surface (127).

[0162] For example, after mounting the camera assembly (101) on an edge (119) or corner (174) where two or three orthogonal surfaces (e.g., 171, 172, and / or 173) meet, the user can activate the camera assembly (101) to establish a communication connection with a remote server (113) and / or a mobile device (117). The server (113) and / or the mobile device (117) can instruct the user to move around the monitored area such that the full height of the user's thermal image (133) is detected in the image generated by the camera assembly (101). The user may be prompted to provide the user's height standing on a floor surface (127) so that the imaging system can calculate the mounting height (123) of the camera assembly (101) based on the user's (131) actual height and the measurement of the user's height in the image, and the user may be fully captured by the camera assembly (101). The user's height may be provided via a graphical user interface of the mobile device (117) or via the user's gestures detected by the camera assembly (101) in conjunction with audio prompts provided by the server (113).

[0163] Optionally, the user is instructed to move a heat detectable object (e.g., a glass of hot or cold water or the user's body) to the point of interest in the area (e.g., room (109)) monitored by the camera assembly (101) to enable the imaging system to register the point of interest in the images generated by the camera assembly (101) according to the location of a thermal image (133) of the object located at or near the point of interest in the monitored area of ​​the imaging system. For example, the camera assembly (101) images based on sensing infrared radiation, and the point of interest is not visible in the images generated by the camera assembly and therefore cannot be directly determined from an analysis of the images generated by the camera assembly at the time of installation.

[0164] Optionally, the imaging system identifies the locations of several points of interest from machine learning of objects identified from the thermal images over a period of time, where temperature changes in certain areas of the monitored area and / or human activity (and / or thermal activity) in the monitored area provide indications of the locations of the points of interest. In such an embodiment, it is not necessary to provide a user interface for the adjustment, calculation and / or identification of configuration parameters such as the mounting height, the location of the points of interest, etc. For example, the user may install the thermal camera assembly (101) and then walk away, and the imaging system captures the height of a reference object and then statistically estimates the possible height range. Such interaction with the imaging system is optional, and the system performs the adjustment in the background based on statistical analysis and / or "machine learning" of object recognition resulting from a large number of camera assemblies installed in various positions and settings. The statistical results of the object and / or environment as viewed by the camera assembly and / or look-up table can be used to train the imaging system to automatically calculate the mounting height and points of interest as they are recognized from images recorded by the camera assembly.

[0165] In some cases, user input is provided to the imaging system by correlation of a known background (e.g., a user is instructed to go to a location of interest) with the thermal image of the object (e.g., the location of the thermal image (133) of the user (131)) observed by the thermal camera assembly (101). User input can also be provided via the mobile device (117) by pressing a button on a user interface, by providing a voice command to a user interface implemented on the mobile device (117), or by providing a gestural input with the mobile device (117). These inputs can also be performed via thermal gestures detectable by the thermal camera assembly (101), such as by moving the object and then holding it still for at least a predetermined period of time.

[0166] Examples of the points of interest include corners of the room in which the camera assembly is located, doors of the room, windows of the room, furniture disposed in the room, hallways within the room, and activity areas within the room.

[0167] After the thermal imaging system has been calibrated or configured with a set of configuration parameters that map the image coordinate system of the image produced by the camera assembly (101) to the real-world coordinate system of the area monitored by the camera assembly (101), the thermal imaging system is able to provide useful services.

[0168] For example, the thermal imaging system identifies a size and orientation of an object visible in the image produced by the camera based on the size and orientation of the object measured in the image produced by the camera and the set of configuration parameters.

[0169] For example, if the set of configuration parameters further identifies a point of interest within the region in the image coordinate system (e.g., a point of interest having a location in an image generated by the camera assembly but not visible in such image), the thermal imaging system generates a monitoring alert that is provided via an output device of the camera assembly referencing the point of interest within the region.

[0170] Optionally, the thermal imaging system can improve the set of configuration parameters by statistical analysis and / or machine learning. For example, the accuracy of the mounting height (123) of the camera assembly (101) above a floor surface (127) of the monitored area can be improved based on matching a statistical distribution of heights of thermal images of people observed by the thermal camera assembly (101) over a period of time with a known distribution.

[0171] Each of the mobile device (117), the server system (113) and the thermal camera assembly (101) may be embodied, at least in part, in the form of one or more data processing systems shown in FIG. 20, having several components.

[0172] The present disclosure includes the above-described method, a computing device configured to perform the method, and a computer storage medium storing instructions that, when executed on the computing device, cause the computing device to perform the method.

[0173] FIG. 20 illustrates a data processing system that may be used to implement some components of embodiments of the present application. Although FIG. 20 illustrates various components of a computer system, it is not intended to represent any particular architecture or manner of interconnecting those components. Other systems having fewer or more components than those illustrated in FIG. 20 may also be utilized.

[0174] In Figure 20, the data processing system (200) includes an interconnect (201) (e.g., a bus and system core logic) that interconnects a microprocessor (203) and a memory device (211). The microprocessor (203), in the embodiment of Figure 20, is coupled to a cache memory (209).

[0175] In Figure 20, the interconnect (201) interconnects the microprocessor (203) and the storage device (211), and also interconnects with input / output (I / O) devices (205) via an I / O controller (207). The I / O devices (205) may include display devices and / or peripherals, such as a mouse, keyboard, modem, network interface, printer, scanner, video camera, and other devices known in the art. If the data processing system is a server system, some of the I / O devices (205), such as a printer, scanner, mouse, and / or keyboard, are optional.

[0176] The interconnect (201) includes one or more buses connected to each other through various bridges, controllers and / or adapters. For example, the I / O controller (207) may include a Universal Serial Bus (USB) adapter to control USB peripherals and / or an IEEE-1394 bus adapter to control IEEE-1394 peripherals.

[0177] The storage device (211) includes one or more of ROM (read only memory), volatile RAM (random access memory), and non-volatile memory such as a hard drive, flash memory, or the like.

[0178] Volatile RAM is typically implemented as dynamic RAM (DRAM), which requires continuous power to refresh or maintain the data in the memory. Non-volatile memory is typically a magnetic hard drive, a magnetic optical drive, an optical drive (e.g., DVD RAM), or other type of memory system that maintains data even after power is removed from the system. The non-volatile memory may be a random access memory.

[0179] The non-volatile memory may be a local device directly coupled to the rest of the components in the data processing system. Non-volatile memory remote from the system may also be utilized, for example a network storage device coupled to the data processing system via a network interface such as a modem or an Ethernet interface.

[0180] In this description, some functions and operations are described as being performed or caused by software code for ease of description, but such terms are also used to specify that the functions result from execution of the codes / instructions by a processor, e.g., a microprocessor.

[0181] Alternatively, or in conjunction, the functions and operations described herein may be implemented using special purpose circuitry, for example, an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA), with or without software instructions. Embodiments may be implemented using hardwired circuitry without software instructions, or in combination with software instructions. Thus, the technology is not limited to any specific combination of hardware circuitry and software, or to any particular source for the instructions executed by the data processing system.

[0182] Although one embodiment can be implemented in a fully functional computer and computer system, various embodiments can be deployed in various forms of computer products and can apply without regard to the particular type of machine or computer-readable medium used to actually implement the deployment.

[0183] At least some aspects disclosed may be embodied, at least in part, in software, i.e., the techniques may be executed in a computer system or other data processing system in response to a processor, such as a microprocessor, executing sequences of instructions contained in a storage device, such as a ROM, volatile RAM, non-volatile memory, cache, or a remote storage device.

[0184] The routines executed to implement the embodiments may be implemented as part of an operating system or a specific application, component, program, object, module, or sequence of instructions referred to as a "computer program." The computer program typically resides at various times in various memory devices and storage devices within a computer, and includes one or more instructions that, when read and executed by one or more processors within a computer, cause the computer to perform operations necessary to implement elements including various aspects.

[0185] A machine-readable medium can be used to store software and data that, when executed by a data processing system, causes the system to perform various methods. The executable software and data can be stored in various locations, including, for example, ROM, volatile RAM, non-volatile memory, and / or cache. Portions of the software and / or data may be stored in any one of these storage devices. Furthermore, the data and instructions can be obtained from a centralized server or a peer-to-peer network. Different portions of the data and instructions can be obtained from different centralized servers and / or peer-to-peer networks at different times and in different communication sessions, or in the same communication session. The data and instructions can be obtained completely prior to execution of the application. Alternatively, portions of the data and instructions can be obtained dynamically in time for execution. Thus, it is not necessary for the data and instructions to be completely on the machine-readable medium at a particular instance of time.

[0186] Examples of computer readable media include, but are not limited to, recordable and non-recordable types of media, such as volatile and non-volatile memory devices, read only memory (ROM), random access memory (RAM), flash memory devices, floppy and other removable disks, magnetic disk storage media, optical storage media (e.g., compact disk read only memory (CD ROM), digital versatile disk (DVD), etc.), among others, on which the instructions can be stored.

[0187] The instructions may also be embodied in digital and analog communication links for electrical, optical, acoustic or other forms of propagated signals, such as carrier waves, infrared signals, digital signals, etc. However, propagated signals, such as carrier waves, infrared signals, digital signals, etc., are intangible machine-readable media and are not configured to store instructions.

[0188] Generally, a machine-readable medium includes any mechanism that generates (i.e., stores and / or transmits) information in a form accessible by a machine (e.g., a computer, a network device, a personal digital assistant, a manufacturing tool, any device with a set of one or more processors, etc.).

[0189] In various embodiments, hardwired circuitry may be used in combination with software instructions to implement the techniques, and thus, the techniques are not limited to any specific combination of hardware circuitry and software, or to any particular source for the instructions executed by the data processing system.

[0190] (Other aspects) The above description and drawings are illustrative and should not be construed as limiting. The present disclosure is an illustration of the features of the invention to enable those skilled in the art to implement and use the technology. The various features described herein should be used in compliance with all current and future rules, laws and regulations related to privacy, security, authorization, consent, consent, and others. Many specific details are described to provide a thorough understanding. However, in some cases, well-known or conventional details are not described to avoid obscuring the description. References to one or an embodiment in the present disclosure are not necessarily to the same embodiment, and such references are intended to mean at least one.

[0191] The use of section headings herein is provided for ease of reference only and should not be construed in any way as limiting this disclosure and the claims that follow.

[0192] References to "one embodiment" or "an embodiment" mean that the specific features, structures, or characteristics described in connection with that embodiment are included in at least one embodiment of the present disclosure. The appearances of the phrase "in one embodiment" in various places in this specification are not necessarily all references to the same embodiment, and not all of them need to be references to independent or alternative embodiments that exclude other embodiments from each other. Further, various features are described that may be presented by one embodiment rather than by other embodiments. Similarly, various requirements are described that may be required by one embodiment rather than by other embodiments. Any combination of the various features described in this description is included herein, unless excluded by express recitation and / or obvious contradiction. For example, the features described above in connection with "in one embodiment" or "in some embodiments" can, if necessary, be included in one embodiment, except when the dependence of the said features on other features specific to the embodiment is such as to limit the option of excluding the features selected from the embodiment, and when the contradiction between the specific features and other features obvious from the description is such as to limit the option of including the features selected together in the embodiment.

[0193] In the foregoing specification, the present disclosure has been described in relation to specific and exemplary embodiments thereof. It will be apparent that various modifications can be made thereto without departing from the broad spirit and scope set forth in the following claims. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a limiting sense.

Claims

1. an enclosure into which the camera assembly is press-fit; at least one imaging device disposed within the enclosure and having a predetermined orientation relative to the enclosure, the at least one imaging device being a thermal camera that generates images based on sensing infrared radiation; a camera assembly including a communication device disposed within the enclosure; a server located at a location remote from where the camera assembly is mounted, the camera assembly and the server communicating over a computer communications network to identify at least one installation measurement of the camera assembly and to establish a mapping from an image coordinate system for images generated by the at least one imaging device to a real-world coordinate system; the at least one installation measurement includes a mounting height of the camera assembly above a floor surface on which a user of the imaging system stands; a mounting height of the camera assembly above a floor surface on which a user of the imaging system stands is calculated based on a preset ratio and a preset reference mounting height of a thermal camera assembly, the reference mounting height being from a mounting position of the thermal camera assembly to a predefined reference floor associated with an image generated by the at least one imaging device; an imaging system, wherein the preset ratio is the ratio between the size of an image of an item in an image coordinate system, the image being generated by the thermal camera assembly, and the corresponding height of a real-world item.

2. The imaging system of claim 1 , wherein the user of the camera assembly is instructed to mount the camera assembly on an edge where two walls meet.

3. The imaging system of claim 1 , wherein the communication device is a wireless communication device.

4. 4. The imaging system of claim 3, further comprising a mobile device that operates a mobile application configured to communicate with at least one of the camera assembly and the server and provide a user interface upon identifying the at least one installation measurement.

5. The imaging system of claim 4 , wherein the user interface is configured to receive an input identifying a height of the user whose thermal image is captured in an image produced by the thermal camera.

6. 2. The imaging system of claim 1, wherein the at least one installation measurement includes a location in the image coordinate system of a point of interest in a scene monitored by the at least one imaging device, the point of interest being within a range of an image produced by the thermal camera but not visible in the image produced by the thermal camera.

7. 7. The imaging system of claim 6, wherein the user is instructed to move to a point of interest to indicate the location of the point of interest within the image coordinate system using the user's thermal image location at the point of interest.

8. the camera assembly is a first camera assembly and the imaging system comprises:

3. The imaging system of claim 2, further comprising a second camera assembly having a mounting height, the mounting height of the first camera assembly being calculated based on the mounting height of the second camera assembly and a correlation of objects simultaneously captured in images produced by the first camera assembly and the second camera assembly.

9. The imaging system of claim 1 , wherein the camera assembly further comprises a sensor for measuring a mounting height between the camera assembly and a floor surface.

10. 1. A method of adjusting an imaging system, comprising: activating the camera assembly to establish a communication connection with a remote server; providing a height of a user standing on a floor and completely captured in an image generated by said camera assembly; calculating a mounting height of the camera assembly based on a preset ratio and a preset reference mounting height of a thermal camera assembly, the reference mounting height being from a mounting position of the thermal camera assembly to a predefined reference floor associated with an image generated by the camera assembly, the preset ratio being a ratio between a height of the user and a measurement of the height of the user in the image; The method further includes moving the object to a point of interest within the area monitored by the camera assembly to enable the imaging system to register the point of interest in an image generated by the camera assembly using a position of an image of the object detectable by the camera assembly that is located at or near the point of interest within the area monitored by the camera assembly.

11. The method of claim 10 , wherein the camera assembly images based on sensing infrared radiation, and the point of interest is not visible in the image produced by the camera assembly.

12. 12. The method of claim 11, further comprising providing input to the imaging system while the object is at the location of interest to enable the imaging system to correlate the location of the image of the object in an image coordinate system with the location of interest.

13. The input is: Pressing a button within a user interface; leaving the object standing at the point of interest for at least a predetermined period of time after moving the object; Providing voice commands to a user interface implemented in the mobile device; 13. The method of claim 12, comprising one of providing a gesture input using the mobile device.

14. The point of interest is a corner of a room in which the camera assembly is located; The door to the room; The window of the room, Furniture arranged in the room; A passageway within the room; The method of claim 13, wherein the area is one of the activity areas within the room.

Images