DETERMINATION OF THE CONDITION OF A WINDOW OR DOOR IN A HOUSEHOLD

Description

[0001] The invention relates to the monitoring of a household. In particular, the invention relates to the determination of the condition of a window or door in the household.

[0002] A floor cleaning robot is designed to clean floor areas in a household. The robot can navigate autonomously within the home and, as part of its cleaning routine, cover virtually the entire floor.

[0003] It has been suggested that the floor robot could be used for household monitoring, for example to replace or enhance an alarm system. For instance, the floor robot could include a pyrotechnic sensor or a smoke detector to warn of an impending fire, a LiDAR motion sensor, or a passive infrared sensor to detect a person in the household.

[0004] US 2014 / 0207280 A1 or CN 113 341 752 A proposes a room monitoring system. The system includes a mobile robot that can be used to detect an open door or window.

[0005] The floor robot is typically limited to operating close to the ground. Detecting an open window or door can be complicated by perspective, an object in its field of view such as furniture, a pet, or a person. The window may be partially or completely obscured by a curtain, and objects like potted plants or books may be on the windowsill. The window or door may be bordered by a wall, which may have an object such as wallpaper, a picture, or a poster attached to it. An object may also be attached to the window or door itself; for example, a translucent picture or a thermometer on a window, or a sticker or label on a door.

[0006] It can also be difficult to correctly identify a window or door in a household. For example, a door might include a recessed window, or a window might open like a door. A serving hatch cannot be considered a door or window, even if it can be opened and closed and objects can pass through it. A transparent skylight above a door cannot be opened and therefore does not need to be monitored, whereas a skylight in a flat roof can be considered a window that requires monitoring. A sliding door may not be recognizable as such when open. A window pane may be covered, making it difficult to identify a window.

[0007] One of the problems underlying the present invention is to provide an improved technique for determining the state of an access element in a household. The invention solves this problem by means of the subject matter of the independent claims. Dependent claims describe preferred embodiments.

[0008] An individual access element to a room in a household can be set to an open and a closed state. A method for detecting an individual state of the access element comprises the steps of determining a position where an individual access element is located in the household; non-contact scanning of the individual access element while it is in a predetermined state; storing information about the individual access element based on its scan and state; capturing another non-contact scan of the individual access element in the household; and detecting a state of the individual access element based on the further non-contact scan, the stored information, and the predetermined information.

[0009] It has been recognized that a common technique for detecting a generic access element or determining its state can be significantly improved in its detection performance by additional training on samples of an individual access element in a predetermined household. Much of the preliminary work required to detect a generic access element or its state can be performed centrally using collected or generated data. The additional acquisition of information about the individual access element can require minimal effort and can also be at least partially automated.

[0010] This allows a well-known and established technique for object or feature recognition to be used advantageously without having to intervene in an associated preparation phase. If the information for recognizing a generic access element or its state needs to be updated, certain information relating to the individual access element can also be retained.

[0011] The access element preferably comprises a door or a window. More generally, the access element can comprise any structural element capable of closing an opening through which a person can enter the household or a room within the household. The access element can also include, for example, a sliding door, a double door, a gate, a folding door, a trapdoor, a revolving door, a hinged door, a casement window, a tilt window, or a dormer window. The window can be rectangular or have a different shape, such as round, triangular, or oval.

[0012] It should be noted that an access element can be set to multiple states, each representing a different degree of obstruction to an intruder. For example, the access element can be fully open, partially open, ajar, tilted, closed, latched, or locked. Some access elements may also assume states other than these. The point at which a state is considered closed or open can be defined individually for each access element. This definition can apply to all access elements in a room or household, or only to a single access element. In one embodiment, an individual access element can be considered closed if it cannot be opened from the outside by a person without tools or the use of force.Another example of an embodiment would be a tilted window through which no one can enter, but through which moisture could penetrate during heavy rain.

[0013] In a preferred embodiment, the position of the access element is specified by a user or a person associated with the household. Optionally, in addition to the specific state, the degree of certainty with which the state was detected can also be specified. An example message could read: "The kitchen window is fully open (detected with 95% certainty)." The positions of multiple access elements can also be specified, which can then be processed accordingly. The selection of which access elements are located can be made implicitly by the user, thus eliminating the need for automatic differentiation.

[0014] The access point can be scanned by an automated floor robot. This robot is typically designed to move across a floor surface in the home and, optionally, to clean it either dry or wet. The robot can be controlled autonomously. Alternatively, it can be guided through the home to assume a predetermined position or follow a predetermined path, as described in this document.

[0015] Further scanning can be performed during a cleaning of a floor area in the home. In addition to its primary task of floor cleaning, the floor robot can thus be advantageously used to monitor the condition of the access point. Separately, a request or prompt to scan can be detected, and the scan can be performed in response. For example, a request can be triggered by an event, time, or user, and the floor robot can react accordingly. An ongoing floor cleaning process can be interrupted for a scan to provide rapid feedback on the condition of the access point.

[0016] In another embodiment, the specific state of an individual access element within the household can be made available to a person outside the household. Of course, the states of multiple access elements can also be determined and made accessible to the person. This allows the person to easily decide whether they have left the household in a sufficiently secure condition. Several specific states can also be automatically summarized, so that the person can be informed, for example, that either all or not all access elements of the room or the household are closed.

[0017] Based on the position of the individual access element, a scanning position can be determined from which the scanning takes place. Optionally, an advantageous orientation of the floor robot relative to the individual access element can also be determined. In a preferred embodiment, the position of the access element is determined relative to an environmental map of the room or household. An obstacle or room boundary can be taken into account so that only a suitable and preferably informative scanning position can be determined.

[0018] Regarding the position, multiple scanning positions can be defined, each offering a different perspective on the access element, thus enabling comprehensive scanning. The scanning positions can also exhibit varying lighting conditions, such as backlighting, daylight, or artificial light. These lighting conditions can be time-dependent, and a scanning position can also be selected based on the time. This allows for improved and comprehensive learning of how to determine the state of the individual access element.

[0019] Information about an individual access element can be stored separately from information about the recognition of a generic access element or its state. This information can be in different formats. For example, information about the recognition of a generic access element or its state can be preprocessed and stored compactly. Information about an individual access element, on the other hand, can be stored in a less processed format. In one embodiment, raw data or unprocessed but compressed data can be stored, making subsequent processing or verification easier.

[0020] In one embodiment, a link is established between information about the individual access element and information about generic access elements. For example, it can be determined where a hand lever is located on the individual access element, and the hand lever can be linked to the generic information in such a way that its position can be detected more accurately. In another embodiment, the position of the hand lever on the individual access element can be determined and, based on the link, compared with the positions of generic access elements in order to derive a state from the position.

[0021] Information associated with different individual access elements in the household can be stored separately. In one embodiment, each set of information associated with an individual access element is assigned its own storage area, database, or model.

[0022] The information for recognizing a generic access element or its state can be stored as a trained artificial neural network (ANN). The stored information for the individual access element can also be stored as an ANN, or alternatively as raw data, an image, a digest, or as a link to information about a generic access element.

[0023] It should be noted that the information stored for the recognition of generic objects need not be limited to access elements. In one embodiment, such information can also be configured to recognize other objects found in a household, for example, a piece of furniture, a pet, or an obstacle lying on the floor, such as a cable or a rug.

[0024] The floor robot can be controlled to close an individual access element after it has been determined that it is not currently closed. For example, the floor robot can approach an open door protruding into the room at a specific angle and push it towards the door frame until it snaps shut. This is particularly useful for an individual access element that extends close to the floor surface on which the floor robot is traveling. Information can be assigned to an individual access element indicating whether closing is desired (or possible). For instance, a connecting door between two rooms in the house can be left open, while a balcony door can be closed if possible.Closing the access element not only reduces the risk of unwanted entry by a person, but also the ingress of unwanted external influences, such as rain.

[0025] According to a further aspect of the present invention, a floor robot comprises a device for determining the position of an individual access element within the household; wherein the individual access element can be brought into an open and a closed state; a scanning device for non-contact scanning of the individual access element while it is in a predetermined state; a first memory containing predetermined information for recognizing a generic access element or its state; a second memory for storing information about the individual access element based on its scanning and its state; and a processing device configured to recognize a state of the individual access element within the household based on a further non-contact scanning and the predetermined and stored information.

[0026] The processing equipment may be configured to execute all or part of a method described herein. For this purpose, the processing equipment may be electronic and, for example, include a programmable microcomputer or microcontroller, and the method may be in the form of a computer program product with program code. The computer program product may also be stored on a computer-readable data carrier. Features or advantages of the method may be transferred to the equipment or vice versa.

[0027] The floor robot can be configured to clean a floor area in the home and to further scan the individual access point during this cleaning process. This cleaning can include, in particular, cleaning the floor surface. The cleaning can be dry, such as vacuuming or sweeping, or wet, using a cleaning fluid, such as mopping or damp squeegeeing.

[0028] According to yet another aspect of the present invention, a system comprises a floor robot described herein and an interaction device for a user, wirelessly coupled to the processing unit. The interaction device can, in particular, comprise a device assigned to the user, such as a smartphone, a laptop computer, or a tablet computer. Other forms of interaction, for example, by means of voice control, are also possible. The interaction device enables cooperative or interactive scanning of an individual access element. A user can ensure a predetermined state of the access element or specify its position. The interaction device is preferably capable of graphically outputting information, so that information regarding a graphical representation of the household can be provided.The graphical representation can be determined based on map data that the ground robot has collected or been provided with.

[0029] The invention will now be described in more detail with reference to the accompanying figures, in which: Figure 1 shows a floor robot in a household; Figure 2 shows a flowchart of a process; Figure 3 shows exemplary scans of an individual door in a household; and Figure 4 shows exemplary scan positions in a household and scans of an individual door in the household from different scan positions.

[0030] Figure 1Figure 1 shows a system 100 with a floor robot 105 in a household 110. The household 110 typically comprises at least one room, which can be separated by an access element 115. The access element 115 typically comprises a door or a window and can be set to various states that allow or prevent passage. The household 110 contains a floor surface 118 on which the floor robot 105 can move. Preferably, the floor robot 105 is configured to work on the floor surface 118 and, in particular, to clean it. Elements of the floor robot 105 for locomotion and for working on the floor surface 118 are shown in Figure 1. Figure 1 not shown.

[0031] The floor robot 105 preferably comprises a processing unit 120 and a sensor 125 for non-contact and preferably optical scanning of its environment. The sensor 125 can, for example, be a camera, a radar sensor, or a LiDAR sensor. Furthermore, a first memory 130 containing information for recognizing a generic access element 115 or its state, and more preferably a second memory 135 for storing information about an individual access element 115 present in the household 110, can be provided. Multiple second memories 135 or memory areas can also be provided, which are logically separated from one another so that the stored information does not interact with each other in an uncontrolled manner. Each second memory 135 can be assigned to its own individual access element 115 in the household 110.

[0032] Furthermore, the floor robot 105 preferably includes a map storage device 140 for storing map data about the household 110. Based on a scan of its surroundings, the floor robot 105 can detect a boundary of the household 110, for example, a wall, or an object, for example, a piece of furniture. Based on this, the map data can be created or updated. A route to be traveled within the household 110 can be planned using the map data.

[0033] A wireless interface 145 can be provided for communication with an interaction device 150. In addition to the floor robot 105, the system 100 can include an interaction device 150.

[0034] A generic object in household 110 can be detected by scanning it with sensor 125 using information from the first memory 130. It is proposed to additionally train the system to detect an individual access element 115 found in household 110 and to store information about the individual access element 115 in the second memory 135. Detection of the individual access element 115 and / or its state can then be performed based on the information stored in the second memory 135. Preferably, information from the first memory 130 is also used for this purpose.

[0035] Figure 2Figure 1 shows a flowchart of a procedure 200, which can be executed in particular by means of a system 100. The procedure 200 comprises two phases, which can also be considered independent procedures. In the following, a window is discussed as an example access element 115; another access element can be treated accordingly.

[0036] A first phase begins in step 205, in which the position of an individual window 115 is specified. This specification can be made by a user, for example, using the interaction device 150. Preferably, the user is provided with a graphical view of map data for the household 110 and can specify a position on the map data for the individual window 115.

[0037] In step 210, the individual window 115 can be set to a predetermined state. This state can, in particular, be a closed state, in which the passage of a person or object through the window 115 into the household 110 is prevented. The person can be asked to set the predetermined state on the window 115.

[0038] In step 215, one or more initial scanning positions can be determined in the area of ​​the individual window 115. These scanning positions can offer advantageous and diverse perspectives of the individual window 115 and can be reached by the floor robot 105 on the floor surface 118. Here, not only the precise scanning position of the robot is relevant, but also the orientation of the fixed camera towards the access element and the time of day, in order to ensure comparable lighting conditions in the future.

[0039] In step 220, the floor robot can move to a scanning position and perform a scan of the individual window 115. If there are multiple scanning positions, these can be approached sequentially, and multiple scans can be performed. The scans can be stored in the second memory 135, either unprocessed or processed. A note about the predetermined state and / or position can be added to each scan.

[0040] Optionally, in step 225, the individual window 115 can be moved into a second predetermined state, for example, open, tilted, or ajar. In step 230, one or more second scanning positions can be defined. Optionally, the first scanning positions can also be reused. In step 235, a second scanning position can be approached and the individual window 115 scanned. Steps 230 and 235 can correspond to steps 215 and 220, respectively.

[0041] Optionally, steps 225 to 235 can be performed regarding another state of the individual window. This process can be repeated for each state that can be detected.

[0042] Step 240 determines whether a further scan of the individual window 115 is required. This may be the case if the stored information, optionally supplemented by generic information in the first memory 130, is insufficient to recognize the individual window or its state based on a further scan. In this case, the user may be asked in step 245 to ensure predetermined conditions, such as specific lighting conditions or the absence of furniture or wall decorations in the area of ​​the individual window 115. A further scan can then be performed. A scanning position for this can be chosen beforehand. Steps 240 and 245 can also be repeated multiple times if necessary, for example, at different times of day under varying lighting conditions.

[0043] If sufficient information is stored in the second memory 135 to allow recognition of the individual window 115 or its state, the first phase of the procedure 200 can be completed.

[0044] A second phase begins in step 250, in which a request to check the status of an individual window 115 can be received. The request can be explicit and, for example, triggered by a user who wants to know the status of one or more windows or doors 115 of the household 110. The request can also be time-controlled, for example. In yet another embodiment, the request is implicit, for example, when the floor robot 105 performs a cleaning task in the household 110 and is prompted to determine the status of the window 115.

[0045] In step 255, the floor robot 115 can move to a scanning position in the area of ​​the individual window 115. The scanning position can be precisely planned, or a scanning position can be used that is located within a predetermined area around the window's location, for example, at a predetermined maximum distance. Then, the window 115 can be scanned.

[0046] In step 260, the scan can be evaluated based on previously stored information to determine the state of window 115. For this purpose, window 115 can first be detected in the scan. It may be necessary to examine a predetermined detail of window 115 more closely, for example, a vertical edge or the position of an operating lever. Information contained in the scan can be compared with stored information to determine the state. If a result is obtained, it can be determined whether the result could be determined with a sufficiently high degree of certainty or whether there is another possible result, such that no unambiguous statement about the state can be made. In this case, the procedure 200 can branch back to step 255 and perform a new scan, possibly from a different scanning position.

[0047] Otherwise, the result can be provided in step 265, for example, in the form of a message to a person. The result can also be provided in another way, for example, in machine-readable form, so that it can be further processed by another system. Optionally, in step 270, an attempt can be made to close a window 115 that has been determined to be open using the floor robot 105. To do this, the floor robot 105 can carefully move to the open window and then follow a course that pushes the window 115 into a closed position.

[0048] Figure 3Figure 1 shows exemplary scans of an individual door 115 in a household 110. It should be understood that the individual door 115 represents any individual access element 115 in a household 110. The scans were created based on optical recordings made by a floor robot 105, which assumed a scanning position on the floor surface 118 in the area of ​​the individual door 115.

[0049] Initial scans 305, 310, and 315 concern the individual door 115 in a closed state from various scanning positions. Gaps around the door leaf opposite a door frame are closed and of constant width. An operating lever is in a substantially vertical position. No gap is visible at any point on the individual door 115.

[0050] Second scans 320, 325, and 330 concern the individual door 115 in an open position. The operating lever is in a horizontal position, so it can be seen that the individual door 115 is not locked. In scan 325, a gap is visible at one upper edge of the door leaf towards the frame. In scan 330, it is even possible to see through to the left of the door leaf.

[0051] Third scans 335 and 340 concern the individual door 115 in a partially closed state, in which the door leaf is tilted inwards vertically within the frame. Whether this state is interpreted as closed or not may be predetermined. The operating lever is in the vertical position, so the individual door 115 is locked in its current state. In scan 335, the tilted state is barely perceptible. In scan 340, it can be seen in an upper right area of ​​the door leaf that the door is tilted relative to the door frame.

[0052] In summary, it can be stated that the determination of the state of the individual door 115 may include the detection of the individual door 115 on a scan 305-340, the identification of a predetermined feature of the individual door 115, the recognition of the feature and / or a logical linking with another recognized feature.

[0053] Figure 4 shows exemplary scanning positions 405 in a household 110 and scanning of an individual door 115 in the household 110 from different scanning positions 405.

[0054] Regarding a cartographic representation of a section of house 110, a position 410 of the individual door 115 was defined, for example, by a user of system 100. Naturally, the individual door 115 closes off a room of house 110. On one side of the individual door 115, where the floor robot 105 is located, various scanning positions 405 are defined, offering different perspectives of the individual door 115 and accessible by the floor robot 105. From these scanning positions 405, the individual door 115 can be advantageously scanned.

[0055] Fourth scans 415-430 each relate to the individual door 115, either in its full height or in a lower section down to the floor surface 118. Scans 415-430 are purely exemplary and are not assigned to specific scanning positions 405 shown in the upper section. It can be seen how different scanning positions 405 have a strong influence on the size ratios and distortion of the scanned individual door 115. Furthermore, lighting conditions during scanning can have an influence. For good recognition of the individual door 115, a feature thereof, or a state of the individual door 115, it may be useful to choose a scanning position 405 that allows for a clear representation and is preferably as similar as possible to a scanning position 405 that was used to learn the individual door 115.However, not every desirable scanning position 405 can be assumed, for example because a scanning position 405 or a view from the scanning position 405 to the individual door 115 is obstructed by a piece of furniture. Reference sign

[0056] 100 System 105 Floor robot 110 Household 115 Access element, in particular window or door 118 Floor surface 120 Processing unit 125 Sensor 130 First memory 135 Second memory 140 Map memory 145 Wireless interface 150 Interaction unit 200 Procedure 205 Specify window position 210 Window in first position 215 Determine first scanning positions 220 Move to first scanning positions, perform scans 225 Window in second position 230 Determine second scanning positions 235 Move to second scanning positions, perform scans 240 Further scanning required? 245 Request support, perform scanning 250 Request to check window received 255 Move to scanning position and perform scan 260 Unique result? 265 Provide message 270 Close window 305-315 first scans 320-330 second scans 335, 340 third scans 405 Scanning positions 410 Position of an individual access element 415-430 Fourth scans

Claims

1. Method (200) for identifying a state of an individual access element (115) to a room in a household (110), wherein the individual access element (115) can be brought into an open and into a closed state; wherein the method (200) comprises the following steps: - determining (205) a position (410) at which an individual access element (115) is located in the household (110), - contactless scanning (220, 235, 245) of the individual access element (115) while it is in a predetermined state; - storing (220, 235, 245) information relating to the individual access element (115) on the basis of its scanning and its state; characterised by the following steps: - detecting (255) a further contactless scanning of the individual access element (115) in the household (110); and - identifying (260) a state of the individual access element (115) on the basis of the further contactless scanning and the stored and predetermined information.

2. Method (200) according to claim 1, wherein the access element (115) comprises a door or a window.

3. Method (200) according to claim 1 or 2, wherein the access element (115) is scanned by means of an automatic floor robot.

4. Method (200) according to claim 3, wherein the further scanning is detected (255) in the context of cleaning a floor surface in the household (110).

5. Method (200) according to one of the preceding claims, wherein a determined state of an individual access element (115) in the household (110) is provided to a person outside of the household (110).

6. Method (200) according to one of the preceding claims, wherein a scanning position (405), from which the scanning is taking place, is determined on the basis of the position (410) of the individual access element (115).

7. Method (200) according to claim 6, wherein the scanning position (405) is determined on the basis of a map of the surroundings of the household (110).

8. Method (200) according to one of the preceding claims, wherein the information relating to the individual access element (115) is stored separately from the information relating to the identification of a generic access element (115) or its state.

9. Method (200) according to one of the preceding claims, wherein information which is assigned to different individual access elements (115) in the household (110) is stored separately from one another.

10. Method (200) according to one of the preceding claims, wherein the information relating to the identification of a generic access element (115) or its state is present as a trained neural network.

11. Method (200) according to one of the preceding claims, wherein the floor robot (105) is activated in order to bring the individual access element (115) into the closed state (270) once it has been determined that its state is not closed.

12. Floor robot (105), comprising - a facility (150) for determining a position (410) at which an individual access element (115) is located in the household (110); - wherein the individual access element (115) can be brought into an open and into a closed state; - a scanning facility (125) for contactless scanning of the individual access element (115) while it is in a predetermined state; - a first memory (130) with predetermined information relating to the identification of a generic access element (115) or its state; characterised by: - a second memory (135) for storing information relating to the individual access element (115) on the basis of its scanning and its state; and - a processing facilty (120), which is designed to identify a state of the individual access element (115) in the household (110) on the basis of a further contactless scanning and the predetermined and the stored information.

13. Floor robot (105) according to claim 12, wherein the floor robot (105) is designed to process a floor surface (118) in the household (110), and to detect the further scanning of the individual access element (115) in the context of processing the floor surface.

14. System (100), comprising a floor robot (105) according to claims 12 or 13 and an interaction facility (150), coupled wirelessly to the processing facility (120), for a user.