Local recommendation system for healthy smart home

Abstract

The present subject matter relates to a method and apparatus for identifying a hazardous location within a smart home environment. The method includes detecting one or more entities in an interior space. Thereafter, the method includes obtaining information associated with at least one external activity performed by the one or more entities. Thereafter, the method includes monitoring at least one subsequent activity of the one or more entities in the interior space. Thereafter, the method includes identifying at least one potential hazard location by associating at least one subsequent activity with the obtained information.

Description

Technical Field

[0001] The present invention relates to dangerous location detection in a smart home environment. In particular, the present invention relates to a method and apparatus for identifying dangerous locations in a smart home environment. Background Art

[0002] The hazard detection system is used to detect possible hazards using multiple sensors present in the smart home environment. The hazard detection system continuously tracks multiple hazardous conditions and locations within the smart home environment. The hazard detection system can generate notifications to users of the smart home environment based on the detection of multiple hazardous conditions and locations within the smart home environment.

[0003] Figure 1 and Figure 2 An example scenario for a hazard detection application in a smart home environment according to an existing solution is shown. Consider a smart home environment, wherein a user comes in from an outside market and enters the smart home environment with a bag. After entering, the user places the bag and wallet at different places in the smart home environment. A monitoring system of the smart home environment keeps tracking multiple hazardous situations that may occur in the smart home environment. Since the user enters the smart home from outside, placing the bag and wallet may contaminate the place where it is stored. The monitoring system helps to locate the contaminated location and inform the user of the contaminated location.

[0004] In another scenario, another user sneezes within 30 minutes of entering the smart home environment from outside and may contaminate the bed. In yet another scenario, a pet enters the smart home environment and jumps on the sofa. The pet may contaminate the sofa. In another scenario, the user comes in from outside and enters the smart home environment after disposing of garbage. The user may contaminate an area of ​​the smart home environment. However, the user does not know why and how external exposure (such as going to the market or shopping mall, meeting people, bringing external objects into the smart home environment, visiting external locations, disposing of garbage, and sneezing) increases the possibility that contamination is easily spread to other residents of the smart home environment.

[0005] In another scenario, the user places the bag of vegetables on the table after entering the smart home environment and walks to the bathroom to clean himself. However, due to the user's movement, other areas of the home environment may be contaminated. In this scenario, conventional solutions cannot provide solutions for providing the most appropriate cleaning solution to reduce the contamination caused by the user's movement. In addition, the user may sometimes be unaware of the feasible countermeasures and deadlines that should be followed to reduce the possibility of spreading the contamination to other residents of the smart home environment.

[0006] Therefore, from the above, it can be seen that existing solutions are unable to determine the potential for contaminants to spread within a smart home environment and fail to notify the user to take specific actions or measures to improve the health and safety of the occupants.

[0007] Therefore, a method and apparatus are needed to overcome the above problems. Summary of the invention

[0008] This summary is provided to introduce a selection of concepts in a simplified format that are further described in the detailed description of the invention. This summary is not intended to identify key or essential inventive concepts of the invention, nor is it intended to determine the scope of the invention.

[0009] In an embodiment of the present disclosure, a method for identifying a potential hazardous location performed by an electronic device includes detecting one or more entities in an interior space. The method includes obtaining information associated with at least one external activity performed by the one or more entities. The method includes monitoring at least one subsequent activity of the one or more entities in the interior space. The method includes identifying at least one potential hazardous location by associating the at least one subsequent activity with the obtained information.

[0010] In an embodiment of the present disclosure, an electronic device for identifying a potential dangerous location includes a memory and at least one processor combined with the memory. The processor is configured to detect one or more entities in an internal space. The processor is configured to obtain information associated with at least one external activity performed by the one or more entities. The processor is configured to monitor at least one subsequent activity of the one or more entities in the internal space. The processor is configured to identify at least one potential dangerous location by associating the at least one subsequent activity with the obtained information.

[0011] In an embodiment of the present disclosure, a machine-readable medium containing instructions is provided. When the instructions are executed, at least one processor of the device performs operations corresponding to the above method.

[0012] In order to further illustrate the advantages and features of the inventive device, a more specific description of the present invention will be presented by reference to the specific embodiments of the present invention shown in the accompanying drawings. It should be understood that these drawings only describe typical embodiments of the present invention and therefore should not be considered to limit its scope. Additional features and details will be used in conjunction with the accompanying drawings to describe and explain the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] These and other features, aspects and advantages of the present disclosure will become better understood when the following detailed description is read with reference to the accompanying drawings, in which like characters refer to like parts throughout the drawings, and in which:

[0014] Figure 1An example scenario for a hazard detection application in a smart home environment according to existing solutions is shown;

[0015] Figure 2 An example scenario for a hazard detection application in a smart home environment according to existing solutions is shown;

[0016] Figure 3A A general system architecture for identifying dangerous locations in a smart home environment according to an embodiment of the present disclosure is shown;

[0017] Figure 3B A schematic block diagram of an electronic device for identifying dangerous locations in a smart home environment according to an embodiment of the present disclosure is shown;

[0018] Figure 4 A detailed system architecture for identifying dangerous locations in a smart home environment according to an embodiment of the present disclosure is shown;

[0019] Figure 5 Object recognition based on variability and modeling of signal reception according to an embodiment of the present disclosure is shown;

[0020] Figure 6 An example scenario for stationary object and obstacle detection according to an embodiment of the present disclosure is shown;

[0021] Figure 7 An example scenario for identifying dangerous locations in a smart home environment according to an embodiment of the present disclosure is shown;

[0022] Figure 8 A block diagram of an entry determination unit according to an embodiment of the present disclosure is shown;

[0023] Fig. 9 A block diagram of an external activity determination unit according to an embodiment of the present disclosure is shown;

[0024] Fig.10 A block diagram of an exposure determination unit according to an embodiment of the present disclosure is shown;

[0025] Fig.11 shows a block diagram of an internal activity monitoring unit according to an embodiment of the present disclosure;

[0026] Fig.12 shows a block diagram of a pollution tracker unit according to an embodiment of the present disclosure;

[0027] Fig.13 shows a block diagram of an action recommender unit according to an embodiment of the present disclosure;

[0028] Fig.14A flow chart for identifying a dangerous location in an electronic device according to an embodiment of the present disclosure is shown;

[0029] Fig.15 Various example scenarios for implementing an electronic device for identifying hazardous locations in a system according to an embodiment of the present disclosure are illustrated.

[0030] In addition, those skilled in the art will appreciate that the elements in the drawings are shown for simplicity and may not necessarily be drawn to scale. For example, a flow chart illustrates a method in terms of the most important steps involved to help improve understanding of various aspects of the present disclosure. In addition, with respect to the construction of an apparatus, one or more components of the apparatus may have been represented in the drawings by conventional symbols, and the drawings may show only those specific details relevant to understanding the embodiments of the present disclosure so as not to obscure the drawings with details that would be readily understood by a person of ordinary skill in the art having the benefit of the description herein. DETAILED DESCRIPTION

[0031] It should be understood at the outset that although illustrative implementations of embodiments of the present disclosure are shown below, any number of techniques (whether currently known or existing) may be used to implement the present disclosure. The present disclosure should not be limited to the illustrative implementations, drawings, and techniques shown below (including the exemplary designs and implementations shown and described herein), but may be modified within the scope of the appended claims and the full scope of their equivalents.

[0032] The term "some", as used herein, is defined as "none, or one, or more than one, or all". Thus, the terms "none", "one", "more than one", "more than one but not all" or "all" would all fall within the definition of "some". The term "some embodiments" may mean no embodiments or one embodiment or several embodiments or all embodiments. Thus, the term "some embodiments" is defined to mean "none embodiments, or one embodiment, or more than one embodiments, or all embodiments".

[0033] The terms and structures used herein are used to describe, teach, and illustrate some embodiments and specific features and elements thereof, and are not intended to limit, restrict, or reduce the spirit and scope of the claims or their equivalents.

[0034] More specifically, unless otherwise stated, any terms used herein (such as but not limited to "include", "comprising", "having", "consisting of" and grammatical variations thereof) do not specify exact limitations or restrictions, and certainly do not exclude the possibility of adding one or more features or elements, and further, unless otherwise specified by restrictive language "must include" or "need to include", the possibility of removing one or more listed features and elements shall not be excluded.

[0035] Regardless of whether a feature or element is used only once, it can still be referred to as "one or more features" or "one or more elements" or "at least one feature" or "at least one element" in any way. In addition, unless otherwise indicated by limiting language (such as "one or more are required..." or "one or more elements are required"), the use of the term "one or more" or "at least one" feature or element does not exclude the absence of the feature or element.

[0036] Unless otherwise defined, all terms used herein, especially any technical and / or scientific terms, may be considered to have the same meaning as commonly understood by one of ordinary skill in the art.

[0037] Embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. The present disclosure provides identification of potential dangerous locations within an Internet of Things (IoT) environment. As an example, an IoT environment may be, for example, but not limited to, a home environment, an office environment, or a commercial environment, etc. According to the present disclosure, the method and system disclosed herein use one or more sensors present in a smart home environment to determine the detection of one or more entities entering the smart home environment. The method also obtains information corresponding to external exposure, wherein the external exposure is associated with at least one external activity performed by the one or more detected entities. In addition, the method includes using one or more sensors present in the smart home environment to monitor at least one subsequent activity of the one or more detected entities after entering the smart home environment. The method also includes identifying at least one potential dangerous location within the smart home environment by associating at least one monitored subsequent activity with the acquired information corresponding to the external exposure. Accordingly, the present disclosure focuses on the health and safety of residents of the smart home environment. A detailed description will be provided in the following paragraphs.

[0038] Figure 1 An example scenario for a hazard detection application in a smart home environment according to existing solutions is shown.

[0039] Figure 1 Four possible scenarios of contamination caused by external activities performed by one or more entities are shown. Consider a smart home environment where a user enters with a bag from a market and places the bag and wallet in different places (110). The bag and wallet then contaminate these areas and these areas should be cleaned (120). The monitoring system of the smart home environment keeps track of multiple dangerous situations that may occur in the smart home environment. Also, the monitoring system can help locate the contamination location and notify the user of the contamination location.

[0040] Another scenario is that the user sneezes within 30 minutes of entering the house from outside (130). Also, the air conditioner is running, blowing air toward the bed where the user sneezed, and the windows are closed. Then, because the user sneezed within the threshold time, the bed is contaminated (140). In addition, due to the blowing direction of the air conditioner, some areas will be contaminated by droplets. Then, the monitoring system of the smart home environment can notify the user of the contaminated location.

[0041] In yet another scenario, a pet enters and jumps on the sofa (150). The pet will contaminate the sofa and the floor. The sofa is then contaminated with dust, bacteria, pollen from outside and should be cleaned before being used by the user (160). In another scenario, the user enters from outside after throwing away garbage and contaminates the indoor air (170). The indoor air is then contaminated and the air purifier should be turned on (180).

[0042] However, users may not be aware of why and how external exposure (such as going to the market or shopping mall, meeting people, bringing external objects into the smart home environment, visiting outside locations, handling garbage, and sneezing) increases the likelihood that contamination will be easily spread to other occupants of the smart home environment.

[0043] Figure 2 An example scenario for a hazard detection application in a smart home environment according to existing solutions is shown.

[0044] In one scenario, a user places a bag of vegetables on a table upon entering the smart home and walks to the bathroom to clean himself (210). Due to the user's movement, other areas of the home environment may be contaminated. In this scenario, conventional solutions do not provide a solution for providing the most appropriate cleaning solution to reduce contamination caused by the user's movement. The user partially contaminates the table by placing the bag of vegetables on the table and partially contaminates the floor leading to the bathroom. However, using conventional solutions, the user may also be unaware of feasible countermeasures and deadlines that should be followed to reduce the possibility of spreading contamination to other residents of the smart home environment.

[0045] Therefore, from Figure 1 and Figure 2 As can be seen in the figure, existing solutions cannot determine the possibility of contaminants spreading in a smart home environment and cannot notify users to take specific actions or measures to improve the health and safety of residents. A method for identifying possible hazardous locations and providing appropriate cleaning solutions according to an embodiment of the present disclosure will be described below.

[0046] Figure 3A1 shows a system block diagram of an electronic device 300 for identifying dangerous locations in a smart home environment according to an embodiment of the present disclosure. The electronic device 300 includes one or more processors 302 and a memory 304. It should be noted that Figure 3A This is merely one example of a particular implementation and is intended to illustrate the types of components that may be included as part of electronic device 300 . Figure 3B is another example of a specific implementation as described below.

[0047] In embodiments of the present disclosure, one or more processors 302 may be operably coupled with memory 304 to execute various algorithms, processes, or functions. Such programs or instructions executed by processor 302 may be stored in any suitable article of manufacture including one or more tangible computer-readable media that collectively store at least instructions or routines, such as memory 304.

[0048] In an embodiment of the present disclosure, the processor 302 may detect one or more entities and obtain information associated with at least one external activity performed by the entity. The processor 302 may monitor the activities of the entity in the interior space and identify at least one potential hazardous location that may be contaminated by the entity.

[0049] Memory 304 may include any non-transitory computer-readable medium known in the art, including, for example, volatile memory (such as static random access memory (SRAM) and dynamic random access memory (DRAM)) and / or non-volatile memory (such as read-only memory (ROM), erasable programmable ROM, flash memory, hard disks, optical disks, and tapes).

[0050] In an embodiment of the present disclosure, the memory 304 may store information associated with at least one external activity performed by an entity. In addition, the memory 304 may store attributes of the entity, which is used to detect the entity in the interior space. The memory 304 stores at least one corrective action for cleaning the interior space and a weight for each corrective action.

[0051] Figure 3B A general system architecture for identifying dangerous locations in a smart home environment according to an embodiment of the present disclosure is shown. Figure 3B An electronic device 300 for identifying dangerous locations within a smart home environment is shown. The electronic device 300 includes one or more processors 302, memory 304, modules / units 308, sensors 306, and databases 310, each of which are integrated with one another.

[0052] As examples, the electronic device 300 may correspond to various devices, such as a personal computer (PC), a tablet PC, a personal digital assistant (PDA), a mobile device, a palmtop computer, a laptop computer, a desktop computer, a communication device, a dashboard, a navigation device, a computing device, or any other machine capable of running an instruction set.

[0053] In an example, the processor 302 may be a single processing unit or multiple units, all of which may include multiple computing units. The processor 302 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, logic processors, virtual processors, state machines, logic circuits, and / or any device that manipulates signals based on operational instructions. In other capabilities, the processor 302 is configured to retrieve and execute computer-readable instructions and data stored in the memory 304.

[0054] In an embodiment of the present disclosure, the processor 302 may be composed of one or more processors. The one or more processors 302 may be operably combined with the memory 304 to execute various algorithms, processes, or functions. Such programs or instructions executed by the processor 302 may be stored in any suitable article including one or more tangible computer-readable media, wherein the one or more tangible computer-readable media at least collectively store instructions or routines, such as the memory 304.

[0055] In an embodiment of the present disclosure, the processor 302 may detect one or more entities and obtain information associated with at least one external activity performed by the entity. The processor 302 may monitor the activities of the entity in the interior space and identify at least one potential hazardous location that may be contaminated by the entity.

[0056] Memory 304 may include any non-transitory computer-readable medium known in the art, including, for example, volatile memory (such as static random access memory (SRAM) and dynamic random access memory (DRAM)) and / or non-volatile memory (such as read-only memory (ROM), erasable programmable ROM, flash memory, hard disks, optical disks, and tapes).

[0057] In an embodiment of the present disclosure, the memory 304 may store information associated with at least one external activity performed by an entity. In addition, the memory 304 may store attributes of the entity, which is used to detect the entity in the interior space. The memory 304 may store at least one corrective action for cleaning the interior space and a weight for each corrective action.

[0058] In an example, a module, an engine and / or a unit 308 may include a program, a subroutine, a part of a program, a software component or a hardware component capable of performing a prescribed task or function. As used herein, a module, an engine and / or a unit 308 may be implemented on a hardware component (such as a server) independently of other modules, or a module may exist on the same server with other modules, or exist in the same program. A module, an engine and / or a unit 308 may be implemented on a hardware component (such as a processor, one or more microprocessors, a microcomputer, a microcontroller, a digital signal processor, a central processing unit, a state machine, a logic circuit and / or any device based on an operation instruction to manipulate a signal). A module, an engine and / or a unit 308 may be configured to perform any described function when being run by a processor 302. According to an embodiment of the present disclosure, a module, an engine and / or a unit 308 may include a monitoring unit, an entry determination unit, an external activity determination unit, an exposure determination unit, an internal activity monitoring unit, a pollution tracker unit, an action priority and a recommender unit and a feedback analyzer unit. Each unit has hardware and software with a unique design and configuration.

[0059] As another example, the database 310 may be implemented with integrated hardware and software. The hardware may include a hard disk controller with programmable search capabilities or a software system running on general hardware. Examples of databases include, but are not limited to, in-memory databases, cloud databases, distributed databases, embedded databases, and the like. The database 310, among other functions, serves as a repository for storing data processed, received, and generated by one or more of the processors and modules / engines / units. As another example, the sensor 306 may include a plurality of ultra-wideband (UWB)-based sensors for tracking and identifying one or more entities of a smart home environment. The one or more entities may include, but are not limited to, various stationary objects and non-stationary objects within a smart home environment.

[0060] Module / engine / unit 308 can be implemented with an AI unit that may include multiple neural network layers. Examples of neural networks include, but are not limited to, convolutional neural networks (CNNs), deep neural networks (DNNs), recursive neural networks (RNNs), and restricted Boltzmann machines (RBMs). Learning technology is a method for training a predetermined target device (e.g., a robot) using multiple learning data to enable, allow, or control the target device to make a determination or prediction. Examples of learning technology include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. At least one of multiple CNNs, DNNs, RNNs, RMB models, etc. can be implemented to implement the execution of the mechanism of the present subject through an AI model. Functions associated with AI can be performed by non-volatile memory, volatile memory, and a processor. The processor may include one or more processors. At this time, one or more processors may be a general-purpose processor (such as a central processing unit (CPU), an application processor (AP), etc.), a graphics processing unit (such as a graphics processing unit (GPU), a visual processing unit (VPU)), and / or an AI-specific processor (such as a neural processing unit (NPU)). One or more processors control the processing of input data according to predetermined operating rules or artificial intelligence (AI) models stored in non-volatile memory and volatile memory. The predetermined operating rules or artificial intelligence models are provided by training or learning. The following paragraphs will explain in detail how Figure 3B A detailed description of each of the above components is shown in FIG.

[0061] Figure 4 The detailed system architecture for identifying dangerous locations in a smart home environment according to an embodiment of the present disclosure is shown. Figure 3A and Figure 3B To explain the electronic device 300. In addition, for the convenience of explanation, the reference numerals remain the same. In addition, for the convenience of explanation, the electronic device 300 will be explained by Figure 7 To explain Figure 4 .

[0062] According to an embodiment of the present disclosure, the electronic device 300 includes a monitoring unit 301, an entry determination unit 303, an external activity determination unit 305, an exposure determination unit 308, an internal activity monitoring unit 312, a pollution tracker unit 314, an action priority and recommender unit 316, and a feedback analyzer unit 318, which are combined with each other. Figure 3B The model / unit 308 shown may include all of the above-mentioned units / models. In addition, the database 310 may include, for example but not limited to, an identity classifier parameter database, a location-activity database, a location-hazard mapping database, an internal object database 310-1, an internal activity log database, a human / pet activity log database 310-2, a historical intelligent action database 310-3, and a user intelligent action log database 310-4.

[0063] like Figure 4 As shown, in a smart home environment, multiple UWB-based sensors 306 continuously monitor various stationary objects and non-stationary objects present in the range of multiple UWB-based sensors in the location boundary for their identification. In a non-limiting example, the various stationary objects and non-stationary objects may include users, organisms, non-organisms, objects (such as tables, shelves, pillars, etc.), the location of the object, and the context related to the location of the object. In a non-limiting example, the location of the object may include a bedroom, a kitchen, a living room, etc. Without departing from the scope of the present invention, the term "multiple UWB-based sensors" may be optionally referred to as "UWB sensors" throughout the disclosure.

[0064] like Figure 4 The UWB sensor 306 of the monitoring unit 301 shown may be configured to send and receive sensor data from one or more entities. The received sensor data may be further sent to the monitoring unit 301. The monitoring unit 301 further identifies one or more entities based on the received sensor data. As an example, the identified one or more entities may include various stationary objects or non-stationary objects. Stationary objects may include objects such as but not limited to non-living things, tables, shelves, pillars, etc. Non-stationary objects may include objects such as but not limited to users, living things, robots, animals, pets, etc. With the help of Figure 5 and Figure 6 Explain the detailed object recognition process.

[0065] Figure 5 An example of object recognition based on variability and modeling of signal reception according to an embodiment of the present disclosure is shown. Figure 5 In the UWB radar-based system of FIG. (a), a signal pulse (Tx) is sent to various stationary and non-stationary objects, and the scattered signal (Rx) is received by the antenna. Using various properties of the signal (such as signal strength (SS), time of arrival (ToA), and angle of arrival (AoA)), small changes in movement can be tracked, and unique physical properties of the object can be identified. The physical properties may include, for example, the length of the object, the width of the object, the distance of the object relative to the user or the distance between the object and any other object, the gap between objects, the direction of the object, etc. Such physical characteristics are unique to each of the objects or people, and therefore, the classification type can be marked with a uniquely identifiable item / object / person for its identification, and the classification type is stored in the identity classifier parameter database.

[0066] Figure 5FIG. (b) 520 shows the UWB scattered signal variation and propagation geometry of the received signal. Therefore, the electronic device 300 can accurately understand the position, size, direction and movement of each of the stationary objects and non-stationary objects in the location.

[0067] Figure 6 An example scenario for stationary object and obstacle detection according to an embodiment of the present disclosure is shown. It can be seen that stationary objects and various obstacles in the location of the smart home environment can be detected based on the UWB scattered signal changes.

[0068] According to an embodiment, Figure 4 The monitoring unit 301 shown periodically monitors the external activities of at least one user through the user's electronic device 320. The user's external activities may include but are not limited to visiting different places (such as markets, shopping malls, etc.), or sports activities (such as going to the gym, etc.), or moving outside the smart home environment. In a non-limiting example, the user's electronic device 320 may include a mobile tracker device, a location tracker device, a smart watch, a smart wearable device, a vehicle GPS, etc. In addition to at least one or more users' external activities, the electronic device 320 may also store the time spent on at least performing one or more users' external activities. Afterwards, the monitored external activity information of at least one user is sent to the external activity determination unit 305.

[0069] Figure 7 An example scenario for identifying dangerous locations in a smart home environment according to an embodiment of the present disclosure is shown. Consider a scenario in which a user comes in from outside and enters the smart home environment (710). After entering, the user places his bag on the table of the smart home environment and walks to other locations within the smart home environment. The electronic device 300 identifies possible dangerous locations within the smart home environment due to the user's subsequent activities. The electronic device (300) may notify the user of possible dangerous locations within the smart home environment. In addition, the electronic device (300) may provide a solution by using other devices (720) (e.g., a speaker). In the following paragraphs, Figure 8-Figure 13 to explain the detailed instructions for identifying dangerous locations within a smart home environment.

[0070] In an embodiment, Figure 8The entry determination unit 303 shown detects the user entering or leaving the smart home environment. In order to detect the entry or exit of the user, the UWB sensor 306 may send a signal to the user passing through or leaving the entrance or exit of the smart home environment. The transmitted signal is reflected by the user and received at the UWB sensor 306 end. The reflected signal has the physical attributes of the user. The UWB sensor 306 also sends the reflected signal to the entry determination unit 303. The entry determination unit 303 compares the received physical attributes of the user with the predefined or pre-stored physical attributes of one or more entities of the smart home environment. The physical attributes of one or more entities are stored in the identity classifier database. After comparing the physical attributes of the user with the predefined physical attributes of one or more entities, it is determined that the user enters. Based on the physical attributes of the user, the user is also identified as a smart home environment entity. The user entry time is also stored. The information is then further sent to the exposure determination unit 308 and the internal monitoring unit 312.

[0071] According to an embodiment of the present disclosure, the monitoring unit 301 uses the UWB sensor 306 to periodically monitor one or more entities and generates a series of timestamp data 801. In a non-limiting example, the generated series of timestamp data 801 includes information (such as height, gender, etc.) of one or more entities at a specific timestamp. The entry determination unit uses the information 303 to classify the type of one or more entities using a DNN model and a RNN model. The classified data is then stored in an identity classifier parameter database 802. Then, the entry determination unit also uses the information 303 to identify an entity from one or more entities in the smart home environment. For example, Figure 8 As shown, the detected entity 803 is the "main door" and the entity is identified as a smart home environment entity.

[0072] Thereafter, the external activity determination unit 305 of the electronic device 300 at least calculates one or more external activities performed by the user.

[0073] Return to reference Figure 7 ,like Fig. 9 The external activity determination unit 305 shown may be configured to at least periodically receive one or more external activities performed by the user. At least one external activity is received from the user's electronic device 320. The external activity determination unit 305 also receives information related to the time spent in at least performing the one or more external activities. This information is stored in the location-activity database 902 and then used by the external activity determination unit 303 to at least detect one or more external activities performed by the user.

[0074] Now, in an embodiment, when the external activity determination unit 305 continuously receives information from the monitoring unit 301, it generates a series of information 903. In a non-limiting example, the information 903 includes the movement type of one or more entities, etc. The information is used by the external activity determination unit 305 to classify the type of external activity using a reinforcement learning-based classification model. Then, the information is stored in the location-activity database 902, and is also used by the external activity determination unit 303 to at least determine one or more external activities performed by one or more entities. The activity determination unit 303 also stores the time spent in performing the one or more external activities, weather conditions, population density, etc. Fig. 9 As shown, in a non-limiting example, the user's external activity 904 is to go to the market. The time spent in the market is determined by the entry time and the exit time. Fig. 9 As shown, the entry time is 11:30 and the departure time is 12. The weather conditions are sunny.

[0075] Thereafter, the exposure determination unit 308 calculates external exposure associated with at least one or more external activities performed by the user.

[0076] Return to reference Figure 7 ,like Fig.10 The exposure determination unit 308 shown may be configured to calculate external exposure associated with at least one or more external activities performed by the user. In order to calculate the external exposure, in an initial step, the exposure determination unit obtains information stored in the location activity database 902 about at least one external activity performed by the user. Afterwards, in the next step, the exposure determination unit 308 calculates the external exposure pollution degree (quotient) for the user based on the information related to the environmental health condition when the user at least performs one or more external activities. The environmental health condition information includes, but is not limited to, population density information, weather condition information, humidity level information, dust information, pollen, or any kind of pollutants obtained. According to an embodiment, the environmental health condition may be obtained from the user's electronic device 320. Afterwards, the exposure determination unit 308 obtains the time spent by the user when at least performing one or more external activities from the activity determination unit 303. Now, the external exposure pollution degree includes two pieces of information, namely, information related to the environmental health condition and information related to the time spent by the user when at least performing one or more external activities. In a further step, the exposure determination unit 308 calculates the external exposure level by associating the acquired information with the external exposure pollution degree. After calculating the external exposure level, the exposure determination unit 308 also determines a risk factor associated with the calculated external exposure. The risk factor can be determined by comparing the external exposure contamination level with a predetermined threshold. If the external exposure contamination level is greater than the threshold, the risk factor is determined to be a high level of risk.

[0077] For example, in Fig. 9 In the example, in the initial step, the exposure determination unit 308 determines that the user's external activity is to go to the market. Then in the next step, the exposure determination unit 308 looks up the environmental sanitation conditions of the market. Then in the next step, the exposure determination unit 308 checks the time of entering from the market and the time of leaving from the market. The entry time is 11:30 and the exit time is 12. Then in the next step, the exposure determination unit 308 associates the information. After association, the exposure determination unit 308 calculates the exposure time. In this way, the user's exposure time is 30 minutes. Since the total time spent by the user in the market is greater than the predetermined threshold, the risk factor is high in this case. In addition, based on the risk factor, the user is identified as a potential pollution carrier. According to an embodiment, the risk factor is determined by comparing the external exposure pollution degree with a predetermined threshold.

[0078] Now, in one embodiment, when the exposure determination unit 308 continuously receives information from the entry determination unit 303 and the external activity determination unit 305, it generates a series of information 1004 and 1003. In a non-limiting example, the information 1003 includes different places visited by one or more entities, the time spent, the environmental health conditions at the visited places, etc. The information 1004 and 1003 can be used by the exposure determination unit 308 to classify the external exposure pollution level using a supervised ML model. Fig. 9 As shown, in a non-limiting example, the external exposure 1005 duration is 30 minutes and the risk factor is high.

[0079] After calculating the external exposure, the internal activity monitoring unit 312 of the electronic device 300 also calculates the subsequent activities of the user after entering the smart home.

[0080] Return to reference Figure 7 ,like Fig.11 The internal activity monitoring unit 312 shown may be configured to monitor at least one subsequent activity of the user after the user enters the smart home environment. The user's entry into the smart home environment is determined by the information received from the entry determination unit 303. Afterwards, the internal activity monitoring unit 312 uses the UWB sensor 306 present in the smart home environment to monitor the subsequent activities. In order to monitor the user's subsequent activities, the UWB sensor 306 sends a signal to the user. The UWB sensor 306 receives the user's physical attributes from the reflected signal. The physical attributes may include, but are not limited to, signal strength (SS), time of arrival (ToA), and angle of arrival (AoA).

[0081] The internal activity monitoring unit 312 also compares the received physical attributes of the user with a plurality of predefined physical attributes of one or more entities stored in the internal object database. The internal activity monitoring unit 312 also monitors subsequent activities of the user based on the comparison. Subsequent activities may include but are not limited to the movement of the user, placing one or more objects existing in the smart home from one place to another, approaching one or more objects existing in the smart home environment, placing one or more external objects on the table, etc. Fig.11 As shown, in a non-limiting example, the user's subsequent activity 1102 is contacting a chair present in the smart home.

[0082] In yet another embodiment, when the internal activity monitoring unit 312 continuously receives information from the entry determination unit 303, it generates a series of information 1004. In a non-limiting example, the information 1003 includes the user's movement, placing one or more objects existing in the smart home from one place to another, etc. The information 1004 is used by the internal activity monitoring unit 312 to track the user's subsequent activities by using the DNN classification model and the RNN classification model. Fig. 9 As shown, in a non-limiting example, the user's subsequent activity includes the user contacting the seat.

[0083] After calculating the subsequent activities, the pollution tracker unit 314 identifies potential dangerous or polluted locations in the smart home environment.

[0084] Return to reference Figure 7 ,like Fig.12 The pollution tracker unit 314 shown can be configured to associate the monitored subsequent activities 1102 of the user with the calculated external exposure information 1005. The pollution tracker unit 314 also identifies at least one potential dangerous location based on the associated information. In other words, the pollution tracker unit 314 receives the monitored subsequent activities 1102 from the internal activity monitoring unit 312. The pollution tracker unit then also associates the monitored subsequent activities of the user with the calculated external exposure information 1005. When associating these two pieces of information, the pollution tracker can determine a potential dangerous location in the smart home environment.

[0085] According to an embodiment, the pollution tracker unit 314 continuously receives information from the internal activity monitoring unit 312 and the external activity determination unit 305, which generates a series of information 1102 and 1005. The information 1102 and 1005 are used by the exposure determination unit 308 to classify potential dangerous locations in the smart home environment using a supervised ML model. Fig. 9 As shown, in a non-limiting example, for example, in Fig.12The subsequent activity 1204 of the user in the smart home environment is to touch the seat. The potential dangerous location in the smart home environment is the seat. Afterwards, the pollution tracker unit sends this information to the action recommender unit 316.

[0086] Return to reference Figure 7 , the action priority and recommender unit 316 may be configured to determine at least one of the corrective actions to mitigate the risk associated with the potentially identified hazardous location within the smart home environment. The action priority and recommender unit 316 also recommends corrective actions to mitigate the risk associated with the potentially identified hazardous location within the smart home environment. Based on the recommendation, the action priority and recommender unit 316 also recommends taking one or more actions. The one or more actions may include, but are not limited to, cleaning from available devices (such as a robotic cleaner, an air purifier, etc.).

[0087] In yet another embodiment, while the action priority and recommender unit 316 continuously receives information 1202 from the pollution tracker unit, it generates a series of information about corrective actions. This information is used by the action priority and recommender unit 316 to classify corrective actions using a reinforcement learning model. Fig.13 As shown, in a non-limiting example, the recommended corrective action 1302 is to clean the sofa with a robotic cleaner.

[0088] In an embodiment, the electronic device 300 further includes a feedback analyzer unit 318. The feedback analyzer unit 318 monitors the corrective actions taken to understand the action priority and the recommendation effect of the recommender unit 316. The feedback analyzer unit 318 also adjusts the weight of the reinforcement model prepared for the recommendation.

[0089] Fig.14 FIG. 3 shows a flow chart for implementing the identification of dangerous locations in an electronic device according to an embodiment of the present disclosure. According to an embodiment, the method 1400 is implemented in the electronic device 300. Fig.13 The detailed description and implementation modes are explained, so the same contents are omitted here for the sake of brevity.

[0090] At operation S1402, method 1400 includes detecting one or more entities in the interior space. Method 1400 may include using one or more sensors present in the smart home environment to detect one or more entities entering the smart home environment.

[0091] According to an embodiment, the internal space may include a place where a user stays, lives, or works (eg, a smart home, an office, a store, etc.) The internal space may be polluted by various reasons and needs to be cleaned for the user.

[0092] Thereafter, at operation S1404, the method 1400 includes obtaining information associated with at least one external activity performed by one or more entities. The method 1400 may include acquiring information corresponding to an external exposure, wherein the external exposure is associated with at least one external activity performed by the detected one or more entities.

[0093] According to an embodiment, at least one external activity may include any activity performed by the user in a place other than the internal space. For example, at least one external activity may include taking out the trash, walking the dog, shopping in a mall, exercising, working, etc.

[0094] Thereafter, at operation S1406, method 1400 includes monitoring at least one subsequent activity of the one or more entities in the interior space. Method 1400 may include using one or more sensors present in the smart home environment to monitor at least one subsequent activity of the one or more detected entities after entering the smart home environment.

[0095] According to an embodiment, at least one subsequent activity may include all activities performed by the user after entering the internal space after the external activity. For example, at least one subsequent activity may include washing hands, cooking, lying on the sofa, changing clothes, etc.

[0096] Thereafter, at operation S1408, method 1400 includes identifying at least one potential danger location by associating at least one subsequent activity with the obtained information. Method 1400 may include identifying at least one potential danger location within the smart home environment by associating at least one subsequent activity monitored with the obtained information corresponding to the external exposure.

[0097] Various exemplary scenarios will be explained next. Fig.15 Various example scenarios for implementing identification of dangerous locations within a smart home in a system are shown. According to example scenario (a) (1510), a first user of the smart home sneezes in his bedroom within 30 minutes of entering the smart home environment. Accordingly, first, the monitoring unit 301 monitors the external activity of the first user. Thereafter, the external activity determination unit 305 determines that the external activity of the first user is a sneeze. Thereafter, the entry determination unit 303 determines that the entity is a bed and the AC is in the on state. Thereafter, the exposure determination unit 308 determines that there is a high exposure to the bed. Thereafter, the internal monitoring unit 312 monitors that the second user immediately sits on the bed that is contaminated due to the sneeze droplets of the first user. Thereafter, the contamination tracker unit 314 monitors that the first user affects the second user due to contamination. Thereafter, the action recommender unit 316 recommends to the second user that there is a possibility of catching a cold due to contamination from the first user.

[0098] According to example scenario (b) (1520), it is identified that the dangerous location or contaminated area is a bed, and no recommendation is made. Accordingly, first, the monitoring unit 301 monitors the external activities of the first user. Thereafter, the external activity determination unit 305 determines that the first user sneezes within 30 minutes after entering the bedroom of the smart home, but the window is open and there is no downward AC airflow. Thereafter, the entry determination unit 303 determines that the entity is a bed and the AC is turned off. Thereafter, the exposure determination unit 308 determines that there is a high exposure to the bed. Thereafter, the internal monitoring unit 312 monitors that the second user sits on the bed that is contaminated due to the sneeze droplets of the first user after 10 minutes. Thereafter, the contamination tracker unit 314 monitors that the second user contacts after the high contamination time. Thereafter, the action recommender unit 316 does not recommend anything to the second user because there is a 10-minute gap and the window is also open.

[0099] According to example scenario (c) (1530), when a pet enters and jumps onto the sofa of the smart home environment, a dangerous location or contaminated area is identified. Accordingly, first, the monitoring unit 301 monitors the external activities of the pet. Thereafter, the external activity determination unit 305 determines that the external activities of the pet are playing. Thereafter, the entry determination unit 303 determines that the entity is a pet. Thereafter, the exposure determination unit 308 determines that there is moderate exposure. Thereafter, the internal monitoring unit 312 monitors that a child comes in 3 minutes later and sits on the sofa that is contaminated due to the pet jumping off. Thereafter, the contamination tracker unit 314 monitors the contamination of the sofa surface caused by the pet's activities. Thereafter, the action recommender unit 316 recommends cleaning the sofa and the child.

[0100] According to example scenario (d) (1540), when a person enters a smart home with a garbage bag after disposing of garbage, a dangerous location or a polluted area is identified. Accordingly, first, the monitoring unit 301 monitors the external activities of the person. Thereafter, the external activity determination unit 305 determines that the external activity of the person is to dispose of garbage. Thereafter, the entry determination unit 303 determines that the entity is a person. Thereafter, the exposure determination unit 308 determines that there is a high exposure due to the garbage bag. Thereafter, the internal monitoring unit 312 detects that there is an odor from the garbage bag and that the floor tiles are contaminated due to the garbage bag. Thereafter, the pollution tracker unit 314 detects that the surface of the floor tiles is contaminated by the activities of the person. Thereafter, the action recommender unit 316 recommends using a robot cleaner to clean the surface of the floor tiles and starting an air purifier to purify the polluted air.

[0101] Some of the example embodiments disclosed herein may be implemented using a processing circuit.For example, some of the example embodiments disclosed herein may be implemented using at least one software program that runs on at least one hardware device and performs a network management function to control an element.

[0102] Although the present disclosure has been described using specific language, no limitation is intended thereby. It will be apparent to those skilled in the art that various working modifications may be made to the methods described in order to implement the inventive concepts taught herein.

[0103] The accompanying drawings and the foregoing description provide examples of embodiments. It will be appreciated by those skilled in the art that one or more of the elements described may also be combined into a single functional element. Alternatively, a particular element may be divided into a plurality of functional elements. Elements from one embodiment may be added to another embodiment. For example, the order of the processes described herein may be changed and are not limited to the methods described herein.

[0104] Furthermore, the actions of any flow chart need not be implemented in the order shown; nor do all actions necessarily need to be performed. Furthermore, those actions that are independent of other actions may be performed in parallel with the other actions. The scope of the embodiments is not limited by these specific examples. Many variations (such as differences in structure, size, and use of materials) are possible, whether or not explicitly given in the specification. The scope of the embodiments is at least as wide as the scope given by the appended claims.

[0105] Benefits, other advantages, and solutions to problems have been described above with respect to specific embodiments. However, the benefits, advantages, solutions to problems, and any components that enable or make apparent any benefit, advantage, or solution should not be construed as a critical, required, or essential feature or component of any or all claims.

[0106] In an embodiment, the present subject matter relates to a method for identifying dangerous locations within a smart home environment. The method includes using one or more sensors present in the smart home environment to detect one or more entities entering the smart home environment. The method also includes obtaining information corresponding to external exposure, wherein the external exposure is associated with at least one external activity performed by the detected one or more entities. The method also includes using one or more sensors present in the smart home environment to monitor at least one subsequent activity of the detected one or more entities after entering the smart home environment. The method also includes identifying at least one potentially dangerous location within the smart home environment by associating at least one monitored subsequent activity with the obtained information corresponding to the external exposure.

[0107] In an embodiment, the present subject matter relates to an electronic device for identifying dangerous locations within a smart home environment. The electronic device includes at least one processor in combination with a memory. The processor is configured to detect one or more entities entering the smart home environment using one or more sensors present in the smart home environment. The processor is also configured to obtain information corresponding to external exposure, wherein the external exposure is associated with at least one activity performed by the detected one or more entities. The processor is also configured to monitor at least one subsequent activity of the detected one or more entities after entering the smart home environment using one or more sensors present in the smart home environment. The processor is also configured to identify at least one potential dangerous location within the smart home environment by associating the monitored at least one subsequent activity with the acquired information corresponding to the external exposure.

[0108] In an embodiment of the present disclosure, a method for identifying a potential hazardous location performed by an electronic device includes detecting one or more entities in an interior space. The method includes obtaining information associated with at least one external activity performed by the one or more entities. The method includes monitoring at least one subsequent activity of the one or more entities in the interior space. The method includes identifying at least one potential hazardous location by associating the at least one subsequent activity with the obtained information.

[0109] In an embodiment of the present disclosure, the method includes sending a signal to one or more entities. The method includes receiving a reflected signal from the one or more entities, and obtaining attributes of the one or more entities based on multiple angle values ​​of the reflected signal. The method includes comparing the obtained attributes of the one or more entities with multiple predefined attributes of the one or more entities stored in a database. The method includes detecting the one or more entities based on the result of the comparison.

[0110] In an embodiment of the present disclosure, the method includes obtaining information related to the environmental health conditions of a visited location based on information associated with at least one external activity. The method includes calculating an external exposure pollution level for one or more entities based on the information related to the environmental health conditions of the visited location. The method includes calculating external exposure by associating the obtained information with the external exposure pollution level.

[0111] In an embodiment of the present disclosure, the method includes information related to the environmental health conditions of the visited place including at least one of the population density of the visited place, weather condition information of the visited place, humidity level information of the visited place, dust information, pollen, or any type of pollutants obtained at the visited place.

[0112] In an embodiment of the present disclosure, the information associated with the at least one external activity includes at least one of a place visited by the one or more entities or a total time spent by the one or more entities in performing an action at the visited place.

[0113] In an embodiment of the present disclosure, the method includes correlating subsequent activities with the calculated external exposure. The method includes identifying at least one potential hazardous location based on the results of the correlation.

[0114] In an embodiment of the present disclosure, the method further includes determining at least one corrective action associated with at least one potentially hazardous location based on a weight of the at least one corrective action. The method includes providing information related to the at least one corrective action. The method includes updating the weight of the at least one corrective action based on feedback information related to the at least one corrective action.

[0115] In an embodiment of the present disclosure, an electronic device for identifying a potential dangerous location includes a memory and at least one processor combined with the memory. The processor is configured to detect one or more entities in an internal space. The processor is configured to obtain information associated with at least one external activity performed by the one or more entities. The processor is configured to monitor at least one subsequent activity of the one or more entities in the internal space. The processor is configured to identify at least one potential dangerous location by associating the at least one subsequent activity with the obtained information.

[0116] In an embodiment of the present disclosure, a machine-readable medium containing instructions is provided. When the instructions are executed, at least one processor of the device performs operations corresponding to the above method.

Claims

1. A method for identifying a potential hazardous location performed by an electronic device 300, the method comprising: Detecting one or more entities in the interior space; obtaining information associated with at least one external activity performed by the one or more entities; monitoring at least one subsequent activity of the one or more entities in the interior space; as well as At least one potentially hazardous location is identified by correlating the at least one subsequent activity with the obtained information.

2. The method of claim 1, wherein: The step of detecting the one or more entities in the interior space further comprises: sending a signal to the one or more entities; receiving a reflected signal from the one or more entities; Obtaining attributes of the one or more entities based on a plurality of angle values ​​of the reflected signal; comparing the obtained attributes of the one or more entities with a plurality of predefined attributes of the one or more entities stored in the database 310; and The one or more entities are detected based on a result of the comparing.

3. The method according to any one of claims 1 to 2, wherein: The step of identifying the at least one potentially hazardous location comprises: obtaining information related to environmental health conditions of a visited location based on information associated with at least one external activity; calculating external exposure contamination levels for the one or more entities based on information related to environmental health conditions at the visited location; and External exposure is calculated by correlating the obtained information related to the environmental health conditions with the external exposure contamination level.

4. The method according to any one of claims 1 to 3, wherein: The information related to the environmental health conditions of the visited places includes at least one of population density of the visited places, weather condition information of the visited places, humidity level information of the visited places, dust information, pollen, or any kind of pollutants obtained at the visited places.

5. The method according to any one of claims 1 to 4, wherein: The information associated with at least one external activity includes at least one of a location visited by the one or more entities or a total time spent by the one or more entities while performing an action at the visited location.

6. The method according to any one of claims 1 to 5, wherein: The step of identifying the at least one potentially hazardous location comprises: associating the subsequent activity with the calculated external exposure; and The at least one potential hazardous location is identified based on a result of the correlation.

7. The method according to any one of claims 1 to 6, further comprising: determining, based on a weight of at least one corrective action, the at least one corrective action associated with the at least one potentially hazardous location; providing information related to the at least one corrective action; as well as A weight of the at least one corrective action is updated based on feedback information related to the at least one corrective action.

8. An electronic device (300) for identifying a potential dangerous location within a smart home environment, comprising: Memory (304); as well as At least one processor (302), in combination with the memory (304), the at least one processor (302) being configured to: Detecting one or more entities in the interior space; obtaining information associated with at least one external activity performed by the one or more entities; monitoring at least one subsequent activity of the one or more entities in the interior space; as well as At least one potentially hazardous location is identified by correlating the at least one subsequent activity with the obtained information.

9. The electronic device (300) according to claim 8, wherein: The at least one processor (302) is configured to: sending a signal to the one or more entities; receiving a reflected signal from the one or more entities; Obtaining attributes of the one or more entities based on a plurality of angle values ​​of the reflected signal; comparing the obtained attributes of the one or more entities with a plurality of predefined attributes of the one or more entities stored in a database (310); as well as The one or more entities are detected based on a result of the comparing.

10. The electronic device (300) according to any one of claims 8 to 9, wherein: The at least one processor (302) is configured to: obtaining information related to environmental conditions of a visited location based on information associated with at least one external activity; calculating an external exposure contamination level for the one or more entities based on information related to environmental health conditions at the visited location; as well as External exposure is calculated by correlating the obtained information related to the environmental health conditions with the external exposure contamination level.

11. The electronic device (300) according to any one of claims 8 to 10, wherein: The information related to the environmental health conditions of the visited places includes at least one of population density of the visited places, weather condition information of the visited places, humidity level information of the visited places, dust information, pollen, or any kind of pollutants obtained at the visited places.

12. The electronic device (300) according to any one of claims 8 to 11, wherein: The information associated with at least one external activity includes at least one of a location visited by the one or more entities or a total time spent by the one or more entities while performing an action at the visited location.

13. The electronic device (300) according to any one of claims 8 to 12, wherein: The processor (302) is configured to: associating the subsequent activity with the calculated external exposure; and The at least one potential hazardous location is identified based on a result of the correlation.

14. The electronic device (300) according to any one of claims 8 to 13, wherein: The at least one processor (302) is further configured to: determining, based on a weight of at least one corrective action, the at least one corrective action associated with the at least one potentially hazardous location; providing information related to the at least one corrective action; as well as A weight of the at least one corrective action is updated based on feedback information related to the at least one corrective action.

15. A machine-readable medium containing instructions that, when executed, cause at least one processor of a device to perform operations corresponding to the method of any one of claims 1 to 7.

Images