System and method for remote monitoring and control of household appliances

Abstract

A household appliance includes a camera assembly operable to obtain an image. A method of operating the household appliance includes obtaining a first image of an exterior of the household appliance and a second image of an interior of the household appliance. The method may also include obtaining a reading from a sensor. The method further includes generating a real-time interactive visual representation of the household appliance from the first image and the second image. The visual representation may also be generated from the sensor reading. The method further includes displaying the real-time interactive visual representation of the household appliance on a remote user interface device. The method may also include receiving an inspection input at the remote user interface device. The method further includes receiving a control input at the remote user interface device and activating the household appliance in response to the control input.

Description

FIELD

[0001] The present subject matter relates generally to household appliances, such as laundry appliances or cooking appliances, and more specifically, to systems and methods for using a camera assembly to facilitate remote monitoring and control of household appliances.BACKGROUND

[0002] Household appliances are utilized generally for a variety of tasks by a variety of users. For example, a household may include such appliances as laundry appliances, e.g., a washing machine appliance and / or dryer appliance, kitchen appliances, e.g., an oven appliance, a dishwasher appliance, etc., along with other various appliances.

[0003] In some situations, operation of a household appliance by certain users or under certain conditions may be undesirable. For example, some household appliances may include features which generate high levels of heat, e.g., heating elements of a cooking appliance or a heat system of a dryer appliance, and / or may include enclosable internal volumes, such as inside of a drum of a dryer appliance, a wash basket of a washing machine appliance, or a food storage compartment in a refrigerator or freezer appliance. Accordingly, many household appliances include components or features which are not desired for unsupervised or unverified use.

[0004] Laundry appliances, such as washing machine appliances and dryer appliances, are commonly used to wash and dry, respectively, a load of clothes. Specifically, washing machine appliances generally include a wash tub for containing water or wash fluid and a wash basket rotatably mounted within the wash tub for receiving the load of clothes. These washing machines are typically equipped to operate in one or more modes or cycles, such as wash, rinse, and spin cycles. After the washing machine processes are complete, the load of clothes is moved over the to the dryer, which includes a cabinet with a drum rotatably mounted therein and a heating assembly that supplies heated air into a chamber of the drum, e.g., through a duct mounted to a back wall of the drum, to facilitate a drying process.

[0005] Notably, however, the efficacy of operating cycles in laundry appliances depends in large part on characteristics of the added load, user inputs, cycle selections and parameters, and other user-dependent variables. Moreover, incorrect cycle settings or improper load type or preparation may result in damaged clothes and even dangerous operating conditions. For example, if an undesirable object or item is included in the load of clothes, damage to the clothes or the appliance may result during normal operation. In addition, improper cycle settings for a particular type of load, such as hot water for a load of delicate items, can result in ruined clothing. Moreover, operation of laundry appliances by children or unauthorized users may result in safety hazards.

[0006] Users can operate an oven appliance to cook food items as desired by selecting or manipulating various operational features of the oven appliance, such as e.g., the temperature setting or mode of operation (i.e., bake, broil, etc.). In some instances, users may desire to select or manipulate these operational features remotely or “hands free.” Some oven appliances may include a “remote enable” mode that allows users to remotely modify operational features of the oven appliance. Some features, however, may require that a user be in attendance or in the presence of the oven appliance when operating the oven appliance, which is referred to herein as “attended operation.” Examples of features that may require attended operation can include turning on the oven appliance, selecting a temperature setting, or choosing a mode of operation of the oven appliance. Among other safety reasons, requiring attended operation ensures that a user has had an opportunity to check for non-food items placed within the cooking chamber of the oven appliance before it is turned on.

[0007] Accordingly, household appliances with features for improved safety and performance of operating cycles would be desirable. More specifically, systems and methods for monitoring and adjusting operation of the appliance remotely, e.g., remotely verifying contents of a cavity such as the wash tub of a washing machine appliance or cooking chamber of an oven appliance, remotely monitoring heating element status in appliances such as a dryer appliance or an oven appliance, etc., would be particularly beneficial.BRIEF DESCRIPTION OF THE INVENTION

[0008] Aspects and advantages of the invention will be set forth in part in the following description, or may be apparent from the description, or may be learned through practice of the invention.

[0009] In one exemplary embodiment, a method of operating a household appliance is provided. The household appliance includes a camera assembly. The camera assembly is operable to obtain an image. The method includes obtaining a first image from a first camera of the camera assembly and obtaining a second image from a second camera of the camera assembly. The first image is of an exterior of the household appliance, and the second image is of an interior of the household appliance. The method also includes obtaining a reading from a sensor of the household appliance. The reading is indicative of a status of a component of the household appliance. The method further includes generating a real-time interactive visual representation of the household appliance from the first image, the second image, and the reading from the sensor and displaying the real-time interactive visual representation of the household appliance on a remote user interface device. The method also includes receiving a control input at the remote user interface device and activating the household appliance in response to the control input received at the remote user interface device. Activating the household appliance includes initiating an operation cycle of the household appliance.

[0010] In another exemplary embodiment, a method of operating a household appliance is provided. The household appliance includes a camera assembly. The camera assembly is operable to obtain an image. The method includes obtaining a first image from a first camera of the camera assembly and obtaining a second image from a second camera of the camera assembly. The first image is of an exterior of the household appliance, and the second image is of an interior of the household appliance. The method further includes generating a real-time interactive visual representation of the household appliance from the first image and the second image, and displaying the real-time interactive visual representation of the household appliance on a remote user interface device. The method also includes receiving an inspection input at the remote user interface device while displaying the real-time interactive visual representation of the household appliance on the remote user interface device. The method further includes receiving a control input at the remote user interface device after the inspection input and activating the household appliance in response to the control input received at the remote user interface device. Activating the household appliance includes initiating an operation cycle of the household appliance.

[0011] These and other features, aspects and advantages of the present invention will become better understood with reference to the following description and appended claims. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.BRIEF DESCRIPTION OF THE DRAWINGS

[0012] A full and enabling disclosure of the present invention, including the best mode thereof, directed to one of ordinary skill in the art, is set forth in the specification, which makes reference to the appended figures.

[0013] FIG. 1 provides a front view of exemplary household appliances, e.g., an exemplary washing machine appliance and an exemplary dryer appliance, in accordance with one or more exemplary embodiments of the present disclosure.

[0014] FIG. 2 provides a transverse cross-sectional view of the exemplary washing machine appliance of FIG. 1.

[0015] FIG. 3 provides a perspective view of the exemplary dryer appliance of FIG. 1 with portions of a cabinet of the dryer appliance removed to reveal certain components of the dryer appliance.

[0016] FIG. 4 provides a cross-sectional view of the exemplary washing machine appliance of FIG. 2 with a camera assembly mounted on a door according to an exemplary embodiment of the present subject matter.

[0017] FIG. 5 provides a schematic view of a door and gasket sealed against a cabinet of the exemplary washing machine of FIG. 2, along with a camera mounted within the gasket according to an exemplary embodiment of the present subject matter.

[0018] FIG. 6 provides a perspective view of an exemplary household appliance, e.g., an exemplary oven appliance, according to one or more additional exemplary embodiments of the present subject matter.

[0019] FIG. 7 provides a section view of the oven appliance of FIG. 3 taken along line 7-7 of FIG. 6.

[0020] FIG. 8 provides a diagrammatic illustration of a household appliance in communication with a remote computing device and with a remote user interface device according to one or more exemplary embodiments of the present subject matter.

[0021] FIG. 9 provides a flow diagram of an exemplary method for operating a household appliance according to one or more exemplary embodiments of the present subject matter.

[0022] FIG. 10 provides a flow diagram of an additional exemplary method for operating a household appliance according to one or more exemplary embodiments of the present subject matter.

[0023] Repeat use of reference characters in the present specification and drawings is intended to represent the same or analogous features or elements of the present invention.DETAILED DESCRIPTION

[0024] Reference now will be made in detail to embodiments of the invention, one or more examples of which are illustrated in the drawings. Each example is provided by way of explanation of the invention, not limitation of the invention. In fact, it will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the scope or spirit of the invention. For instance, features illustrated or described as part of one embodiment can be used with another embodiment to yield a still further embodiment. Thus, it is intended that the present invention covers such modifications and variations as come within the scope of the appended claims and their equivalents.

[0025] As used herein, the terms “includes” and “including” are intended to be inclusive in a manner similar to the term “comprising.” Similarly, the term “or” is generally intended to be inclusive (i.e., “A or B” is intended to mean “A or B or both”). Approximating language, as used herein throughout the specification and claims, is applied to modify any quantitative representation that could permissibly vary without resulting in a change in the basic function to which it is related. Accordingly, a value modified by a term or terms, such as “about,”“approximately,” and “substantially,” are not to be limited to the precise value specified. In at least some instances, the approximating language may correspond to the precision of an instrument for measuring the value. For example, the approximating language may refer to being within a 10 percent margin.

[0026] Directional terms such as “left” and “right” are used herein with reference to the perspective of a user standing in front of a household appliance to access the appliance and / or items therein. Terms such as “inner” and “outer” refer to relative directions with respect to the interior and exterior of the appliance. For example, “inner” or “inward” refers to the direction towards the interior of the appliance. Terms such as “left,”“right,”“front,”“back,”“top,” or “bottom” are used with reference to the perspective of a user accessing the appliance. For example, a user stands in front of the appliance to open the door(s) and reaches into the appliance to add, move, or withdraw items therein.

[0027] As used herein, the terms “first,”“second,” and “third” may be used interchangeably to distinguish one component from another and are not intended to signify location or importance of the individual components. As used herein, terms of approximation, such as “generally,” or “about” include values within ten percent greater or less than the stated value. When used in the context of an angle or direction, such terms include within ten degrees greater or less than the stated angle or direction. For example, “generally vertical” includes directions within ten degrees of vertical in any direction, e.g., clockwise or counterclockwise.

[0028] Exemplary household appliances are illustrated in FIGS. 1 through 3, e.g., the household appliance may, in various embodiments, be a laundry appliance such as a washing machine appliance or a dryer appliance. In various embodiments of the present subject matter, a laundry appliance may be any suitable laundry appliance, such as a washing machine appliance, a dryer appliance, a combination washer-dryer appliance, etc. The dryer appliance 11 is separately labelled in FIG. 1 to distinguish the dryer appliance 11 from the washing machine appliance 10, where both the washing machine appliance 10 and the dryer appliance 11 are example embodiments of a household appliance 10 which may be usable in one or more exemplary methods described herein and / or may be operable and configured to perform such methods.

[0029] According to various embodiments of the present disclosure, the household appliance 10 may take the form of any of the example laundry appliances described herein, or may be any other household appliance. Thus, it will be understood that the present subject matter is not limited to any particular household appliance.

[0030] It should be understood that “household appliance” and / or “appliance” are used herein to describe appliances typically used or intended for common domestic tasks, such as a laundry appliance, e.g., as illustrated in FIGS. 1 through 3, or a cooking appliance (see, e.g., FIGS. 6 and 7) appliance, a dishwashing appliance, a refrigerator, a water heater, etc., and any other household appliance which performs similar functions in addition to network communication and data processing. Thus, devices such as a personal computer, router, and other similar devices whose primary functions are network communication and / or data processing are not considered household appliances as used herein.

[0031] As may be seen generally throughout FIGS. 1, 2, 3, 6, and 7, a user interface panel 100 and a user input device 102 may be positioned on an exterior of the household appliance. The user input device 102 is generally positioned proximate to the user interface panel 100, and in some embodiments, the user input device 102 may be positioned on the user interface panel 100.

[0032] In various embodiments, the user interface panel 100 may represent a general purpose I / O (“GPIO”) device or functional block. In some embodiments, the user interface panel 100 may include or be in operative communication with user input device 102, such as one or more of a variety of digital, analog, electrical, mechanical or electro-mechanical input devices including rotary dials, control knobs, push buttons, and touch pads. The user interface panel 100 may include a display component 104, such as a digital or analog display device designed to provide operational feedback to a user. The display component 104 may also be a touchscreen capable of receiving a user input, such that the display component 104 may also be a user input device in addition to or instead of the user input device 102.

[0033] User interface panel 100, including user input device 102 and display 104 thereon, provides a local user interface, e.g., a means for users to communicate with and operate the household appliance 10. Thus, user interface panel 100 and the local user interface provide a means for users to communicate with and operate household appliance 10. It will be appreciated that other components or devices that provide for communication with household appliance 10 for operating household appliance 10 may also be included in the user interface. For example, the user interface may include a speaker, a microphone, a camera (still or video) or motion detection camera for detecting a user's proximity to household appliance 10 or for picking up certain motions, and / or other user interface elements in various combinations.

[0034] Generally, each appliance may include a controller 210 in operative communication with the user input device 102. As used herein, “processing device” or “controller” may refer to one or more microprocessors, microcontroller, ASICS, or semiconductor devices and is not restricted necessarily to a single element. The controller 210 may be programmed to operate household appliance 10 by executing instructions stored in memory (e.g., non-transitory media). The user interface panel 100 and the user input device 102 may be in communication with the controller 210 via, for example, one or more signal lines or shared communication busses. Input / output (“I / O”) signals may be routed between controller 210 and various operational components of the appliance. Operation of the appliance can be regulated by the controller 210 that is operatively coupled to the user interface panel 100. A user interface panel 100 may for example provide selections for user manipulation of the operation of an appliance, e.g., via user input device 102 and / or display 104. In response to user manipulation of the user interface panel 100 and / or user input device 102, the controller 210 may operate various components of the appliance. Controller 210 may include a memory and one or more microprocessors, CPUs or the like, such as general or special purpose microprocessors operable to execute programming instructions or micro-control code associated with operation of the appliance. The memory may represent one or more memory elements, e.g., random access memory such as DRAM, or read only memory such as ROM or FLASH, or electrically erasable, programmable read only memory (EEPROM). In one embodiment, the processor executes programming instructions stored in memory. The memory may be a separate component from the processor or may be included onboard within the processor. Alternatively, a controller 210 may be constructed without using a microprocessor, e.g., using a combination of discrete analog and / or digital logic circuitry (such as switches, amplifiers, integrators, comparators, flip-flops, AND gates, and the like) to perform control functionality instead of relying upon software.

[0035] The controller 210 may be programmed to operate the appliance by executing instructions stored in memory. For example, the instructions may be software or any set of instructions that when executed by the processing device, cause the processing device to perform operations. Controller 210 can include one or more processor(s) and associated memory device(s) configured to perform a variety of computer-implemented functions and / or instructions (e.g. performing the methods, steps, calculations and the like and storing relevant data as disclosed herein). Controller 210 may include a network interface such that controller 210 can connect to and communicate over one or more networks with one or more network nodes. Controller 210 can also include one or more transmitting, receiving, and / or transceiving components for transmitting / receiving communications with other devices communicatively coupled with household appliance 10. Additionally or alternatively, one or more transmitting, receiving, and / or transceiving components can be located off board controller 210. It should be noted that controllers 210 as disclosed herein are capable of and may be operable to perform any methods and associated method steps as disclosed herein. For example, in some embodiments, methods disclosed herein may be embodied in programming instructions stored in the memory and executed by the controller 210.

[0036] As generally seen throughout FIGS. 1 through 3, in at least some embodiments, each laundry appliance 10 and 11 includes a cabinet 12 which defines a vertical direction V, a lateral direction L, and a transverse direction T that are mutually perpendicular. Each cabinet 12 extends between a top side 16 and a bottom side 14 along the vertical direction V. Each cabinet 12 also extends between a left side 18 and a right side 20, e.g., along the lateral direction L, and between a front side 22 and a rear side 24 along the transverse direction T.

[0037] Additional exemplary details of each laundry appliance are illustrated in FIGS. 2 and 3. For example, FIG. 2 provides a cross-sectional view of the exemplary washing machine appliance 10. As illustrated in FIG. 2, a wash tub 124 is non-rotatably mounted within cabinet 12. As may be seen in FIG. 2, the wash tub 124 defines a central axis A. In the example embodiment illustrated by FIG. 2, the central axis A may be oriented generally along or parallel to the transverse direction T of the washing machine appliance 10. Accordingly, the washing machine appliance 10 may be referred to as a horizontal axis washing machine.

[0038] Referring again to FIG. 2, a wash basket 120 is rotatably mounted within the tub 124 such that the wash basket 120 is rotatable about an axis of rotation, which generally coincides with central axis A of the tub 124. A motor 122, e.g., such as a pancake motor, is in mechanical communication with wash basket 120 to selectively rotate wash basket 120 (e.g., during an agitation or a rinse cycle of washing machine appliance 10). Wash basket 120 defines a wash chamber 126 that is configured for receipt of articles for washing. The wash tub 124 holds wash and rinse fluids for agitation in wash basket 120 within wash tub 124. As used herein, “wash fluid” may refer to water, detergent, fabric softener, bleach, or any other suitable wash additive or combination thereof. The wash basket 120 and the tub 124 may collectively define at least a portion of a tub assembly for the washing machine appliance 10.

[0039] Wash basket 120 may define one or more agitator features that extend into wash chamber 126 to assist in agitation and cleaning of articles disposed within wash chamber 126 during operation of washing machine appliance 10. For example, as illustrated in FIG. 2, a plurality of ribs 128 extends from basket 120 into wash chamber 126. In this manner, for example, ribs 128 may lift articles disposed in wash basket 120 during rotation of wash basket 120.

[0040] Referring generally to FIGS. 1 and 2, cabinet 12 also includes a front panel 130 which defines an opening 132 that permits user access to wash basket 120 within wash tub 124. More specifically, washing machine appliance 10 includes a door 134 that is positioned in front of opening 132 and is rotatably mounted to front panel 130. Door 134 is rotatable such that door 134 permits selective access to opening 132 by rotating between an open position (not shown) facilitating access to a wash tub 124 and a closed position (FIG. 1) prohibiting access to wash tub 124.

[0041] A window 136 in door 134 permits viewing of wash basket 120 when door 134 is in the closed position, e.g., during operation of washing machine appliance 10. Door 134 also includes a handle (not shown) that, e.g., a user may pull when opening and closing door 134. Further, although door 134 is illustrated as mounted to front panel 130, it should be appreciated that door 134 may be mounted to another side of cabinet 12 or any other suitable support according to alternative embodiments. The door 134 may further include a latching assembly or door lock that may be selectively actuated or engaged to lock door 134 in the closed position.

[0042] Referring again to FIG. 2, wash basket 120 also defines a plurality of perforations 140 in order to facilitate fluid communication between an interior of basket 120 and wash tub 124. A sump 142 is defined by wash tub 124 at a bottom of wash tub 124 along the vertical direction V. Thus, sump 142 is configured for receipt of and generally collects wash fluid during operation of washing machine appliance 10. For example, during operation of washing machine appliance 10, wash fluid may be urged by gravity from basket 120 to sump 142 through plurality of perforations 140.

[0043] A drain pump assembly 144 is located beneath wash tub 124 and is in fluid communication with sump 142 for periodically discharging soiled wash fluid from washing machine appliance 10. Drain pump assembly 144 may generally include a drain pump 146 which is in fluid communication with sump 142 and with an external drain (not shown). During a drain cycle, drain pump 146 urges a flow of wash fluid from sump 142 and to the external drain. More specifically, drain pump 146 includes a motor (not shown) which is energized during a drain cycle such that drain pump 146 draws wash fluid from sump 142 and urges it to the external drain.

[0044] A spout 152 is configured for directing a flow of fluid into wash tub 124. For example, spout 152 may be in fluid communication with a water supply (not shown) in order to direct fluid (e.g., clean water) into wash tub 124. Spout 152 may also be in fluid communication with the sump 142. For example, pump assembly 144 may direct wash fluid disposed in sump 142 to spout 152 in order to circulate wash fluid in wash tub 124.

[0045] As illustrated in FIG. 2, a detergent drawer 156 is slidably mounted within front panel 130. Detergent drawer 156 receives a wash additive (e.g., detergent, fabric softener, bleach, or any other suitable liquid or powder) and directs the fluid additive to wash chamber 124 during operation of washing machine appliance 10. According to the illustrated embodiment, detergent drawer 156 may also be fluidly coupled to spout 152 to facilitate the complete and accurate dispensing of wash additive.

[0046] During operation of washing machine appliance 10, e.g., during a wash cycle of the washing machine appliance 10, a laundry items are loaded into wash basket 120 through opening 132, and washing operation is initiated through operator manipulation of input selectors 102 (FIG. 1). Wash tub 124 is filled with water, detergent, and / or other fluid additives, e.g., via spout 152 and / or detergent drawer 156. One or more valves (not shown) can be controlled by washing machine appliance 10 to provide for filling wash basket 120 to the appropriate level for the amount of articles being washed and / or rinsed. By way of example for a wash mode, once wash basket 120 is properly filled with fluid, the contents of wash basket 120 can be agitated (e.g., with ribs 128) for washing of laundry items in wash basket 120.

[0047] After the agitation phase of the wash cycle is completed, wash tub 124 can be drained. Laundry articles can then be rinsed by again adding fluid to wash tub 124, depending on the particulars of the cleaning cycle selected by a user. Ribs 128 may again provide agitation within wash basket 120. One or more spin cycles may also be used. In particular, a spin cycle may be applied after the wash cycle and / or after the rinse cycle in order to wring wash fluid from the articles being washed. During a spin cycle, basket 120 is rotated at relatively high speeds. After articles disposed in wash basket 120 are cleaned and / or washed, the user can remove the articles from wash basket 120, e.g., by opening door 134 and reaching into wash basket 120 through opening 132.

[0048] While described in the context of a specific embodiment of horizontal axis washing machine appliance 10, using the teachings disclosed herein it will be understood that horizontal axis washing machine appliance 10 is provided by way of example only. It should be appreciated that the present subject matter is not limited to any particular style, model, or configuration of washing machine appliance. Other washing machine appliances having different configurations, different appearances, and / or different features may also be utilized with the present subject matter as well, e.g., vertical axis washing machine appliances.

[0049] FIG. 3 provides a perspective view of the dryer appliance 11 of FIG. 1, which is an example embodiment of a laundry appliance, and is an example embodiment of a household appliance 10, with a portion of a cabinet or housing 12 of dryer appliance 11 removed in order to show certain components of dryer appliance 11. Dryer appliance 11 generally defines a vertical direction V, a lateral direction L, and a transverse direction T, each of which is mutually perpendicular, such that an orthogonal coordinate system is defined. While described in the context of a specific embodiment of dryer appliance 11, using the teachings disclosed herein, it will be understood that dryer appliance 11 is provided by way of example only. Other dryer appliances having different appearances and different features may also be utilized with the present subject matter as well.

[0050] Cabinet 12 includes a front side 22 and a rear side 24 spaced apart from each other along the transverse direction T. Within cabinet 12, an interior volume 29 is defined. A drum or container 26 is mounted for rotation about a substantially horizontal axis within the interior volume 29. Drum 26 defines a chamber 25 for receipt of articles of clothing for tumbling and / or drying. Drum 26 extends between a front portion 37 and a back portion 38. Drum 26 also includes a back or rear wall 34, e.g., at back portion 38 of drum 26. A supply duct 41 may be mounted to rear wall 34 and receives heated air that has been heated by a heating assembly or system 40.

[0051] As used herein, the terms “clothing” or “articles” includes but need not be limited to fabrics, textiles, garments, linens, papers, or other items from which the extraction of moisture is desirable. Furthermore, the term “load” or “laundry load” refers to the combination of clothing or articles that may be washed together in a washing machine or dried together in a dryer appliance 11 (e.g., clothes dryer) and may include a mixture of different or similar articles of clothing of different or similar types and kinds of fabrics, textiles, garments and linens within a particular laundering process.

[0052] A motor 31 is provided in some embodiments to rotate drum 26 about the horizontal axis, e.g., via a pulley and a belt (not pictured). Drum 26 is generally cylindrical in shape, having an outer cylindrical wall 28 and a front flange or wall 30 that defines an opening 32 of drum 26, e.g., at front portion 37 of drum 26, for loading and unloading of articles into and out of chamber 25 of drum 26. A plurality of lifters or baffles 27 are provided within chamber 25 of drum 26 to lift articles therein and then allow such articles to tumble back to a bottom of drum 26 as drum 26 rotates. Baffles 27 may be mounted to drum 26 such that baffles 27 rotate with drum 26 during operation of dryer appliance 11.

[0053] The rear wall 34 of drum 26 may be rotatably supported within the cabinet 12 by a suitable fixed bearing. Rear wall 34 can be fixed or can be rotatable. Rear wall 34 may include, for instance, a plurality of holes that receive hot air that has been heated by heating system 40. The heating system 40 may include, e.g., a heat pump, an electric heating element, and / or a gas heating element (e.g., gas burner). Moisture laden, heated air is drawn from drum 26 by an air handler, such as blower fan 48, which generates a negative air pressure within drum 26. The moisture laden heated air passes through a duct 44 enclosing screen filter 46, which traps lint particles. As the air passes from blower fan 48, it enters a duct 50 and then is passed into heating system 40. In some embodiments, the dryer appliance 11 may be a conventional dryer appliance, e.g., the heating system 40 may be or include an electric heating element, e.g., a resistive heating element, or a gas-powered heating element, e.g., a gas burner. In other embodiments, the dryer appliance may be a condensation dryer, such as a heat pump dryer. In such embodiments, heating system 40 may be or include a heat pump including a sealed refrigerant circuit. Heated air (with a lower moisture content than was received from drum 26), exits heating system 40 and returns to drum 26 by duct 41. After the clothing articles have been dried, they are removed from the drum 26 via opening 32. A door (FIG. 1) provides for closing or accessing drum 26 through opening 32.

[0054] In some exemplary embodiments, the dryer appliance 11 may include one or more temperature sensors, such as inlet temperature sensor 43 and / or outlet temperature sensor 47. The temperature sensor(s) may be in operative communication with the controller 210. For example, in various embodiments, the controller 210 may be operable to detect, measure, and / or monitor one or more temperatures within the dryer appliance 11. Such temperatures which may be detected, measured, and / or monitored include, for example, an inlet temperature measured with the inlet temperature sensor 43 and / or an outlet temperature measured with the outlet temperature sensor 47. The temperature sensors 43 and 47 may be, in some embodiments, thermistors. As used herein, “temperature sensor” or the equivalent is intended to refer to any suitable type of temperature measuring system or device positioned at any suitable location for measuring the desired temperature. Thus, for example, temperature sensor(s) 43 and / or 47 may each be any suitable type of temperature sensor, such as a thermistor, a thermocouple, a resistance temperature detector, a semiconductor-based integrated circuit temperature sensors, etc. In addition, temperature sensor(s) 43 and / or 47 may be positioned at any suitable location and may output a signal, such as a voltage, to a controller that is proportional to and / or indicative of the temperature being measured. Although exemplary positioning of temperature sensors is described herein, it should be appreciated that dryer appliance 11 may include any other suitable number, type, and position of temperature, humidity, and / or other sensors according to alternative embodiments.

[0055] In some embodiments, one or more selector inputs 102, such as knobs, buttons, touchscreen interfaces, etc., may be provided or mounted on the cabinet 12 (e.g., on a backsplash 71) and are in operable communication (e.g., electrically coupled or coupled through a wireless network band) with the processing device or controller 210. Controller 210 may also be provided in operable communication with components of the dryer appliance 11 including motor 31, blower 48, or heating system 40. In turn, signals generated in controller 210 direct operation of motor 31, blower 48, or heating system 40 in response to the position of inputs 102.

[0056] As will be described in more detail below, household appliance 10 may further include features that are generally configured to permit a user to remotely inspect the household appliance 10. More specifically, such features may include one or more camera assemblies including external cameras such as cameras 192 (see, e.g., FIG. 1), and / or other detection devices that are used to monitor the household appliance 10 and areas outside of, e.g., in front of and around, the cabinet 12, such as an area in which a child or pet interacting with the household appliance 10 is likely to be present. In this regard, the household appliance 10 may use data from each of these devices to obtain a representation or knowledge of the identity, position, and / or other qualitative or quantitative characteristics of one or more users.

[0057] As shown schematically in FIG. 1, the household appliance may include a camera assembly 190 including external cameras 192, which may also be referred to a front cameras, each of which is generally positioned and configured for obtaining images of the household appliance 10 and adjoining areas, e.g., in front of the household appliance 10, during operation of the camera assembly 190. In some exemplary embodiments, camera assembly 190 includes one or more cameras 192, and camera assembly 190 may also include additional cameras, such as one or more additional external cameras (e.g., positioned at the left side and / or right side of the cabinet) and one or more internal cameras, such as camera 178 described below. The one or more cameras 192 may be mounted to cabinet 12 or otherwise positioned in view of an area in front of the cabinet 12. As shown in FIG. 1, a camera 192 of camera assembly 190 is mounted to user interface panel 100 at the front side 22 of cabinet 12 and is forward-facing, e.g., is oriented to have a field of vision or field of view directed towards an area in front of the cabinet 12, such as directly and immediately in front of the cabinet 12. Additionally, as noted above, features illustrated or described as part of one embodiment can be used with another embodiment to yield a still further embodiment, such as the cameras 192 illustrated in FIG. 1 may be provided with any other household appliance, such as the exemplary cooking appliance of FIGS. 6 and 7.

[0058] Referring now specifically to FIGS. 2 and 4, the camera assembly 190 may further include internal cameras, such as in embodiments where the household appliance is a washing machine appliance 10, an internal camera 178 of camera assembly 190 may be generally positioned and configured for obtaining images of contents within a cavity of the household appliance 10, e.g., a load of clothes (as identified schematically by reference numeral 172 in FIG. 2) within wash chamber 126 of washing machine appliance 10. Specifically, according to the illustrated embodiment, door 134 of washing machine appliance 10 comprises an inner window 174 that partially defines wash chamber 126 and an outer window 176 that is exposed to the ambient environment. According to the illustrated exemplary embodiment, camera assembly 190 includes a camera 178 that is mounted to inner window 174. Specifically, camera 178 is mounted such that is faces toward a bottom side of wash tub 124. In this manner, camera 178 can take images or video of an inside of wash chamber 126 and remains unobstructed by windows that may obscure or distort such images. In additional embodiments, an internal camera such as the illustrated door camera may be provided in any exemplary household appliance according to various embodiments of the present disclosure, and / or an internal camera may be provided elsewhere within the cabinet of any of the exemplary household appliances according to the present disclosure.

[0059] Referring now briefly to FIG. 5, another installation of camera assembly 190 will be described according to an exemplary embodiment of the present subject matter. Due to the similarity between this and other embodiments, like reference numerals may be used to refer to the same or similar features. According to this exemplary embodiment, camera assembly 190 includes an internal camera 178 mounted within a gasket 180 that is positioned between a front panel 130 of cabinet 12 and door 134. In some embodiments, e.g., as illustrated in FIG. 5, cameras 178 may include a lens 182 that is constructed from a clear hydrophobic material or which may otherwise be positioned behind a hydrophobic clear lens. So positioned, camera assembly 190 may obtain one or more images or videos of clothes 172 (or other object, animals, children, etc.) within wash chamber 126, as described in more detail below. Referring generally to FIGS. 2, 4, and 5, washing machine appliance 10 may further include a tub light 184 that is positioned within cabinet 12 or wash chamber 126 for selectively illuminating wash chamber 126 and any contents positioned therein. In additional embodiments, the household appliance 10 may include various internal lights, such as a drum light positioned and configured to illuminate the drum 26 of the dryer appliance 11 or an oven light positioned and configured to illuminate the cooking chamber of the oven appliance 310 (FIG. 7). Although exemplary camera assemblies 190 are illustrated and described herein, it should be appreciated that according to alternative embodiments, household appliance 10 may include any other camera or system of imaging devices for obtaining images within and around the household appliance.

[0060] In embodiments where camera assembly 190 includes a plurality of cameras, each of the plurality of cameras may have a specified monitoring zone or range positioned in and / or around household appliance 10, such as multiple cameras oriented in or facing various directions, and / or additional forward-facing cameras. In this regard, for example, the field of view of each camera may be limited to or focused on a specific area. As such, each camera may be positioned and oriented to monitor one or more areas of the household appliance 10 and adjoining areas, such as while a user is accessing or attempting to access the household appliance 10, such as in response to a request to activate the household appliance remotely, as will be discussed further below. It should be appreciated that according to alternative embodiments, camera assembly 190 may include any suitable number, type, size, and configuration of camera(s) for obtaining images of any suitable areas or regions within or around household appliance 10. In addition, it should be appreciated that each camera may include features for adjusting the field of view and / or orientation.

[0061] It should be appreciated that the images obtained by camera assembly 190 may vary in number, frequency, angle, resolution, detail, etc. in order to improve the clarity of the particular regions surrounding or within household appliance 10. In addition, according to exemplary embodiments, controller 210 may be configured for illuminating the household appliance 10 and / or surrounding areas using one or more light sources prior to obtaining images. Notably, controller 210 of household appliance 10 (or any other suitable dedicated controller) may be communicatively coupled to camera assembly 190 and may be programmed or configured for analyzing and / or processing the images obtained by camera assembly 190, e.g., in order to generate a real-time interactive visual representation of the household appliance 10, as described in more detail below.

[0062] In general, controller 210 may be operably coupled to camera assembly 190 for analyzing, manipulating, synthesizing and / or otherwise processing one or more images obtained by camera assembly 190 to extract useful information regarding objects or people within the field of view of the one or more cameras. In this regard, for example, images obtained by camera assembly 190 may be used to generate a real-time interactive visual representation of the household appliance 10. Notably, such processing may be performed locally (e.g., on controller 210) or the images may be transmitted to a remote server (e.g., in the “cloud,” as those of ordinary skill in the art will recognize as referring to a remote server or database in a distributed computing environment including at least one remote computing device) for analysis, processing, etc.

[0063] Notably, controller 210 of household appliance 10 (or any other suitable dedicated controller) may be communicatively coupled to camera assembly 190, a light such as tub light 184, and other components of household appliance 10. As explained in more detail below, controller 166 may be programmed or configured for obtaining one or images within and outside of the household appliance 10, transmitting those images to a remote device (e.g., such as a user's mobile phone 1000, e.g., as noted in FIG. 8), receiving feedback from the user's remote device (e.g., as a cycle authorization or parameter adjustment), and implementing responsive action based on that user input for improved performance and safety of operation.

[0064] In another example embodiment, the household appliance 10 may be a cooking appliance, such as an oven appliance 310, e.g., as illustrated in FIGS. 6 and 7.

[0065] FIG. 6 provides a front perspective view of an oven appliance 310 according to exemplary embodiments of the present subject matter. FIG. 7 provides a section view of exemplary oven appliance 310 taken along line 7-7 of FIG. 7. Oven appliance 310 is shown in FIGS. 6 and 7 as a free-standing range oven appliance, but it will be appreciated that oven appliance 310 is provided by way of example only and is not intended to limit the present subject matter in any aspect. Other cooking appliances having different configurations, different appearances, and / or different features may also be utilized with the present subject matter as well. Thus, the present subject matter may be used with other oven appliance configurations, e.g., wall ovens and / or oven appliances that define one or more interior cavities for the receipt of food items and / or having different pan or rack arrangements than what is shown in FIG. 7, as well as with cooktop-only appliances.

[0066] Oven appliance 310 includes an insulated cabinet 12 with an interior cooking chamber 340 defined by an interior surface 305 of cabinet 12. Cooking chamber 340 is configured for receipt of one or more food items to be cooked. Cabinet 12 extends between a bottom portion 330 and a top portion 332 along a vertical direction V. Cabinet 12 also extends between a front portion 307 and a back portion 309 along a transverse direction T and between a first side 311 and a second side 312 along a lateral direction L. Vertical direction V, lateral direction L, and transverse direction T are mutually perpendicular and form an orthogonal direction system.

[0067] Oven appliance 310 includes a door 306 rotatably mounted to cabinet 12, e.g., with a hinge (not shown). A handle 308 is mounted to door 306 and assists a user with opening and closing door 306. For example, a user can pull or push handle 308 to open or close door 306 to access cooking chamber 340. Oven appliance 310 includes a seal (not shown) between door 306 and cabinet 12 that maintains heat and cooking fumes within cooking chamber 340 when door 306 is closed as shown in FIGS. 6 and 7. Multiple parallel glass panes 322 provide for viewing the contents of cooking chamber 340 when door 306 is closed and provide insulation for cooking chamber 340. A baking rack 324 is positioned in cooking chamber 340 for receipt of food items or utensils containing food items. Baking rack 324 is slidably received onto embossed ribs or sliding rails 326 such that rack 324 may be conveniently moved into and out of cooking chamber 340 when door 306 is open.

[0068] A top heating element or broil element 342 is positioned in cooking chamber 340 of cabinet 12 proximate top portion 332 of cabinet 12. Top heating element 342 is used to heat cooking chamber 340 for both cooking / broiling and cleaning of oven appliance 310. The size and heat output of top heating element 342 can be selected based on, e.g., the size of oven appliance 310. In the exemplary embodiment shown in FIG. 7, top heating element 342 is shown as an electric resistance heating element. In additional embodiments, the top heating element 342 may be any suitable heating element, e.g., a magnetron, a gas burner, a heat lamp, and combinations of one or more of such heating elements of the same or varied types. In some embodiments, oven appliance 310 may include one or more heating elements in addition to or other than the top heating element, such as a bottom heating element, which may include an electric resistance heating element, an induction heating element, a magnetron, a gas burner, a heat lamp, and combinations of one or more of such heating elements of the same or varied types.

[0069] As shown in FIG. 6, oven appliance 310 includes a cooktop 350. Cooktop 350 is disposed on and is attached to or integral with cabinet 12. Cooktop 350 includes a top panel 352, which by way of example may be constructed of glass, ceramics, enameled steel, or combinations thereof. One or more burners 354 extend through top panel 352. A utensil (e.g., pots, pans, etc.) holding food and / or cooking liquids (e.g., oil, water, etc.) may be placed onto grates 356 disposed adjacent burners 354. Burners 354 provide thermal energy to cooking utensils placed on grates 356. Burners 354 can be any suitable type of burners, including e.g., gas, electric, electromagnetic, a combination of the foregoing, etc. It will be appreciated that the configuration of cooktop350 is provided by way of example only and that other suitable configurations are contemplated.

[0070] Oven appliance 310 includes a user interface panel 100. For this exemplary embodiment, the user input devices 102 of the user interface panel 100 include a number of knobs 102 (e.g., knobs are an embodiment of a user input device 102) that each correspond to one of the burners 354. Knobs 102 allow users to activate each burner 354 and to determine the amount of heat input provided by each burner 354 to a cooking utensil located thereon.

[0071] User interface panel 100 also includes a display component 104 that provides visual information to a user and may also allow the user to select various operational features for the operation of oven appliance 310, e.g., the display component 104 may be a touchscreen which is configured to receive user input by a touch on the screen. In some embodiments, the oven appliance 310 may include one or more touchpad buttons 102 (which are another exemplary embodiment of user input devices 102), as well as or instead of the display component 104, e.g., when the display component 104 is not provided or is not a touchscreen. One or more of a variety of electrical, mechanical or electro-mechanical input devices including rotary dials, push buttons, toggle / rocker switches, and / or touch pads can also be used singularly or in combination as user input devices 102.

[0072] The display component 104 on user interface panel 100 may present certain information to users, such as, e.g., whether a particular burner 354 is activated and / or the level at which the burner 354 is set. Display 104 can be a touch sensitive component (e.g., a touch-sensitive display screen) that is sensitive to the touch of a user input object (e.g., a finger or a stylus). Display 104 may include one or more graphical user interfaces that allow for a user to select or manipulate various operational features of oven appliance 310 or its cooktop 350.

[0073] Referring now specifically to FIG. 7, the operation of oven appliance 310 is controlled by a processing device or controller 210. As shown, controller 210 is communicatively coupled with user interface panel 100 and its user input devices 102. Controller 210 may also be communicatively coupled with various operational components of oven appliance 310 as well, such as a heating assembly, e.g., heating element 342 and / or other similar heating elements which may be provided, as discussed above, temperature sensors, cameras, speakers, and microphones, etc. Input / output (“I / O”) signals may be routed between controller 210 and the various operational components of oven appliance 10. Thus, controller 210 can selectively activate and operate these various components. Various components of oven appliance 10 are communicatively coupled with controller 210 via one or more communication lines (e.g., represented by dashed lines in FIG. 7), such as, e.g., signal lines, shared communication busses, or wirelessly. Controller 210 can be positioned in a variety of locations throughout oven appliance 310. For this embodiment, controller 210 is located proximate user interface panel 100 toward top portion 332 of oven appliance 10.

[0074] The oven appliance 310 may further include one or more temperature sensors, such as to measure temperature in or around the oven appliance 310, e.g., a temperature in the cooking chamber 340 or at the cooktop 350. For example, oven appliance 310 may include a temperature sensor 343 (FIG. 7). The temperature sensor(s) of the oven appliance 310 may be, for example, thermistors or any other suitable type of temperature sensor, such as a thermistor, a thermocouple, a resistance temperature detector, a semiconductor-based integrated circuit temperature sensors, etc., as discussed above with respect to sensor(s) 43 and 47 of the dryer appliance 11.

[0075] As noted above, the configuration of oven appliance 310 illustrated in FIGS. 6 and 7 is by way of example only, and aspects of the present disclosure may also be used with other cooking appliances, such as cooktop appliances, wall ovens, or various other oven appliances having different heating elements, such as gas burners on the cooktop and / or one or more gas heating elements in the cooking chamber, or other heating elements, as well as variations in the number or size of burners, or variations in the location, position, or type of controls on the user interface, among numerous other possible variations in the configuration of the oven appliance 310 within the scope of the present disclosure.

[0076] Turning now to FIG. 8, a general schematic is provided of a household appliance 10 (such as washing machine appliance 10, dryer appliance 11, oven appliance 310, or other suitable household appliance), which communicates wirelessly with a remote user interface device 1000 and a network 1100. For example, as illustrated in FIG. 8, the household appliance 10 may include an antenna 90 by which the household appliance 10 communicates with, e.g., sends and receives signals to and from, the remote user interface device 1000 and / or network 1100. The antenna 90 may be part of, e.g., onboard, a communications module92. The communications module 92 may be a wireless communications module operable to connect wirelessly, e.g., over the air, to one or more other devices via any suitable wireless communication protocol. For example, the communications module 92 may be a WI-FI® module, a BLUETOOTH® module, or a combination module providing both WI-FI® and BLUETOOTH® connectivity. The remote user interface device 1000 may be a laptop computer, smartphone, tablet, personal computer, wearable device, smart speaker, smart home system, and / or various other suitable devices. The communications module 92 may be onboard the controller 210 or may be a separate module.

[0077] The household appliance 10 may be in communication with the remote user interface device 1000 device through various possible communication connections and interfaces. The household appliance 10 and the remote user interface device 1000 may be matched in wireless communication, e.g., connected to the same wireless network. The household appliance 10 may communicate with the remote user interface device 1000 via short-range radio such as BLUETOOTH® or any other suitable wireless network having a layer protocol architecture. As used herein, “short-range” may include ranges less than about ten meters and up to about one hundred meters. For example, the wireless network may be adapted for short-wavelength ultra-high frequency (UHF) communications in a band between 2.4 GHz and 2.485 GHz (e.g., according to the IEEE 802.15.1 standard). In particular, BLUETOOTH® Low Energy, e.g., BLUETOOTH® Version 4.0 or higher, may advantageously provide short-range wireless communication between the household appliance 10 and the remote user interface device 1000. For example, BLUETOOTH® Low Energy may advantageously minimize the power consumed by the exemplary methods and devices described herein due to the low power networking protocol of BLUETOOTH® Low Energy.

[0078] The remote user interface device 1000 is “remote” at least in that it is spaced apart from and not physically connected to the household appliance 10, e.g., the remote user interface device 1000 is a separate, stand-alone device from the household appliance 10 which communicates with the household appliance 10 wirelessly. Any suitable device separate from the household appliance 10 that is configured to provide and / or receive communications, information, data, or commands from a user may serve as the remote user interface device 1000, such as a smartphone (e.g., as illustrated in FIG. 8), smart watch, personal computer, smart home system, or other similar device. For example, the remote user interface device 1000 may be a smartphone operable to store and run applications, also known as “apps,” and some or all of the method steps disclosed herein may be performed by a smartphone app.

[0079] The remote user interface device 1000 may include a memory for storing and retrieving programming instructions. Thus, the remote user interface device 1000 may provide a remote user interface which may be an additional user interface to the user interface panel 100. For example, the remote user interface device 1000 may be a smartphone operable to store and run applications, also known as “apps,” and the additional user interface may be provided as a smartphone app.

[0080] As mentioned above, the household appliance 10 may also be configured to communicate wirelessly with a network 1100. The network 1100 may be, e.g., a cloud-based data storage system including one or more remote computing devices such as remote databases and / or remote servers, which may be collectively referred to as “the cloud.” For example, the household appliance 10 may communicate with the cloud 1100 over the Internet, which the household appliance 10 may access via WI-FIR, such as from a WI-FI® access point in a user's home.

[0081] Exemplary methods for operating a household appliance, such as a laundry appliance or cooking appliance as described above, are provided. In this regard, for example, a controller of the household appliance, e.g., controller 210, may be configured for implementing some or all steps of one or more of the following exemplary methods. However, it should be appreciated that the exemplary methods are discussed herein only to describe exemplary aspects of the present subject matter, and are not intended to be limiting.

[0082] Now that the construction of a household appliance 10 and the configuration of controller 210 according to exemplary embodiments have been presented, exemplary methods of operating a household appliance will be described. In exemplary embodiments, the various method steps as disclosed herein may be performed by controller 210 or a separate, dedicated controller. Furthermore, some or all of the various method steps may be performed remotely, e.g., in a distributed computing environment such as the cloud, fog, or edge, wherein the controller 210 communicates with one or more remote computing devices of the distributed computing environment, such as processing, e.g., image analysis and / or process, may be performed in the cloud and the output of such process may be transmitted to and received by the household appliance 10, such as by the controller 210 thereof via the communications module 92.

[0083] FIGS. 9 and 10 depict steps performed in a particular order for purposes of illustration and discussion. Referring now generally to FIGS. 9 and 10, the methods 800 and / or 900 may be interrelated and / or may have one or more steps from one of the methods 800 and 900 combined with the other method 800 or 900. Thus, those of ordinary skill in the art will recognize that the various steps of the exemplary methods described herein may be combined in various ways to arrive at additional embodiments within the scope of the present disclosure. Those of ordinary skill in the art, using the disclosures provided herein, will understand that (except as otherwise indicated) methods 800 and 900 are not mutually exclusive. Moreover, the steps of the methods 800 and 900 can be modified, adapted, rearranged, omitted, interchanged, or expanded in various ways without deviating from the scope of the present disclosure.

[0084] An exemplary method 800 of operating a household appliance is illustrated in FIG. 9. The household appliance may include a camera assembly operable to obtain an image. As shown in FIG. 9, method 800 may include (810) obtaining a first image and a second image. The first and second images may be obtained by or from the camera assembly. The image may be an image of an exterior of the household appliance and the second image may be an image of an interior of the household appliance. For example, the first camera may be an external camera such as front camera 192 described above with reference to FIG. 1, and the second camera may be an internal camera, such as a camera positioned on or adjacent to a door of the household appliance and positioned such that an internal cavity of the household appliance is within the field of vision of the second camera. For example, the internal camera may be camera 178 described above, e.g., the household appliance may be a washing machine appliance or dryer appliance, and the camera may be positioned with a wash chamber of the washing machine appliance or a drum of the dryer appliance in the field of view of the camera. As another example, the household appliance may be an oven appliance and the internal camera may be positioned, e.g., on the door or a gasket around the door, or otherwise positioned with at least a portion the cooking cavity of the oven appliance in the field of view of the camera.

[0085] Exemplary methods according to the present disclosure may also include (820), obtaining a reading from a sensor of the household appliance. In such embodiments, the reading may be indicative of a status of a component of the household appliance. For example, the sensor may be a temperature sensor, such as one or more of the exemplary temperature sensors 43, 47, and 343 described above, which provides a reading indicative of a status of one or more heating elements of the household appliance. As another example, the sensor may be a position sensor, such as a door latch sensor (the structure and operation of which are understood by those of ordinary skill in the art and, as such, are not specifically illustrated or described herein for the sake of clarity and concision) which detects whether a door of the household appliance is in a closed position. As another example, the sensor may also be a motion sensor, such as an infrared (IR) camera. According to still other embodiments, household appliance 10 may also or instead include one or more other motion sensors (e.g., optical, acoustic, electromagnetic, etc.) that are triggered when an object or user moves into or through the area in front of and / or surrounding the household appliance 10. As such, the status of which the reading may be indicative may be proximity of the component of the household appliance to an object or user. For example, a pet or child may be playing in, on, or near the household appliance, and such may be detected by the motion sensor. In some embodiments, e.g., where the household appliance is a cooking appliance, remote operation may not be preferred when a pet or child is near the appliance.

[0086] Exemplary methods according to the present disclosure may further include generating a real-time interactive visual representation of the household appliance from the first image, the second image, and the reading from the sensor, e.g., as indicated at (830) in FIG. 9. The real-time interactive visual representation may be or may include, for example, a digital twin of the household appliance, a simulation of the household appliance, or an interactive live video feed from the cameras. The real-time interactive visual representation may be presented in a three-dimensional (3D) format (e.g., which may use special lenses or glasses to produce the perception of a 3D image via a wearable device), or may otherwise represent depth and proportion of components of the household appliance depicted therein, such as the real-time interactive visual representation may be a two-and-half dimensional (2.5D, e.g., a 3D virtual environment on a two-dimensional display which uses visual projection to simulate depth) representation of the household appliance. The real-time interactive visual representation may, in some embodiments, be provided in a virtual reality (VR) environment, or in an augmented reality (AR) environment. For example, the AR representation may include images from one or more cameras, such as a live image, along with an overlay comprising graphical and / or text elements. The overlay may, in some exemplary embodiments, be based on the reading from the sensor, such as the overlay may include or correspond to the reading from the sensor. As another example, the overlay may include a graphical representation of the sensor data, such as a color-coded graphical element which represents a temperature reading from a temperature sensor, an icon which represents motion detected near the household appliance, etc. The camera image, e.g., live image, may be composited or synthesized with the overlay to provide the AR representation of the household appliance. The image may be live in that the image is updated in real time or near real time on an associated display. Thus, it is to be understood that a “live image” as used herein is intended to include images which are continuously updated in real time or with some delay and which may be updated at least about once per second, e.g., which have a refresh rate of 1 Hz or greater.

[0087] Exemplary methods according to the present disclosure may also include (840) displaying the real-time interactive visual representation of the household appliance on a remote user interface device. The remote user interface device may be any suitable device, such as a smartphone or other device, e.g., as described above with reference to FIG. 8. As an additional example, the remote user interface device may be a VR interface device.

[0088] Method 800 may further include (850) receiving a control input at the remote user interface device and (860) activating the household appliance in response to the control input received at the remote user interface device. Activating the household appliance may include initiating an operation cycle of the household appliance. Thus, the household appliance may be activated remotely, e.g., from the remote user interface device, without requiring a local input (e.g., an input from a control or user interface device directly connected to the household such as user interface panel 100 and user input device 102) to initiate the operation cycle of the household appliance. Further, such remote operation may be safely provided in that the user has had the opportunity to inspect the household appliance and surrounding areas virtually, e.g., via the real-time interactive visual representation.

[0089] An exemplary method 900 of operating a household appliance is illustrated in FIG. 10. The household appliance may include a camera assembly operable to obtain an image. As shown in FIG. 10, method 900 may include (910) obtaining a first image from a first camera of the camera assembly and a second image from a second camera of the camera assembly. The first image may be an image of an exterior of the household appliance. The second image may be an image of an interior of the household appliance. Thus, the first camera may be an external camera, e.g., as described above with respect to FIG. 1, and the second camera may be an internal camera, e.g., as described above with respect to FIGS. 2, 4, and 5.

[0090] Method 900 may also include (920) generating a real-time interactive visual representation of the household appliance from the first image and the second image. The real-time interactive visual representation of (920) may be similar to the real-time interactive visual representation described above, e.g., may be provided in 3D or 2.5D, and may be provided in a VR environment or an AR environment, or other similar representation of the household appliance based on the first image and the second image. Also as discussed above, the first image and the second image may each be live or real-time images.

[0091] Method 900 may further include (930) displaying the real-time interactive visual representation of the household appliance on a remote user interface device and (940) receiving an inspection input at the remote user interface device while displaying the real-time interactive visual representation of the household appliance on the remote user interface device. The inspection input may, for example, ensure or verify that the user has adequately viewed the appliance and surrounding areas. Thus, the inspection input may include, for example, panning and or zooming the real-time interactive visual representation such that the user has viewed a sufficient area of the appliance and surrounding areas, such as all of or a majority of an internal cavity (e.g., tub or drum of a laundry appliance or cooking chamber of an oven appliance) and / or an entire front side, three sides, or a full perimeter of the exterior of the household appliance. The inspection input may also or instead include a confirmation input manually provided by the user to indicate that the user has satisfactorily inspected the household appliance and surrounding areas via the real-time interactive visual representation in order for the user to deem the household appliance safe for remote operation.

[0092] Method 900 may also include remote operation of the household appliance, e.g., after the inspection input has been received, such as unlocking the household appliance for remote activation in response to the inspection input. For example, method 900 may include (950) receiving a control input at the remote user interface device after the inspection input and (960) activating the household appliance in response to the control input received at the remote user interface device, where activating the household appliance may include initiating an operation cycle of the household appliance.

[0093] Activating the appliance, e.g., in (860) of method 800 and / or in (960) of method 900, includes causing at least one mechanical component of the appliance to be operated. For example, the mechanical component may be a motor, such as the motor 122 of the washing machine appliance or the motor 31 of the dryer appliance, a fan, a heating element such as heating element 342 (FIG. 7) or a heating element of heating system 40 (FIG. 3), a pump, a compressor, or a valve, among other possible example mechanical components of a household appliance. Also, operating the mechanical component includes changing a physical status of the component, e.g., a speed, position, etc. of the component, such as accelerating the motor, fan, etc., e.g., from a zero starting speed, opening a valve, and / or other changes in the physical state of one or more mechanical components of the household appliance. Thus it is to be understood that the household appliance may also include a mechanical component, and methods according to the present disclosure may further include activating the mechanical component. For example, the mechanical component may be activated remotely after determining, e.g., using the real-time interactive visual representation, that the household appliance is in a safe condition. For example, such determination may include determining that no undesired articles are in a cavity of the household appliance and / or that no unauthorized users (e.g., children, elderly persons, or pets) are in, on, or near the household appliance.

[0094] As noted above, the methods 800 and / or 900 may be interrelated and / or may have one or more steps from one of the methods 800 and 900 combined with the other method 800 or 900. Thus, for example, method 800 may also include receiving an inspection input at the remote user interface device while displaying the real-time interactive visual representation of the household appliance on the remote user interface device prior to receiving the control input at the remote user interface device, e.g., similar to (940) described above. As another example, method 900 may further include obtaining a reading from a sensor of the household appliance, the reading may be indicative of a status of a component of the household appliance, and the real-time interactive visual representation of the household appliance may be further generated from the reading from the sensor, e.g., similar to (820) and (830) described above.

[0095] In some embodiments, methods according to the present subject matter, such as method 800 and / or 900, may include activating the household appliance in response to the control input received at the remote user interface device without a direct physical input at a local user interface of the household appliance, e.g., the inspection and operation of the household appliance may be entirely remote.

[0096] In some exemplary methods, e.g., method 800 and / or 900, initiating the operation cycle of the household appliance may include locking a door of the household appliance. In additional exemplary embodiments, initiating the operation cycle of the household appliance may also or instead include activating a heating element of the household appliance.

[0097] Exemplary methods according to the present subject matter, including methods 800 and / or 900, may further include detecting motion with a motion detector of the household appliance. In such embodiments, the real-time interactive visual representation of the household appliance may include a visual indicator of the detected motion. For example, in some embodiments, the motion detector may be the sensor or one of the sensors from which the reading is obtained at (820), and in other embodiments, the detected motion may be in addition to the reading obtained from the sensor.

[0098] In exemplary methods which include obtaining a reading from a sensor, such as (820) in method 800, the sensor may be a temperature sensor and the reading from the temperature sensor may be indicative of a status of a heating element of the household appliance. In some embodiments, the sensor may be a door switch and the reading from the door switch may be indicative of a position of a door of the household appliance. In additional embodiments, the sensor may be a temperature sensor and the reading from the temperature sensor may be indicative of a temperature at a motor of the household appliance.

[0099] In some embodiments, the household appliance may be a laundry appliance. In such embodiments, the image from the second camera of the interior of the laundry appliance may be of a tub (e.g., when the laundry appliance is a washing machine appliance) or drum (e.g., when the laundry appliance is a dryer appliance) of the laundry appliance. In some embodiments, the household appliance may be an oven appliance. In such embodiments, the image from the second camera of the interior of the oven appliance may be of a cooking chamber of the oven appliance.

[0100] This written description uses examples to disclose the invention, including the best mode, and also to enable any person skilled in the art to practice the invention, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the invention is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they include structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insubstantial differences from the literal languages of the claims.

Claims

1. A method of operating a household appliance, the household appliance comprising a camera assembly, the camera assembly operable to obtain an image, the method comprising:obtaining a first image from a first camera of the camera assembly, the first image of an exterior of the household appliance;obtaining a second image from a second camera of the camera assembly, the second image of an interior of the household appliance;obtaining a reading from a sensor of the household appliance, the reading indicative of a status of a component of the household appliance;generating a real-time interactive visual representation of the household appliance from the first image, the second image, and the reading from the sensor;displaying the real-time interactive visual representation of the household appliance on a remote user interface device;receiving a control input at the remote user interface device; andactivating the household appliance in response to the control input received at the remote user interface device, wherein activating the household appliance comprises initiating an operation cycle of the household appliance.

2. The method of claim 1, wherein the household appliance is activated in response to the control input received at the remote user interface device without a direct physical input at a local user interface of the household appliance.

3. The method of claim 1, wherein initiating the operation cycle of the household appliance comprises locking a door of the household appliance.

4. The method of claim 1, wherein initiating the operation cycle of the household appliance comprises activating a heating element of the household appliance.

5. The method of claim 1, further comprising detecting motion with a motion detector of the household appliance, wherein the real-time interactive visual representation of the household appliance comprises a visual indicator of the detected motion.

6. The method of claim 1, wherein the sensor is a temperature sensor and the reading from the temperature sensor is indicative of a status of a heating element of the household appliance.

7. The method of claim 1, wherein the sensor is a door switch and the reading from the door switch is indicative of a position of a door of the household appliance.

8. The method of claim 1, wherein the sensor is a temperature sensor and the reading from the temperature sensor is indicative of a temperature at a motor of the household appliance.

9. The method of claim 1, wherein the household appliance is a laundry appliance, wherein the image from the second camera of the interior of the laundry appliance is of a tub or drum of the laundry appliance.

10. The method of claim 1, wherein the household appliance is an oven appliance, wherein the image from the second camera of the interior of the oven appliance is of a cooking chamber of the oven appliance.

11. The method of claim 1, further comprising receiving an inspection input at the remote user interface device while displaying the real-time interactive visual representation of the household appliance on the remote user interface device prior to receiving the control input at the remote user interface device.

12. A method of operating a household appliance, the household appliance comprising a camera assembly, the camera assembly operable to obtain an image, the method comprising:obtaining a first image from a first camera of the camera assembly, the first image of an exterior of the household appliance;obtaining a second image from a second camera of the camera assembly, the second image of an interior of the household appliance;generating a real-time interactive visual representation of the household appliance from the first image and the second image;displaying the real-time interactive visual representation of the household appliance on a remote user interface device;receiving an inspection input at the remote user interface device while displaying the real-time interactive visual representation of the household appliance on the remote user interface device;receiving a control input at the remote user interface device after the inspection input; andactivating the household appliance in response to the control input received at the remote user interface device, wherein activating the household appliance comprises initiating an operation cycle of the household appliance.

13. The method of claim 12, wherein the household appliance is activated in response to the control input received at the remote user interface device without a direct physical input at a local user interface of the household appliance.

14. The method of claim 12, further comprising detecting motion with a motion detector of the household appliance, wherein the real-time interactive visual representation of the household appliance comprises a visual indicator of the detected motion.

15. The method of claim 12, further comprising obtaining a reading from a sensor of the household appliance, the reading indicative of a status of a component of the household appliance, wherein the real-time interactive visual representation of the household appliance is further generated from the reading from the sensor.

16. The method of claim 15, wherein the sensor is a temperature sensor and the reading from the temperature sensor is indicative of a status of a heating element of the household appliance.

17. The method of claim 15, wherein the sensor is a door switch and the reading from the door switch is indicative of a position of a door of the household appliance.

18. The method of claim 15, wherein the sensor is a temperature sensor and the reading from the temperature sensor is indicative of a temperature at a motor of the household appliance.

19. The method of claim 12, wherein the household appliance is a laundry appliance, wherein the image from the second camera of the interior of the laundry appliance is of a tub or drum of the laundry appliance.

20. The method of claim 12, wherein the household appliance is an oven appliance, wherein the image from the second camera of the interior of the oven appliance is of a cooking chamber of the oven appliance.

Images