METHOD FOR MONITORING THE LOAD STATE OF A DISHWASHER AND DISHWASHER

Description

[0001] The invention relates to a method for monitoring a loading state of a rinsing device and to a rinsing device.

[0002] Artificial intelligence (AI) devices can be integrated into household appliances to increase their usefulness and ease of use. Furthermore, various sensors, such as camera systems, can be built into dishwashers to obtain information about the current load and its soiling.

[0003] DE 102 04 455 A1 describes a method for influencing and controlling a program sequence of a dishwasher. A microphone signal is recorded and evaluated, and this is used to modify the program sequence or to detect defects in machine components. In particular, it is disclosed that the microphone signal can be used to determine whether a dishwasher basket is full or only slightly loaded.

[0004] US 2021 / 0093150 A1 discloses a dishwasher with a camera for recording a camera image of the dishes arranged in a dishwashing basket, which image is evaluated by means of a neural network.

[0005] DE 10 2005 061 806 A1 discloses a dishwasher with a sensor assigned to the circulation pump for detecting foam formation.

[0006] The object of the invention is to provide an improved method for monitoring a loading state of a rinsing device and an improved rinsing device.

[0007] According to the invention, this object is achieved by a method for monitoring the load status of a dishwasher and a dishwasher having the features of the main claims. Advantageous embodiments and further developments of the invention are set forth in the following subclaims.

[0008] The advantages achieved with the invention include optimizing the loading of dish racks and increasing the operational reliability of the dishwashing appliance. For example, the approach described is suitable for detecting overturned dishes or for detecting an accumulation of incorrectly loaded dishes that carry dishwater.

[0009] A method for monitoring a loading state of an interior of a dishwasher is presented, the method comprising the following steps: Reading in at least one microphone signal and at least one pump signal, wherein the microphone signal represents a noise pattern in the interior detected by a microphone unit and the pump signal represents a pumping state of a pump of the flushing device detected by a pump sensor, processing the microphone signal and additionally or alternatively the pump signal using an artificial neural network to classify a current loading state of the flushing device as correct or faulty, and, if the loading state is classified as incorrect, providing a loading signal representing the faulty loading state, and outputting a control signal to a device interface and additionally or alternatively an information signal to a user interface to a user in response to the loading signal, wherein the control signal represents a change in a device setting for adapting the flushing process to the current loading state,and wherein the information signal represents information about the current loading condition. ,

[0010] The method described above for monitoring a loading state of a dishwashing appliance is in particular part of a method for cleaning dishes in which dishes are washed in the dishwashing appliance.

[0011] The loading state can be represented in particular by one or more of the following characteristics of the items to be washed: quantity of items to be washed, weight of the items to be washed or individual items to be washed, type of items to be washed or individual items to be washed (e.g. plate, cup, pot, pan), type of material of the items to be washed or individual items to be washed, positioning of the items to be washed or individual items to be washed in a washing basket or the washing baskets, alignment, i.e. orientation of individual items to be washed.

[0012] The washing appliance could, for example, be a dishwasher, which may have two or more baskets inside for storing items such as dishes. Despite standard recommendations for loading dish baskets, which can also be referred to as dish racks, the loading pattern in practice can often be rather heterogeneous. On the one hand, the spectrum of bowls, cups, pots and pans is very broad, which can result in a very household-specific assortment. On the other hand, loading the dish baskets can usually be done quite intuitively, especially when it comes to larger items of tableware such as pots and bowls. This can lead to the problem that larger items of tableware can be placed in the wrong basket (larger bowls should ideally be in the lower basket), in a suboptimal position in the basket, or even upside down in the dish basket.In addition, with light plastic bowls, dishes and cups there is a risk that they could be knocked over by the water pressure from the spray jets during washing, and that washing water could then collect in a hollow space, making it impossible for the water circuit to access the dishwasher. This can lead to unstable pump operation and reduced cleaning and drying performance. The pump could also be damaged if it delivers too little water. Furthermore, items of crockery that do not belong in the dishwasher, such as wooden spatulas or aluminum dishes, can be placed in the basket. Furthermore, items of crockery, such as coffee cups, can tip over when the basket is pushed in if they were previously placed in an unstable state.With the method presented here, an operator of the dishwashing machine can advantageously be provided with monitored load management, particularly supported by today's digitalization possibilities, in order to optimize the loading of the baskets and increase the operational reliability of the machine.

[0013] For this purpose, the dishwasher can, for example, comprise a plurality of different sensors or a sensor array that can supply information to an artificial intelligence or a neural network. The neural network, in turn, can be connected to a device control system that can, for example, control machine actuators and a dispensing system, i.e., an interface to a user. The dispensing system can, for example, be an acoustic signal generator in the dishwasher, the device's display, or a smartphone with a corresponding app. With the method presented here, the loading situation of the dishwasher is recorded by combining several sensor signals that support the neural network. Based on this, corrective messages and warnings can be given to the user.

[0014] For this purpose, the AI ​​or the neural network can be trained to determine how items of crockery should not be positioned in the basket or how they should not be loaded. Using the information about the number of incorrectly loaded items, the neural network offers the operator a notification via the dispensing system, allowing them to correct the cup position or to start the program with the announcement of a later automatic wash parameter adjustment, for example, an additional water change or an increase in the water quantity. If an object changes position during a wash cycle, such as a bowl falling over, information can be provided to the user, and additionally or alternatively, wash parameters can be adjusted to the changed load condition.

[0015] According to one embodiment, in the reading step or in a repeated reading step, at least one camera signal can be read in, which represents an image of the interior captured by a camera unit. In the processing step, the camera signal can be processed to provide the loading signal. For example, a camera can be located both inside the dishwasher and outside the appliance; for example, it can be the camera in a smartphone or a camera installed in the kitchen, for example under the ceiling. The loading situation of the dishwasher baskets can advantageously be recorded using the camera unit or optionally with multiple cameras. With conventional image processing, for example, the opening side and additionally or alternatively the support side of critical items of crockery, such as cups and bowls, can be captured by at least one camera.Additionally, the shape and orientation of the handles can be used as a criterion for cups. For example, the handles may be wider at the cup opening than at the base, which, along with the other criteria, can provide reliable information about the orientation of this item of crockery in the basket. Furthermore, some cups may have a manufacturer's stamp on the base, which can also be captured by the cameras and used as a location criterion. If the stamp is visible, then the cup was loaded the right way up.

[0016] In the next reading step, the camera signal can be read in if the current load status is classified as faulty using the microphone signal and / or the pump signal. This allows incorrectly loaded dishes, for example, to be initially detected using a pump pressure sensor and a microphone. Only secondarily can a further downstream evaluation be performed using image analysis. The image analysis can be triggered by previously detecting, for example, overturned dishes or an accumulation of wash liquor in incorrectly loaded dishes using the pump pressure sensor and the microphone.

[0017] According to a further embodiment, at least one weight signal can be read in the reading step, which represents the weight of a dish rack located in the interior, as detected by a load sensor. In the processing step, the weight signal can be processed to provide the load signal.

[0018] For example, the load sensor or multiple load sensors can be designed as strain gauges or acceleration sensors. Using the load sensor, the neural network can advantageously receive additional information about the weight of the baskets and thus also about the weight of the basket load. The weight of the empty baskets and that of a standard and intensive load can be known in advance or can be easily determined. If the measured weight deviates from empirical values, this could indicate, for example, an incomplete load, a load containing predominantly plastic tableware, or a load of pots or pans. In conjunction with the image information from the camera unit, the neural network can verify the plausibility of the loading situation and, if necessary, provide the user with recommendations for correction.For example, if the load sensors in the upper basket detect an excessive weight and the neural network detects pots or pans in the upper basket using the learned camera images, the load status can be classified as incorrect, and the display unit can indicate that the heavy items should be placed in the lower basket. Using the weight signal, the most realistic and comprehensive picture of the load status can be created.

[0019] According to the invention, the signal from one or more of the sensors is read in and processed by an artificial neural network to classify a current load condition as correct or incorrect. In this case, the strength and / or profile, particularly with regard to uniformity, fluctuations, and periodicity, of at least the signal from the microphone and the pump sensor, which can be designed as a pump pressure sensor, speed sensor, or armature current sensor, is processed. An incorrect classification can be defined, for example, by the current load condition not conforming to a predetermined standard or expectation. In particular, a load condition can be classified as incorrect if one or more characteristics of the washware do not lie within predetermined or predeterminable limits or a predetermined or predeterminable selection. Otherwise, the load condition can be classified as correct.

[0020] A loading condition that is classified as faulty is one from the following, incomplete list: at least one item of a certain type of dishware is arranged in an unsuitable location, at least one item of dishware is arranged in an incorrect orientation (e.g. has tipped over), at least one item of dishware does not correspond to a type suitable for washing in the dishwashing appliance or a type known to the system.

[0021] To improve the accuracy of classification, especially the classification of load states into "correct" or "incorrect," thus reducing the risk of incorrect classification and thus an incorrect load signal, at least both the microphone and pump sensor signals are read in and processed together. Only by considering both signals and evaluating their correlation can certain load states be detected with sufficient accuracy and classified as incorrect. To further increase the accuracy of the classification, the signals from other sensors, such as a camera signal and / or a weight signal, can be processed.

[0022] According to a further embodiment, the method can comprise a step of modifying the neural network using a confirmation signal read in via the user interface, wherein the confirmation signal requests a reclassification of the current load state. In practice, for example, it can happen that rather unusual items of crockery, in particular atypical items newly purchased by the user or items to be washed outside the dishwashing context, are placed in the dishwasher and are not yet recognized by the neural network. Such items of crockery can be identified by the neural network as incorrectly placed or as unknown, and the load state can be classified as faulty accordingly. Using the information signal, relevant information can be provided. The user can react to this by actively confirming that the load state can indeed be classified as correct.This can be done, for example, with a smartphone program at the end of the wash cycle. Using the confirmation signal, the neural network's learning can be proactively triggered, and the designated object can be added to a database, for example, to prevent future false alarms.

[0023] According to a further embodiment, in the step of processing the pump signal, it can be compared with the camera signal if the pump signal represents irregular pump pressure in order to detect incorrectly arranged objects in the interior. The pump sensor can be designed, for example, as a speed sensor, pump pressure sensor, or armature current sensor and can monitor stable pump operation during dishwashing. Additionally or alternatively, the switching behavior of the heating pressure switch can also be evaluated. If the user starts the program after loading the baskets with the need for later program adjustment, i.e., if, for example, a warning about incorrectly loaded dishes is ignored, dishwashing liquid can collect in the volumes of the cups or bowls, which can lead to instability in the pump pressure.The pump pressure, the pump speed or even the armature current of the pump motor can fluctuate in such a case. If all the dishes are loaded correctly, the pump pressure, the speed and the armature current can be relatively stable. If dishwashing liquid collects in the cavities of the dishes, the pump can pump irregularly, which can lead to fluctuations in the signal. When such a fluctuating pump signal is read in, the neural network can, for example, compare the pump signal with the camera signal in order to compare the pump behavior with the load situation derived from the camera images. This can determine, for example, whether the unstable pump pressure was triggered by an object that was loaded the wrong way round, in which water now collects, and which is then missing from the pump's water circuit.If such or a similar condition is detected by comparing the pump signal with the camera signal, the amount of water to be refilled, which would lead to stable pump behavior again, can be advantageously determined by the neural network based on the detected number of incorrectly loaded dishes and passed on to the appliance control system, for example, using the control signal. The water quantity can then advantageously be adjusted.

[0024] According to a further embodiment, the pump can be stopped in the outputting step using the control signal. For example, the pump can be stopped or its speed reduced if the read-in pump signal exhibits inconsistencies or fluctuations. The pump can then be stopped, or at least its speed reduced, until the cause of the problem has been found, for example, by comparing it with other sensor signals and remedied using the control signal, or the user has been informed using the information signal, for example, to change a setting. Advantageously, this allows a pump malfunction to be detected early and damage to the flushing device to be avoided.

[0025] According to a further embodiment, the pump can be stopped during the dispensing step if the microphone signal represents an acoustic deviation from the sound pattern. During the dishwashing process, the neural network can, for example, continuously access the microphone unit. The microphone can, for example, be arranged in the interior of the dishwashing appliance in a waterproof or at least water-protected manner and can be designed additionally or alternatively. A characteristic sound pattern can arise during the dishwashing operation of the machine. The uniform movement of the spray arms can, for example, cause periodically recurring signals. If, for example, dishes are tipped or knocked over during the dishwashing operation due to the pressure of the spray jets, this can manifest itself in an acoustic deviation from the characteristic uniform sound profile, for example in a deviation in the form of a so-called signal peak.In order to obtain a reference, the pump can be stopped in such a case to allow the neural network to record camera information triggered by the microphone signal. This has the advantage that the spray jets do not interfere with the image recording. The neural network can, for example, compare the camera image with stored reference profiles and images of the loaded basket before the start of the dishwashing program and can signal the error using the information signal. Since it can be expected that dishwashing liquor can collect in an overturned bowl or cup, a water change can also be initiated in order to return the nominal amount of water to the dishwashing process. This can also dilute the liquid in the container. In addition to the image information, the neural network can, for example, compare the microphone signal with the pump signal.If the signal fluctuation is evident in both the microphone signal and the pump signal, the overturned item of crockery is likely a bowl. To ensure plausibility, the neural network can cross-reference the camera signal with the signals from the other sensors. Since a bowl only slowly fills with dishwashing liquid, the peak in the microphone and pump signals would not occur simultaneously, but rather with a time lag. The hydraulic signal image can therefore be generated later, since the container is only just filling. The neural network, in conjunction with the images of the loading situation, can advantageously determine the amount of water to be refilled based on the length of the delay. Additionally or alternatively, the user can also be informed of the incorrect loading status here.If the user does not respond to the request within a specified time, the program adjustment can be carried out as described.

[0026] According to a further embodiment, the method can comprise a step of temporarily switching on the pump in order to detect a sustained acoustic deviation from the noise pattern. For example, an acoustic deviation from the noise pattern can occur if one or more spray arms are or become blocked, or if one or more spray arms, which can still rotate, touch or slide along dishes that have slipped through the dishwasher basket. So that the neural network can perform an image comparison or make a decision using the camera unit in response to the microphone signal and unaffected by the spray jets, the circulation pump can be switched off, for example. This means that the washing operation can be interrupted. The circulation pump can then be enabled again, for example for at least 5 seconds, before being switched off again afterwards.If the position of one or more spray arms has not changed, a spray arm blockage has occurred, and information about the malfunction can be communicated to the user using the information signal. This can advantageously prevent damage to the dishwasher or objects located inside.

[0027] According to a further embodiment, the processing step can be performed again in response to the dispensing step in order to reclassify the current loading state of the dishwashing appliance as correct or faulty. For example, as a result of the dispensing step, the loading state may have been changed by user intervention or by a change in a device setting. Accordingly, the processing step can be performed again to check the changed circumstances and reclassify. Additionally or alternatively, a loading state classified as faulty by the neural network may also have been confirmed as correct by the user, and the neural network may have modified itself accordingly to consequently classify the loading state as correct. Advantageously, the current loading state can thereby be continuously and optimally monitored.

[0028] According to a further embodiment, the method can comprise a step of providing the camera signal using the camera unit, the microphone signal using the microphone unit, and the pump signal using the pump sensor. For example, the neural network can be connected to the camera unit, the microphone unit, and the pump sensor, as well as additionally or alternatively to other sensors, in a signal-transmitting manner, and can be configured, for example, to control them. This can advantageously facilitate signal transmission and optimize monitoring of the load status of the rinsing device.

[0029] The approach presented here further provides a device designed to perform, control, or implement the steps of a variant of a method presented here in corresponding devices. This embodiment of the invention in the form of a device also allows the problem underlying the invention to be solved quickly and efficiently.

[0030] The device can be designed to read in input signals and to determine and provide output signals using the input signals. An input signal can, for example, represent a sensor signal that can be read in via an input interface of the device. An output signal can represent a control signal or a data signal that can be provided at an output interface of the device. The device can be designed to determine the output signals using a processing rule implemented in hardware or software. For example, the device can comprise a logic circuit, an integrated circuit, or a software module and can, for example, be implemented as a discrete component or be comprised of a discrete component.

[0031] In addition, a flushing device with a variant of the previously presented device is presented. This combination has the advantage of optimally implementing all of the aforementioned advantages.

[0032] Also advantageous is a computer program product or computer program with program code that can be stored on a machine-readable carrier or storage medium such as a semiconductor memory, a hard disk memory, or an optical memory. If the program product or program is executed on a computer or device, the program product or program can be used to carry out, implement, and / or control the steps of the method according to one of the embodiments described above.

[0033] An embodiment of the invention is shown purely schematically in the drawings and is described in more detail below. It shows Figure 1 shows a flowchart of a method for monitoring a loading state of an interior of a rinsing appliance according to an embodiment; Figure 2 shows a schematic representation of an embodiment of a rinsing appliance with a device; Figure 3 shows a flowchart of a method for monitoring a loading state of an interior of a rinsing appliance according to an embodiment; Figure 4 shows a schematic representation of a device according to an embodiment; Figure 5 shows a schematic representation of an embodiment of a rinsing appliance with a device; Figure 6 shows a flowchart of a method for monitoring a loading state of an interior of a rinsing appliance according to an embodiment; Figure 7 shows a schematic representation of a pump signal according to an embodiment; and Figure 8 shows a schematic representation of a microphone signal according to an embodiment.

[0034] Figure 1shows a flowchart of a method 100 for monitoring a loading state of an interior of a washing appliance according to an exemplary embodiment. The method 100 comprises a step 105 of reading in a microphone signal and additionally or alternatively at least one pump signal. The microphone signal represents a noise pattern in the interior detected by a microphone unit, and the pump signal represents a pumping state of a pump of the washing appliance detected by a pump sensor. Optionally, in this step or a subsequently executed step 105 of reading in, a camera signal can also be read in, which can represent an image of the interior detected by a camera unit.

[0035] In a subsequent processing step 110, in this exemplary embodiment, the microphone signal and the pump signal, as well as optionally the camera signal, are processed using an artificial neural network to classify a current load state of the dishwasher as correct or faulty. If the load state is classified as incorrect, a load signal representing the faulty load state is provided. For example, such a load signal indicates that a fallen or incorrectly loaded item has been detected, which could, for example, lead to an accumulation of wash liquor.

[0036] According to one embodiment, the camera signal is read in triggered by an evaluation of the microphone signal and / or the pump signal and used in step 110 to provide the loading signal and / or to reclassify the loading state.

[0037] The method 100 further comprises a step 115 of outputting a control signal to a device interface in response to the load signal. In this exemplary embodiment, in this step 115, an information signal is additionally output to a user interface. The control signal represents a change in a device setting for adapting the rinsing process to the current load state, and the information signal represents information about the current load state.

[0038] Figure 2shows a schematic representation of an embodiment of a dishwasher 200 with a device 205 for monitoring a loading state of an interior 210 of the dishwasher 200. In this embodiment, the dishwasher 200 is designed as a dishwasher and comprises, merely as an example, three washing baskets 211, 212, 213 for holding dishes, wherein the washing baskets 211, 212, 213 in this embodiment are designed as a lower basket 211 in the lower region of the interior 210, an upper basket 212 in the middle region of the interior 210 and as a cutlery drawer 213 in the upper region of the interior 210. Each washing basket 211, 212, 213 can be supplied with washing liquor by means of a spray arm 215, 216, 217 in order to wash dishes arranged in the washing baskets 211, 212, 213. For this purpose, the spray arms 215, 216, 217 are connected to a pump 220 for pumping water.In this exemplary embodiment, the pump 220 is designed as a circulation pump and is connected to a water diverter 222 and to a pump sensor 225. In this exemplary embodiment, the pump sensor 225 is designed as a pump pressure sensor and monitors stable pump operation during a flushing operation. In another exemplary embodiment, the pump sensor can also be designed, for example, as a speed sensor or armature current sensor. In addition or alternatively to monitoring the pump operation, the switching behavior of a heating pressure switch can also be evaluated. In this exemplary embodiment, the pump sensor 225 is also designed to detect a pump state of the pump 220 and to provide the pump state to the device 205 using a pump signal 230.

[0039] In this exemplary embodiment, the device 205 comprises an artificial neural network 232 and is designed to carry out a method for monitoring the load state, as described in the previous figure. For this purpose, the neural network 232 in this exemplary embodiment is trained to determine how items of crockery may not be placed or placed in the basket. Furthermore, the neural network 232 is trained, merely by way of example, to recognize reflective or shiny metallic items of crockery, such as stainless steel pots. One identifying feature is, merely by way of example, the shiny surface, and the pot handles or the characteristic handle of a saucepan. Furthermore, the size of these items of crockery represents an identifying feature, as does the shape in general.Furthermore, in this embodiment, the neural network 232 learns to recognize wooden tableware, such as spatulas and serving boards. While wooden tableware may in principle be cleaned in the dishwasher, washing with detergent leads to leaching of the surface. Both spatulas and serving boards have a characteristic shape, which serves as a recognition feature for the neural network 232 in this embodiment. The learning process can be implemented as supervised learning.During supervised learning, patterns or image patterns of possible basket loads or individual items of crockery, created in the laboratory, for example, are passed to the neural network 232, in particular including additional annotation information for the patterns, from which the loading state of a dishwashing basket 211, 212, 213, i.e., the shape or form of the items of crockery and also their orientation in the basket, can be determined or predicted with a high degree of accuracy. In this exemplary embodiment, this is done solely by way of example using the size, color, characteristic shape, and surface texture or surface gloss. The neural network 232 is designed to develop a prediction about the item of crockery in question from the combination of all features.Furthermore, the position of the crockery item can be identified only by learning the opening, the orientation, and the shape of cup handles, or even by printing on the bottom. For this purpose, the neural network 232 is configured to read the pump signal 230, as well as a camera signal 240 provided by a camera unit 235 and a microphone signal 250 provided by a microphone unit 245.

[0040] The camera unit 235, which can also be referred to as a camera, is arranged on an appliance door 260 in this exemplary embodiment and is designed to visually image the interior 210 and provide it to the device 205 using the camera signal 240. In another exemplary embodiment, the camera unit can also be arranged outside the appliance, for example on a smartphone or under a kitchen ceiling. The camera unit 235 can record a loading situation or a loading state of the washing baskets 211, 212, 213, which can also be referred to as dishwasher baskets. With conventional image processing, the opening side or the support side of critical tableware items, such as cups and bowls, can be recognized by at least one camera. Optionally, the shape and orientation of the handle can be used as a criterion for cups.This is possible if, for example, the cup handles are wider at the opening than at the base, which, along with the other criteria, can provide reliable information about the orientation of this piece of crockery in the basket. Furthermore, many cups have a manufacturer's stamp on the base, which can also be recognized by the camera and used as a positioning criterion. If the stamp is visible, then the cup has been loaded the right way up.

[0041] In this exemplary embodiment, the microphone unit 245 is arranged in the interior 210 and is designed to be watertight merely by way of example. In another exemplary embodiment, the microphone unit, which can also be referred to as a microphone, can be designed to be at least water-protected and additionally or alternatively can be arranged to be watertight or water-protected. During the washing operation of the machine, a characteristic noise pattern results. The uniform movement of the spray arms 215, 216, 217 causes periodically recurring signals merely by way of example. If a piece of crockery tips over or is knocked over during the washing operation, the resulting noise can be detected by the microphone unit 245 as an acoustic deviation from the noise pattern and can be provided to the device 205 or to the neural network 232 using the microphone signal 250.

[0042] The device 205 is designed to read in the pump signal 225, the camera signal 240, and the microphone signal 250 and to process them using the artificial neural network 232 in order to classify a current loading state of the rinsing device 200 as correct or incorrect. In the case of an incorrectly classified loading state, the device 205 in this exemplary embodiment is designed, on the one hand, to output a control signal 270 to a device interface 275 in order to effect a change in a device setting and thus adapt a rinsing process to the current loading state. Furthermore, the device 205 in this exemplary embodiment is designed, in addition to the control signal 270, to output an information signal 280 to a user interface 285 to a user, wherein the information signal 280 represents information about the current loading state.For example only, the user interface 285 is embodied as a display on the appliance door 260 of the dishwasher 200. In another embodiment, the user interface can also be arranged, for example, on a smartphone or tablet with a corresponding app. By providing information about, for example, the number of incorrectly placed cups, the neural network 232 offers the user, via a notification via the dispensing system, the option of correcting the cup position or starting the program with the announcement of a later automatic adjustment of the washing parameters.

[0043] Figure 3 shows a flowchart of a method 100 for monitoring a loading state of an interior of a dishwasher according to an embodiment. The method 100 shown here corresponds to or is similar to the method described in the preceding Figure 1described method. In this exemplary embodiment, in step 105 of the reading process, at least one weight signal is read in in addition to the camera signal, the microphone signal, and the pump signal. The weight signal merely represents, by way of example, a weight of a dish rack arranged in the interior of the dishwasher, as detected by a load sensor. In this exemplary embodiment, the weight signal is processed in step 110 of the processing process to provide the load signal. Since the weight of the empty racks and that of a standard load and also that of an intensive load are known or stored in the neural network in this exemplary embodiment, the weight of the rack load is also determined using the weight signal. If the measured weight deviates from empirical values, this could be due to, for example, an incomplete load, a load containing predominantly plastic crockery, or a load of pots or pans.In conjunction with the image information from the camera unit, the neural network in this embodiment checks the plausibility of the loading situation and, in step 115 of outputting, issues correction recommendations to the user using the information signal.

[0044] Furthermore, in this exemplary embodiment, the method 100 comprises an additional modification step 300. In this step 300, the neural network is modified using a confirmation signal read in via the user interface, wherein the confirmation signal requests a reclassification of the current load state. In other words, the neural network learns from the user's request. For example, in practice, it may happen that, for example, unusual items of crockery, particularly atypical items newly purchased by the user, or items outside the context of the crockery, are placed in the dishwasher and are not yet recognized by the neural network.In such a situation, the user provides the confirmation signal via the user interface, which is merely exemplary as part of a smartphone program, thereby confirming that the dishes identified as faulty should be recorded as the correct load. Using the confirmation signal, the neural network is modified accordingly.

[0045] Figure 4 shows a schematic representation of a device 205 according to an embodiment. The device 205 shown here corresponds to or is similar to the device shown in the previous Figure 2 described devices, with the difference that the device 205 in this embodiment comprises both the neural network 232 and a control device 400 for controlling device settings as separate units.

[0046] The neural network 232 is designed to read sensor signals, which are merely examples of a camera signal 240, a microphone signal 250, a pump signal 230 and a weight signal 405. The camera signal 240 represents an image of an interior of a rinsing device captured by a camera unit, as described in the previous Figure 2described. Likewise, the microphone signal 250 represents a noise pattern in the interior detected by a microphone unit, the pump signal represents a pumping state of a pump of the dishwasher detected by a pump sensor, and the weight signal, in this exemplary embodiment, represents a weight of a dish rack arranged in the interior, in particular loaded with dishes, detected by a load sensor. The neural network 232 is designed to process the read-in signals 230, 240, 250, 405 in order to classify a current loading state of the dishwasher as correct or incorrect, and, in the event of an incorrectly classified loading state, to provide a loading signal 410 representing the incorrect loading state. In this exemplary embodiment, the loading signal 410 is provided to the control device 400 of the device 205.The control device 400 is configured, merely by way of example, to output a control signal 270 to the device interface 275, which can also be referred to as a machine actuator, in response to the loading signal 410. Furthermore, in this exemplary embodiment, the information signal 280 can also be output to the user interface 285 by the control device 400.

[0047] Figure 5 shows a schematic representation of an embodiment of a washing device 200 with a device 205 for monitoring a loading state of an interior 210 of the washing device 200. The washing device 200 shown here corresponds to or is similar to the one shown in the previous Figure 2 described rinsing device and the device 205 shown here corresponds or is similar to the one shown in the previous Figures 2 and 4 described device.

[0048] In contrast to the previous Figure 2In this exemplary embodiment, the rinsing device 200 comprises an additional camera unit 500, which is arranged on a side of the interior 210 opposite the camera unit 235. The additional camera unit 500 is designed merely as an example to capture the interior 210 like the camera unit 235 and to provide an image of the interior to the device 205.

[0049] In this exemplary embodiment, the dishwasher 200 also has a load sensor 505, 506, 507 on each washing basket 211, 212, 213. The load sensors 505, 506, 507 are designed merely as an example to detect a weight of the respective washing basket 211, 212, 213 and to provide it to the device 205 using a weight signal 405. The load sensors 505, 506, 507 are designed as strain gauges for this purpose only as an example. In another exemplary embodiment, a load sensor can also be designed as an acceleration sensor, for example. The device 205 is designed in this exemplary embodiment to read in the weight signal 405 and process it in order to Figures 2 and 4The loading signal described above is to be provided. The weight of the empty dish racks 211, 212, 213 as well as the weight of a standard and intensive load are known. The device is thus configured to determine the weight of a current rack load 510. If the measured weight deviates from empirical values, this indicates an incomplete load, a load containing predominantly plastic crockery, or a load of pots or pans. The loading situation can be verified in conjunction with the image information from the cameras. If an incorrect load is detected, a corresponding correction message can be output via the user interface 285.

[0050] If the current loading state is classified as incorrect even though a user classifies it as correct, the neural network 232 of the device 205 is additionally configured in this exemplary embodiment to be modified. For this purpose, a confirmation signal 515 can be read in via the user interface 285, with the confirmation signal 515 requesting a reclassification of the current loading state. Using the confirmation signal 515, the current loading state can be learned as correct, and the neural network 232 can be modified accordingly.

[0051] Figure 6 shows a flowchart of a method 100 for monitoring a loading state of an interior of a dishwasher according to an embodiment.

[0052] The method 100 presented here corresponds to or is similar to the methods described in the previous Figures 1and described methods, with the difference that the method presented here has 100 additional steps. Thus, in this exemplary embodiment, before step 105 of reading in, a step 600 of providing the camera signal using the camera unit and the microphone signal using the microphone unit and the pump signal using the pump sensor takes place. Only as a result of step 600 of providing are the signals read in and processed in this exemplary embodiment. In this case, the pump is stopped in step 115 of outputting, merely by way of example, if the microphone signal represents an acoustic deviation from the noise pattern.

[0053] During the dishwashing process, in this embodiment, the neural network accesses the dishwasher's microphone. A characteristic sound pattern is produced during the dishwashing operation of the machine. The uniform movement of the spray arms causes periodically recurring signals. If dishes are tipped or knocked over by the pressure of the spray jets during the dishwashing operation, this is indicated by an acoustic deviation in the form of a so-called signal peak from the characteristic uniform sound profile. To obtain a reference, the pump is stopped while the camera information is recorded, triggered by the microphone signal. This has the particular advantage that the spray jets do not interfere with the image recording. The neural network compares these with stored reference profiles and images of the loaded basket before the start of the dishwashing program and signals the error via the user interface.Since it's expected that dishwashing liquid will collect in an overturned bowl or cup, this embodiment initiates a water change to restore the nominal water volume to the dishwashing process. This also dilutes the liquid in the container.

[0054] Accordingly, in this exemplary embodiment, processing step 110 is performed again in response to dispensing step 115 in order to compare the camera signal with the microphone signal and to again classify the current loading state of the dishwasher as correct or incorrect. In other words, in this exemplary embodiment, the neural network compares the microphone signal with that of the pump sensor in addition to the image information. If the signal fluctuation is evident in both the microphone signal and the pump signal, the knocked-over item of crockery is likely to be a bowl. To ensure plausibility, the neural network cross-references the camera images with the signals from the other sensors. Since the volume of a bowl only slowly fills with dishwashing liquid, the peak in the microphone and pump signals will not occur simultaneously, but rather at different times.The hydraulic signal follows later, as the vessel's volume must first fill. In this exemplary embodiment, the neural network uses the duration of the delay, in conjunction with the images of the loading situation, to determine the amount of water to be refilled. Here, too, the user is informed using the information signal. If the user does not respond to the request within a specified time, the program adjustment is carried out as described.

[0055] A similar acoustic pattern occurs when one or more spray arms are or become blocked, or when one or more spray arms, which can still rotate, touch or slide along dishes that have slipped through the basket. To enable the camera or neural network to compare images and make a decision based on the microphone signal without being influenced by the spray jets, the circulation pump is switched off, i.e. the dishwashing operation is interrupted. In the following step 605 of temporary switching on, the pump is released again, for example, for at least 5 seconds, before being switched off again. If the position of one or more spray arms has not changed, a spray arm blockage has occurred, and information about the faulty operating state is communicated to the user.

[0056] Figure 7shows a schematic representation of a pump signal 230 according to an embodiment. The pump signal 230 shown here corresponds to or is similar to the pump signal 230 shown in the previous Figures 2 , 4 and 5 described pump signal. The pump signal 230 is shown as a diagram in the figure shown here, with the time profile shown on the x-axis and the profile of the pump signal 230 shown on the y-axis.

[0057] The pump signal 230 is from a pump sensor as described in the previous Figure 5described above. The pump sensor monitors stable pump operation during dishwashing. Alternatively, the switching behavior of the heating pressure switch can also be evaluated. If the user starts the program after loading the dish racks with the need for later program adjustment (warnings about incorrectly loaded dishes are ignored), dishwashing liquid will collect in the volumes of the cups or bowls, leading to instability in the pump pressure. The pump pressure, the pump speed, or even the armature current of the pump motor fluctuates. If all dishes are loaded correctly, the pump pressure, the speed, and the armature current will be relatively stable.

[0058] This can be represented in a constant partial signal 700. If, however, rinsing solution accumulates in vessel cavities, the pump pumps irregularly, which leads to fluctuations 705 in the signal. The amount of water to be refilled, which would lead to stable pump behavior again, is only an example of a neural network, as described in the previous Figures 2 , 4 and 5 As described above, the amount of water can be set based on the number of incorrectly loaded dishes detected. The water quantity can then be adjusted. In particular, the neural network compares the pump behavior with the load situation derived from the camera images.

[0059] Figure 8 shows a schematic representation of a microphone signal 250 according to an embodiment. The microphone signal 250 shown here corresponds to or is similar to the one shown in the previous Figures 2 , 4 and 5described microphone signal. The microphone signal 250 is shown as a diagram in the figure shown here, with the time course on the x-axis and the course of the microphone signal 250 on the y-axis.

[0060] During the rinsing process, a neural network of a rinsing device, as described in the previous Figures 2 , 4 and 5 As described above, the microphone of the dishwasher is responsible for this. During the dishwasher's rinsing operation, a characteristic noise pattern is produced. The uniform movement of the spray arms causes periodically recurring partial signals 800.

[0061] If items of crockery are tipped over or knocked over during the dishwashing operation due to the pressure of the spray jets, this is reflected in an acoustic deviation 805 in the form of a so-called signal peak from the characteristic uniform noise profile.

Claims

1. Method (100) for monitoring a loading state of an interior (210) of a washing appliance (200), wherein the method (100) comprises the following steps: reading in (105) at least one microphone signal (250) and at least one pump signal (230), wherein the microphone signal (250) represents a sound image in the interior (210) detected by a microphone unit (245) and the pump signal (230) represents a pumping state of a pump (220) of the washing appliance (200) detected by a pump sensor (225), wherein in the step (105) of reading in or in a repeated step (105) of reading in, at least one camera signal (240) is read in, which represents an image of the interior (210) captured by a camera unit (235), and / or in the step (105) of reading in, at least one weight signal (405) is read in, which represents a weight of a washing rack arranged in the interior (210) captured by a load sensor (506); processing (110) the microphone signal (250) and / or the pump signal (230) by means of an artificial neural network (232) in order to classify a current loading state of the washing appliance (200) as correct or faulty, and in the case of a loading state classified as faulty, providing a loading signal (410) representing the faulty loading state, wherein a faulty loading condition is present if at least one piece of washware of a certain type is arranged in an unsuitable location, at least one piece of washware of a certain type is arranged in an incorrect orientation or at least one piece of washware of a certain type is not of a type suitable for washing in the washing appliance or of a type known to the system, wherein in the step (110) of processing, the camera signal (240) and / or the weight signal (405) is additionally processed to provide the loading signal (410); and outputting (115) a control signal (270) to an appliance interface (275) and / or an information signal (280) to a user interface (285) to a user in response to the loading signal (410), wherein the control signal (270) represents a change in an appliance setting for adapting the washing process to the current loading state, and wherein the information signal (280) represents information about the current loading state.

2. Method (100) according to claim 1, wherein in the repeated step (105) of reading in, the camera signal (240) is read in if the current loading state is classified as faulty using the microphone signal (250) and / or the pump signal (230).

3. Method (100) according to either of the preceding claims, comprising a step (300) of modifying the neural network (232) using a confirmation signal (515) read in via the user interface (285), wherein the confirmation signal (515) requests a reclassification of the current loading state.

4. Method (100) according to any of claims 2 to 3, wherein in the step (110) of processing, the pump signal (230) is compared with the camera signal (240) if the pump signal (230) represents an irregular pump pressure in order to detect incorrectly arranged objects in the interior (210).

5. Method (100) according to any of the preceding claims, wherein in the step (115) of outputting, the pump (220) is stopped using the control signal (270).

6. Method (100) according to claim 5, wherein in the step (115) of outputting, the pump (220) is caused to stop when the microphone signal (250) represents an acoustic deviation (805) from the sound image.

7. Method (100) according to either claim 5 or claim 6, comprising a step (605) of temporarily switching on the pump (220) in order to detect a sustained acoustic deviation from the sound image.

8. Method (100) according to any of the preceding claims, wherein the step (110) of processing is carried out again in response to the step (115) of outputting in order to reclassify the current loading state of the washing appliance (200) as correct or faulty.

9. Method (100) according to any of the claims, comprising a step (600) of providing the camera signal (240) using the camera unit (235) and the microphone signal (250) using the microphone unit (245) and the pump signal (230) using the pump sensor (225).

10. Washing appliance (200) comprising a pump sensor and a microphone, wherein the washing appliance is designed to carry out the steps (105, 110, 115) of the method (100) in corresponding units according to any of the preceding claims, wherein the washing appliance additionally comprises a camera unit (235) and / or a load sensor (506).

11. Computer program product comprising program code for carrying out the method (100) according to any of the preceding claims 1 to 9 when the computer program product is executed on a washing appliance according to claim 10.