Method for processing data of a domestic appliance

Abstract

The invention is based on a method for processing data of a domestic appliance (10), wherein a processing unit (12) internal to the domestic appliance (10) processes an image of an interior (16) of the domestic appliance (10) so as to produce an image which can be output to an operator. In order to increase efficiency and flexibility, it is proposed that the image which can be output to the operator is further processed depending on a connection (18) between the internal processing unit (12) and a processing unit (14) external to the domestic appliance (10), said further processing being carried out by the external processing unit (14).

Description

[0001] Method for processing data of a household appliance

[0002] The invention relates to a method for processing data of a household appliance according to the preamble of claim 1, a household appliance device according to claim 8, a household appliance according to claim 9, an external processing unit according to claim 10, a household appliance system according to claim 11, a computer program according to claim 12 and a computer-readable storage medium according to claim 13.

[0003] Camera modules for a refrigerator are already known from the prior art. These modules have a chip for image processing, whereby the camera images are processed, in particular solely based on predefined settings, which are suitable for output to an operator only after further backend processing, which, for example, carries out a perspective transformation. For backend processing and output to the operator, an upload of the image to the backend is necessary, which means that the system has only limited flexibility. Furthermore, the predefined settings cannot be changed or improved after production of the camera module. Such an embodiment is known from EP 3783291 A1 and WO 2015024841 A1.

[0004] The object of the invention is, in particular but not limited to, to provide a generic method, a household appliance device, a household appliance, an external processing unit, a household appliance system, a computer program, and a computer-readable storage medium with improved properties with regard to efficiency and flexibility. This object is achieved according to the invention by the features of claims 1, 8, 9, 10, 11, 12, and 13, while advantageous embodiments and further developments of the invention can be found in the subclaims.

[0005] The invention is based on a method for processing data from a household appliance, wherein a processing unit internal to the household appliance processes at least one image of an interior of the household appliance into an image that can be output by the user. It is proposed that the image that can be output by the user be further processed by the external processing unit, in particular in a further processing step, depending on a connection between the internal processing unit and a processing unit external to the household appliance.

[0006] Such a configuration makes it possible to achieve a particularly efficient method in which an image that can be output by the operator is already provided by the internal processing unit, and in particular can be provided by processing with advantageously low computational effort and / or by particularly cost-effective components. In particular, an image that can be output by the operator can also be provided in the absence of a connection between the household appliance and the external processing unit, in particular a backend. Furthermore, the image that can be output by the operator can advantageously be further processed and in particular improved and / or analyzed, wherein the external processing unit can provide increased computing power for the further processing of the image that can be output by the operator, in particular compared to processing by the internal processing unit.In particular, resource distribution with regard to computational effort for image processing can advantageously be divided between the internal and external processing units. Flexibility can advantageously be increased by providing an image that is as optimized as possible with regard to the available computing power of the respective processing unit, both for the presence and absence of a connection between the external processing unit and the internal processing unit. In particular, a particularly advantageous division of individual processing steps into a method section running in the internal processing unit, which in particular comprises basic processing steps, and a further method section running on the external processing unit, which in particular comprises advanced processing steps, can be achieved.Furthermore, further processing using the external processing unit can particularly effectively improve unfavorable conditions and / or complex features, such as dark lighting conditions and / or text, in the image.

[0007] The household appliance is preferably designed as a household refrigeration appliance. A household appliance designed as a household refrigeration appliance is particularly advantageously provided for cooling refrigerated goods, in particular foodstuffs such as beverages, meat, fish, milk and / or dairy products, in at least one operating state, in particular in order to ensure a longer shelf life for the refrigerated goods. The household appliance designed as a household refrigeration appliance can in particular be a freezer and advantageously a refrigerator and / or freezer. The household appliance could, for example, alternatively be designed as a household cleaning appliance, such as a washing machine and / or a dishwasher and / or a dryer, and / or as a household cooking appliance, such as an oven and / or a microwave.Alternatively, another design of the household appliance that appears to be sensible to a person skilled in the art, other than a household appliance known to the person skilled in the art, which preferably has an interior space, is conceivable.

[0008] Preferably, a household appliance system carries out the above-mentioned method. The household appliance system preferably comprises a household appliance device and an external processing unit, in particular the above-mentioned external processing unit. The household appliance system can comprise the household appliance. The household appliance preferably comprises the household appliance device. The household appliance device is preferably at least a part, in particular a subassembly, of the household appliance, in particular the household refrigeration appliance. In particular, the household appliance device can also comprise the entire household appliance, in particular the entire household refrigeration appliance. Preferably, the household appliance device is integrated into the household appliance, in particular permanently installed.Alternatively, the household appliance device can be designed as a household appliance accessory, in particular separate from the household appliance, and intended to be arranged at least partially in and / or on the household appliance, preferably detachably. For example, the household appliance device could be designed as a module that can be fixed, in particular by an installer and / or an operator, on and / or in the household appliance, for example as a camera module. The household appliance device in particular has the internal processing unit. Preferably, the internal processing unit is part of the household appliance and / or is at least intended to be arranged, preferably detachably, in and / or on the household appliance. Preferably, the internal processing unit is integrated into the household appliance, in particular permanently installed.The household appliance device and / or the internal processing unit are / are preferably fixed in and / or to a remainder of the household appliance and / or the household appliance in every operating state. In this context, "detachable" should be understood in particular as "non-destructively separable". Preferably, the household appliance device and / or the internal processing unit are / are wired to the household appliance and / or the remainder of the household appliance and, in particular, are supplied with at least data and preferably energy via the wired connection. Alternatively, it would be conceivable for the household appliance device and / or the internal processing unit to have their own energy storage device for an energy supply and / or to have a wireless data connection to the household appliance and / or the remainder of the household appliance.The household appliance device, in particular the internal processing unit, is provided in particular for executing part of the above-mentioned method. The internal processing unit preferably has at least one computing unit for image processing and in particular for partially executing the above-mentioned method. The internal processing unit, in particular at least the computing unit, preferably has at least one microcontroller. The internal processing unit, in particular at least the computing unit, can have only one microcontroller. The internal processing unit can provide computing power that is too low for executing the further processing on the internal processing unit. The processing unit preferably has a memory unit, in particular for storing at least one method algorithm and / or at least one processing parameter.Preferably, the internal processing unit is designed at least as part of an existing electrical and / or electronic unit of a conventional household appliance. For example, the internal processing unit, in particular at least the computing unit, can be designed as a processor, in particular as a microcontroller, of the household appliance at least for executing a household function of the household appliance, for example for controlling a cooling circuit. Alternatively, it is conceivable that the internal processing unit, in particular the computing unit, is provided only for executing part of the above-mentioned method and, in particular, is arranged and / or designed internally in and / or on the household appliance at least partially in addition to a conventional electrical and / or electronic unit, in particular at least one processor, of a household appliance.The household appliance and / or the household appliance device preferably has / has a communication unit for providing the connection between the internal processing unit and the external processing unit. The communication unit preferably transmits the operator-outputtable image to the external processing unit depending on the connection between the internal processing unit and the external processing unit. Preferably, the operator-outputtable image is transmitted to the external processing unit, in particular by means of the communication unit, for the presence of the connection between the internal processing unit and the external processing unit, and is preferably further processed, in particular into a further-processed image.If the connection between the internal processing unit and the external processing unit is lost, further processing of the operator-outputtable image is suspended. The term "loss of connection" refers, in particular, to a connection with a data transmission rate below a threshold value, which is assessed as insufficient for data transmission within a desired time interval. The communication unit is preferably wired and / or wirelessly connected to the internal processing unit, at least for data purposes.The internal processing unit can be connected to the external processing unit via the communication unit, preferably wirelessly, for example by means of WiFi, Bluetooth, ZigBee, a radio connection, for example mobile radio, another wireless data communication technology deemed appropriate by a person skilled in the art, a combination of these, or the like. Alternatively, it would be conceivable for the internal processing unit to be connected to the external processing unit via the communication unit in a wired manner, for example by means of a LAN or the like. It is conceivable for the internal processing unit to be connected to the external processing unit, in particular for data purposes, directly, or via at least one connecting element, for example a router. The connecting element can be part of the household appliance system.The communication unit can be designed, in particular, as a WiFi, Bluetooth, ZigBee, or LAN interface and / or as another interface for wireless and / or wired communication that appears appropriate to a person skilled in the art. The external processing unit is provided for executing at least part of the above-mentioned method. The external processing unit is designed separately from the household appliance and, in particular, is arranged and / or can at least be arranged separately from the household appliance. The external processing unit is connected, in particular, to the internal processing unit and / or the household appliance via data transmission and preferably wirelessly. The external processing unit has, in particular, increased available computing power compared to the internal processing unit. The external processing unit is preferably part of the backend.Preferably, the external processing unit is embodied as a cloud unit. The external processing unit, in particular, provides cloud computing. Alternatively, it would be conceivable for the external processing unit to be embodied as a smart device, for example, a tablet, a smartphone, a gateway, a computer, or the like.

[0009] The internal processing unit preferably processes the image of the interior, in particular in an internal processing step and / or in an internal processing step, to produce the operator-outputtable image. The “operator-outputtable image” is to be understood in particular as an image which is intended to be output to the operator and in particular is processed for this purpose. The operator-outputtable image outputs at least a shape of the interior and / or an overview of the interior of the household appliance, in particular in a way that is already understandable for the operator. The operator-outputtable image is output to the operator in at least one operating state. The operator-outputtable image is preferably output to an output unit external to the household appliance, in particular to the operator, depending on the connection between the internal processing unit and the external processing unit.The operator-outputtable image is output to the external output unit, particularly in an external output step, preferably if the connection between the internal processing unit and the external processing unit is absent. Preferably, the image further processed by the external processing unit, particularly if the connection between the internal processing unit and the external processing unit is present, is output to the external output unit, particularly in a further external output step. Depending on the connection between the internal processing unit and the external processing unit, preferably either the operator-outputtable image or the further processed image is output to the operator.Alternatively, it would be conceivable that the operator-outputable image and the further-processed image are output to the external output unit for the presence of the connection between the internal processing unit and the external processing unit.

[0010] Preferably, the output unit external to the household appliance is configured separately from the household appliance and, in particular, is arranged and / or at least arrangeable separately from the household appliance. The external output unit is preferably wirelessly connectable to the internal processing unit and / or the household appliance and / or the external processing unit for data purposes, in particular only for data purposes. Preferably, the external output unit is only wirelessly connectable to the internal processing unit and / or the household appliance and / or the external processing unit. For example, the internal processing unit and / or the external processing unit communicate with the external output unit via Bluetooth, WiFi, ZigBee, and / or the like.The connection between the external processing unit and / or the internal processing unit and the external output unit can be implemented directly or via at least one connecting element, for example the router. The external output unit is provided in particular for a visual display and preferably has at least one display. The external output unit is preferably designed as a computer or as a smart device, in particular as a smartphone or as a tablet or the like. The external output unit in particular has at least one computer program, in particular an app, for displaying the operator-outputtable image and / or the further processed image. For example, the external output unit can have an app for networking and / or controlling more than one household appliance, for example a smart home app.

[0011] The household appliance preferably has an internal output unit which, in particular at least in an assembled state of the household appliance, is firmly and preferably non-detachably connected to the rest of the household appliance without the need for tools. The internal display unit preferably has a data and / or an energy, in particular wired, connection to at least part of the rest of the household appliance. The internal processing unit and the internal output unit are preferably wired to one another, at least in terms of data. The internal output unit is preferably arranged and / or embedded in and / or on a housing of the household appliance. The internal output unit is preferably embedded in a door of the household appliance, in particular of the household refrigeration appliance. The operator-outputtable image is preferably output at the internal output unit, in particular in an internal output step.It would be conceivable for the image further processed by the external processing unit to be output at the internal output unit, particularly when a connection is present between the internal processing unit and the external processing unit. However, the operator-outputtable image is preferably always output at the internal output unit, particularly regardless of a connection between the internal processing unit and the external processing unit and / or an operating state. Alternatively, it is conceivable for the household appliance to be designed without an internal output unit.

[0012] Preferably, a camera unit with at least one camera sensor, in particular a camera unit internal to the household appliance, records the at least one image of the interior of the household appliance and transmits the at least one image, in particular to the internal processing unit. It is conceivable for the internal processing unit to process the at least one image of the interior into just one image that can be output by the operator. Alternatively, it is conceivable for the internal processing unit to process the at least one, in particular more than one, image of the interior into at least one, in particular more than one, image that can be output by the operator. For example, the internal processing unit can process each image of the interior into a respective image that can be output by the operator. The camera unit is preferably part of the household appliance and / or is arranged and / or can be arranged at least partially in and / or on the household appliance, preferably detachably.The camera unit can be part of the household appliance. The camera unit is preferably integrated into the household appliance, in particular permanently installed. Alternatively, it would be conceivable for the internal processing unit and / or the household appliance device to be part of a camera module that can be fixed to the household appliance and in particular detachable from the household appliance. The camera unit is preferably arranged on and / or in the interior of the household appliance. The at least one image of the interior of the household appliance preferably shows and / or describes an image, in particular of at least a part, of an interior of a body of the household appliance and / or at least one interior of the household appliance, in particular of the body of the household appliance, and / or an interior of at least one door of the household appliance.The at least one camera sensor can be arranged and in particular fastened on and / or at least partially in an inner side of a ceiling, a door, a side wall and / or a floor of the household appliance and / or can be arranged and in particular fastened. The camera unit can have exactly two, exactly three or exactly four camera sensors. Preferably, in particular by means of the at least one camera sensor, at least one image of the interior of the household appliance is recorded and in particular processed into at least one operator-outputable image which shows and / or describes the interior of the body of the household appliance and the inside of the at least one door of the household appliance. The camera unit is preferably connected by wire to the internal processing unit, in particular at least for data purposes, at least for transmitting the image of the interior.It is conceivable that the camera unit is triggered periodically and / or in response to a user input and / or door movement of the household appliance and / or the like. The camera sensor preferably provides only one color value per pixel according to a Bayer pattern.

[0013] "Intended" should be understood as specifically programmed, designed, and / or equipped. The fact that an object is intended for a specific function should be understood as meaning that the object fulfills and / or performs this specific function in at least one application and / or operating state.

[0014] It is further proposed that the further processing, in particular of the external processing unit, comprise noise reduction, whereby the image outputtable by the operator can advantageously be reworked into a qualitatively improved image, in particular with an appearance appealing to the operator. In particular, poor lighting conditions in the interior of the household appliance can advantageously be compensated for. The poor lighting conditions are caused in particular by a very low amount of light available in the interior as a result of a required short exposure time when triggering the at least one camera sensor during door movement and / or by design aspects, in particular intentionally dimmed interior lighting of the household appliance and / or a dark color selection for an interior design of the household appliance.The poor lighting conditions in the interior can, in particular, lead to a very noisy image from the camera sensor. Advantageously, flexibility for at least one design aspect of the household appliance and / or ease of use can be increased, in particular through an appealing and / or brand-typical appearance of the interior of the household appliance and / or an image output to the operator. In particular, a dark interior design of the household appliance can be enabled and / or dazzling of the operator by a bright interior design can be advantageously avoided. Preferably, the noise reduction takes place by means of upsampling, wherein in particular a pixel resolution is increased during further processing of the operator-outputtable image to form a further-processed image. In particular, in addition to noise reduction, the image can be sharpened during further processing, in particular automatically and / or simultaneously.Advantageously, noise in the operator-outputable image, which is generated and / or amplified in particular by existing standard image processing processes, such as contrast enhancement and / or color correction, in particular to brighten the image of the interior, can be compensated and / or eliminated or reduced by the noise reduction.

[0015] Furthermore, it is proposed that the further processing, in particular of the external processing unit, be based at least partially on a KI algorithm, in particular a neural network. The abbreviation "KI" here refers to the term "artificial intelligence." In particular, at least the noise reduction is based at least partially on the KI algorithm, in particular the neural network. Advantageously, image quality, in particular image sharpness, and / or a quantity of image information in the further processed image can be increased or retained.Standard noise reductions particularly comprise blurring, preferably through a combination of Gaussian filters over an environment of immediately adjacent pixels of a pixel, with sharpening, whereby high-frequency image information, in particular brightness and / or color differences between the pixels, is reduced during standard noise reduction and, in particular, real image information, such as sharp text edges, can be lost, which can be advantageously avoided or reduced by the Kl algorithm. Advantageously, a general reduction of a high image frequency can be avoided or reduced during further processing, in particular during noise reduction, whereby the further processed image appears particularly sharper for visual perception. Preferably, the further processing is based at least partially and / or entirely on the automated neural network.The further processing, in particular the KL algorithm and / or the noise reduction, preferably comprises at least one GPU processing and / or is designed as such. Preferably, at least part of the further processing, in particular the noise reduction, is based on, in particular real, image information. The real image information is in particular image information that corresponds to a real situation in the interior and differs in particular from artificially generated image information, such as noise, generated during processing and / or by a limitation of the camera sensor. For the further processing, in particular the noise reduction, preferably real, image information from an image area around a pixel is used, wherein the image area is in particular larger than the immediate surroundings of the pixel.Preferably, a further processed image region, in particular at least for noise reduction, is replaced by a partial image generated by the Kl algorithm. The Kl algorithm preferably recognizes a piece of image information, in particular genuine image information, and / or a type of image region, for example, a texture of an apple and / or a text element and / or the like, and can in particular replace and / or improve this image information, for example, by inserting a texture and / or a text element with a reduced noise component. Preferably, the further processing, in particular the noise reduction and / or the Kl algorithm, comprises object and / or text recognition.

[0016] A neural network is generally a model inspired by the workings of the human brain. It is a type of artificial intelligence used to recognize complex patterns in data and make predictions or decisions. A neural network consists of a large number of artificial neurons connected to each other. These neurons are organized into layers called input, hidden, and output layers. The input layer takes in the input data, the hidden layers process the data, and the output layer outputs the results. The connections between the neurons are assigned weights that are adjusted during the training process. Training is usually done by presenting sample data, from which the network learns the correct results.Through repeated training, the neural network can recognize patterns in the data and make predictions. Noise reduction in images with AI can be based on the use of neural networks. Neural networks and machine learning are increasingly being used for noise reduction. These models learn from a set of noisy and clean image pairs to reduce the noise in new images. Examples include the deep image prior or denoising neural networks (DNNs). To train a noise reduction model, large sets of images are collected, containing both noise and their corresponding clean versions. These images serve as training data for the neural network. The neural network is trained with the collected images to learn the relationship between the noisy input image and the clean output image.Training is achieved by adjusting the weights and parameters of the network to achieve the best possible noise reduction. Once the neural network is trained, it can be applied to new images to reduce the noise. The noisy input image is fed into the network, and the network returns a cleaned image as output. During training, the neural network recognizes patterns and structures in the noise and learns to reduce them to improve the image. It can detect different types of noise, such as Gaussian noise, salt-and-pepper noise, or Poisson noise. It is important to note that the performance of the noise reduction model depends heavily on the quality and variety of the training data. The more different types of noise are considered during training, the better the model can respond to different noise patterns.

[0017] It is also conceivable that, for example, "Faster R-CNN" could be used for further processing. "R-CNN" stands for "Region-based Convolutional Neural Network." The Faster R-CNN algorithm consists of two main components: the Region Proposal Network (RPN) and the actual Convolutional Neural Network (CNN). The RPN is responsible for identifying potential regions in an image that may contain an object. It generates proposals for the regions by using various anchor boxes and evaluates them based on their intersection with objects in the image. The CNN is then used to classify and localize the extracted regions. It extracts features from the proposed regions and uses them to detect and classify objects. The Faster R-CNN algorithm has proven to be very effective and powerful and has been used in many image processing and object detection applications.It has the ability to precisely detect and locate a wide variety of objects in images.

[0018] It is also conceivable that YOLO (You Only Look Once) could be used for further processing, for example. The YOLO algorithm is characterized by its efficiency and speed. Unlike other object detection algorithms that use region-based approaches, YOLO divides the input image into a grid and performs only one forward pass of the neural network to simultaneously detect objects in all grid cells. YOLO uses a convolutional neural network divided into multiple layers to extract features from the image and detect objects. The network outputs not only the detected objects but also the bounding box coordinates that enclose the object and the probability that the object is present in the bounding box. By using a single forward pass, YOLO is very fast and can operate in real time on video streams or real-time images.

[0019] Furthermore, it is conceivable that XGBoost (Extreme Gradient Boosting) could be used for further processing, for example. XGBoost uses the gradient boosting framework to make precise predictions. It is particularly effective in solving regression and classification problems, but it can also be used for ranking, anomaly detection, and other machine learning tasks. The main advantage of XGBoost lies in its ability to build complex models with high accuracy while maintaining good scalability and speed. It uses a combination of decision trees to generate a strong ensemble prediction and optimizes the model error through gradient descent and regularization techniques.XGBoost offers many advanced features, such as regularization to prevent overfitting and control model complexity, sparsity-aware splitting to improve efficiency, cross-validation to evaluate model performance and optimize hyperparameters, and parallelization to accelerate training and prediction on multiple processors or compute nodes.

[0020] Alternatively, it is conceivable that the entire further processing and / or at least the noise reduction is based solely on at least one standard algorithm, which differs in particular from a KL algorithm. A standard algorithm should be understood in particular to be an algorithm free of KL technology. Alternatively or additionally, the further processing, in particular the noise reduction and / or the KL algorithm, can be based on at least one algorithm from a program library, preferably OpenCV, in particular with at least one predetermined parameter. In and / or for the further processing, image quality can be measured using an image quality metric, for example BRISQUE, wherein further processing parameters can be adapted, in particular automatically, depending on the image quality.In addition to being dependent on the connection between the internal processing unit and the external processing unit, further processing can take place depending on at least one further feature, for example depending on the image quality and / or an operator input, in particular only as needed. For example, further processing, in particular at least noise reduction, could only take place when the operator zooms in on the image displayed on the external output unit. This can advantageously reduce computing effort and, in particular, operating costs. Alternatively, it is conceivable that further processing always takes place as long as the connection between the internal processing unit and the external processing unit is present.

[0021] Image noise reduction is a process by which the distracting noise that can occur in images is reduced or removed to improve image quality. Noise can be caused by various factors, such as a high ISO setting during capture, poor lighting conditions, or the use of a low-quality sensor. Possible standard noise reduction algorithms can include filter-based noise reduction, whereby filters such as the median filter, the Gaussian filter, or the bilateral filter can be used to smooth and reduce noise. These filters remove noise by calculating the average or median values ​​of pixels in a neighborhood. Furthermore, the wavelet transform can be used, which allows an image to be decomposed into different frequency ranges.By applying wavelet transformation, high-frequency noise components can be identified and reduced while preserving the image structures. Furthermore, non-local averaging can be used. This method is based on the assumption that similar image areas should have similar brightness values. By calculating the average of similar pixels throughout the image, the noise can be reduced. Furthermore, it is proposed that the further processing be based at least partially on a GAN (Generative Adversarial Networks) algorithm. This can particularly advantageously increase the efficiency of the further processing and / or the quality of the further processed image. Preferably, the KL algorithm is designed as a GAN algorithm based on neural networks, particularly for noise reduction.Advantageously, at least one existing GAN algorithm can be used for noise reduction, in particular upsampling, and / or color enhancement, which can in particular reduce development costs. The GAN algorithm is preferably designed as a BSRGAN (Blind Super Resolution Generative Adversarial Network), SRGAN (Super Resolution Generative Adversarial Network), SWINIR (Shifted Window Image Restoration) or a comparable algorithm, or comprises at least one of these network types. The network architecture of a GAN preferably recognizes the true image information and / or the type of image region in the network intermediate layers. The image region, for example the noisy texture of an apple image section, is preferably replaced by the following generator network layers with an improved image section, for example the improved texture of the apple.A weighting of the GAN algorithm stores in particular at least one piece of text and / or object information, for example texture information, in particular at least implicitly.

[0022] A GAN algorithm is generally an algorithm consisting of two neural networks: the generator and the discriminator. The GAN model is used to generate new data that is similar to the training data. First, training data is collected to serve as the basis for the GAN. This data can be images, text, or other types of information. The generator is a neural network that attempts to generate new data similar to the training data. At the beginning of training, the generator generates random data. This generated data is then presented to the discriminator for evaluation. The discriminator is another neural network that attempts to distinguish between the generated data from the generator and the real training data. The discriminator is trained using the real training data to distinguish it from the generated data.The discriminator outputs a probability as to whether the input is real or generated. The GAN is trained through the iterative interaction between the generator and the discriminator. The generator attempts to improve the generated data to the point where the discriminator can no longer distinguish it from real data. The discriminator is then retrained using the updated generated data and the real data. This process is repeated until the generator is able to generate realistic data that resembles the real data. The competition between the generator and the discriminator causes the generator to generate better and more realistic data over time. The GAN model learns to capture the distribution of the training data and generate new data that conforms to this distribution.The success of a GAN depends on the quality and diversity of the training data, the architecture of the neural networks, and other hyperparameters. In addition to or as an alternative to noise reduction, the further processing may include brightness adjustment, contrast adjustment, color balance adjustment, and / or the like, which may be performed, for example, by a corresponding Cl algorithm. Alternatively, it is conceivable that the further processing, in particular at least in a further processing branch mentioned below, may only include noise reduction. The external processing unit could perform both internal and external processing. Furthermore, the external processing, in particular noise reduction, could not take place before the internal processing.Preferably, the brightness adjustment, the contrast adjustment, the color balance adjustment, and / or the like are part of the internal processing. In particular, insufficient color contrast and / or a shifted color balance in the image of the interior, for example, due to a dark interior design, can be advantageously improved. Preferably, all processing substeps necessary for output to the operator for all zoom levels, in particular including a distant view, of the output image are part of the internal processing. This ensures an operator-outputtable image that has a favorable appearance, at least in a zoomed-out state.The internal output unit and / or the external output unit display / displays an overall image, in particular the entire image outputtable by the operator, preferably automatically in an overview display, wherein the operator can preferably zoom in on areas of the image at least in the external output unit. At least in the overview display, the output image is in particular downscaled, thereby reducing in particular visible noise. Furthermore, the use of complex processing methods, in particular AI-based methods, can be reduced and / or the image outputtable by the operator can be improved. Advantageously, the computing effort and thus in particular the service life of cost-intensive GPU further processing used for deep-learning network calculations can be reduced.In particular, the internal processing is designed without a KL algorithm, in particular without the use of a neural network, and / or GPU processing. Preferably, the internal processing unit applies only standard image processing methods to the image of the interior. In particular, at least for color adjustment, an algorithm variant with significantly reduced computing power can be used, advantageously without reducing the overall quality of at least the further processed image. Alternatively, it would be conceivable for the internal processing to comprise at least one KL algorithm. The internal processing is particularly dependent on at least one processing parameter.The internal processing, in particular the internal processing step, preferably comprises at least one partial processing, in particular partial processing steps, each of which is in particular dependent on at least one associated processing parameter. The internal processing preferably processes the image of the interior to a desired quality and / or a shape and / or an appearance and / or an image section. In a further aspect of the invention, which can be considered on its own or in combination with further aspects of the invention, it is proposed that the internal processing unit processes the image depending on at least one adjustable processing parameter.Advantageously, the processing parameter and thus in particular the internal processing by the internal processing unit can be adapted particularly efficiently and / or cost-effectively to a type and / or model of a household appliance. The adjustable processing parameters preferably differ for a household appliance type and / or model and / or are variable, in particular adjustable, according to a household appliance type and / or model. Advantageously, the at least one adjustable processing parameter can be adapted to the interior of the household appliance, in particular to a layout and / or to a color and / or to the lighting of the interior. Processing and / or partial processing dependent on an adjustable processing parameter can in particular be independent of image content.The adjustable processing parameter is preferably variable independently of the image content. This advantageously reduces the required available computing power of the internal processing unit. The adjustable processing parameter is preferably adjustable by an update, in particular starting from the external processing unit and / or the backend and / or the external output unit. Advantageously, at least one partial processing step of the internal processing can be updated and, in particular, optimized even after assembly and, in particular, during operation of the household appliance.The internal processing preferably comprises at least one non-adjustable partial processing step, wherein, in the non-adjustable partial processing step, in particular, processing of image information and / or an image property is carried out which is independent of the type and / or model of the household appliance and in particular independent of the image content, and / or which depends on the image content and in particular is automatically adapted to the image content. The internal processing preferably comprises at least one adjustable partial processing step and at least one non-adjustable partial processing step.

[0023] In the internal processing, in particular in a pixel adaptation step, a number of bits per pixel is preferably adapted and preferably reduced, for example from 16 bpp to 8 bpp. Pixel adaptation is preferably non-adjustable. Preferably, the internal processing, in particular in a pixel correction step, comprises pixel correction. For the pixel correction, a pixel is preferably compared with the neighboring pixel and a deviation, for example with regard to a color and / or brightness, is determined, wherein if a deviation limit value is exceeded, it is assumed in particular that there is a dead and / or defective pixel, which is preferably replaced by a corrected pixel. The corrected pixel preferably has an average value of the neighboring pixels. The pixel correction is preferably non-adjustable.The pixel correction step takes place in a sequence of internal processing, in particular after the pixel adjustment step. Preferably, the internal processing, in particular in a white balance step, comprises a white balance in which a gray world white balance algorithm is preferably executed, which is based in particular on an assumption that the mean of all RGB color channels is the same, in which mean values ​​of all colors are calculated in particular and the pixels are then scaled according to the assumption. The white balance in the white balance step is in particular not adjustable. The white balance step preferably takes place in the sequence of internal processing after the pixel correction step. The internal processing can, preferably for an activated histogram, comprise a histogram equalization, in particular in a histogram equalization step.Histogram equalization is intended, in particular, for contrast enhancement and automatically depends on image content, namely, brightness distribution. Histogram equalization is, in particular, non-adjustable. A histogram-based scaling of individual color channels can take place in histogram equalization according to a non-adjustable processing parameter, which is, in particular, derived from the histogram or, for example, determined by a specification of the camera sensor and / or the household appliance. The histogram equalization step can take place in the internal processing sequence after the white balance step. Alternatively, it is conceivable that the internal processing is free of histogram equalization, in particular for a deactivated histogram.The internal processing preferably comprises a color transformation, in particular in a color transformation step, from a Bayer color image coding form to an RGB color image, wherein in particular missing color values ​​are determined by interpolation. The color transformation is in particular non-adjustable. The color transformation step can take place in the internal processing sequence after the white balance step or the histogram equalization step. The internal processing preferably comprises, in particular in a color correction step, a color correction according to a color correction matrix, preferably a 3x3 color correction matrix. The color correction is preferably adjustable and is in particular dependent on at least one adjustable processing parameter, in particular a matrix entry.For example, at least one matrix entry can be set to adjust a warm or cold exposure phenomenon in the operator-outputtable image. The color correction step takes place in the internal processing sequence, in particular after the color transformation step. In the internal processing, in particular in a perspective transformation step, a perspective of the image is preferably adjusted. A perspective transformation is preferably independent of image content, for example, of markings and / or patterns in the image of the interior, and is defined based on at least three, preferably exactly four, pixel coordinates. The pixel coordinates can, for example, each describe a corner pixel of the image. The pixels at the pixel coordinates can, for example, describe a rectangular shape or a trapezoidal shape.The perspective transformation is preferably adjustable and in particular dependent on at least one adjustable processing parameter, in particular a pixel coordinate. The perspective transformation step preferably takes place after the color correction step in the internal processing sequence. The image is preferably rotated in the internal processing sequence, in particular in a rotation step. A rotation is preferably adjustable and in particular dependent on at least one adjustable processing parameter, in particular a rotation angle. The rotation step preferably takes place after the perspective transformation step in the internal processing sequence. The image is preferably cropped in the internal processing sequence, in particular in a cropping step, in particular according to a region of interest.A crop window is preferably adjustable and in particular dependent on at least one adjustable processing parameter, in particular a pixel coordinate and / or a number of pixels to be removed. The cropping step preferably takes place after the rotation step in the internal processing sequence. The internal processing preferably comprises a gamma correction, in particular in a gamma correction step. The gamma correction is preferably adjustable and in particular dependent on at least one adjustable processing parameter, in particular a gamma value. The gamma correction step preferably takes place after the cropping step in the internal processing sequence. In the image, a brightness and / or a contrast is preferably adjusted in the internal processing, in particular in a brightness and / or contrast step.The brightness and / or contrast adjustment is preferably adjustable and in particular depends on at least one adjustable processing parameter, in particular a respective scaling parameter. The brightness and / or contrast step preferably takes place after the gamma correction step in the internal processing sequence. Color saturation is preferably adjusted in the internal processing, in particular in a saturation step. To adjust the color saturation, in particular an RGB color value is converted into an HSV color value, a color saturation is adjusted according to a color saturation parameter, and the color value is converted back into the RGB color space. The adjustment of a color saturation is preferably adjustable and in particular depends on at least one adjustable processing parameter, in particular the color saturation parameter.The saturation step preferably takes place after the brightness and / or contrast step in the internal processing sequence. The internal processing preferably comprises a resharpening of the image, in particular in a resharpening step. During the resharpening, edges in the image are preferably determined, in particular by means of a transition between light and dark at an edge, and highlighted. The resharpening is preferably adjustable and in particular dependent on at least one adjustable processing parameter, in particular an effect strength, for example in a value interval from 1 to 10. The resharpening step preferably takes place after the saturation step in the internal processing sequence.Preferably, the internal processing, in particular in a conversion step, comprises a conversion of the image into an outputtable image format, in particular into a JPEG or PNG or TIFF image format or another image format that appears appropriate to the person skilled in the art.

[0024] Bayer to RGB (Bayer-to-RGB) is a method for demosaicing images captured with a Bayer filter. A Bayer filter is a type of color filter matrix used in most digital cameras and sensors to capture color information. The Bayer filter consists of an array of red, green, and blue filters arranged in a pattern across the sensor. Each pixel of the sensor is covered with either a red, green, or blue filter. This results in each pixel containing information about only one of the three color channels (red, green, or blue). To obtain a complete color image, the image must be demosaiced by estimating the missing color information for each pixel. This is usually achieved through interpolation techniques, which use the color values ​​of neighboring pixels to estimate the missing color values.The Bayer-to-RGB process applies various algorithms to perform demosaicing and convert the image to the RGB color space. These algorithms can vary depending on the application and range from simple bilinear or bicubic interpolation techniques to more advanced methods such as adaptive color interpolation or machine learning. The result of the Bayer-to-RGB process is a complete color image consisting of the estimated color values ​​for each pixel. This image can then be further processed, displayed, or used in other applications that use the RGB color space.

[0025] Gamma correction is a method used in image processing to adjust the brightness and contrast of an image. It is based on the non-linear relationship between the input and output values ​​of image pixels. Gamma correction is often used to adjust the perception of brightness and contrast on screens and in printed media. It is based on the fact that human vision is not linear, but logarithmic. This means that we perceive changes in brightness more strongly in dark areas than in bright areas. Gamma correction uses a gamma value to adjust the brightness distribution of the image. A gamma value greater than 1 increases the contrast in dark areas and reduces the contrast in bright areas. A gamma value less than 1 has the opposite effect. The gamma value is applied to the intensity of each image pixel to calculate the output value.This can be done by applying a simple power function in which the input value of the pixel is raised to the power of the gamma value. The resulting output value is then the corrected value for the pixel. Gamma correction can help images to be displayed more consistently on different devices or in different environments. It can also be used to adjust the visual impression of an image and achieve certain visual effects. Preferably, the image is compressed during conversion. The conversion, in particular a compression, is preferably adjustable and in particular dependent on at least one adjustable processing parameter, in particular a compression rate, for example in a value interval of 1% to 100%. The conversion step takes place in the internal processing sequence, in particular after the sharpening step.The conversion step is specifically designed as a final step of the internal processing, and the image generated by the conversion step is specifically the user-outputtable image. For example, Libjpeg, an open-source library used for compressing and decompressing images in JPEG format, can be used for this purpose. Libjpeg provides functions for reading and writing JPEG files, as well as for manipulating JPEG images. It supports various compression and decompression modes, including lossy and lossless compression.

[0026] Histogram equalization is a method for adjusting the brightness distribution of an image to improve contrast. It is based on transforming the image histogram to achieve a more even distribution of brightness values. The histogram equalization process essentially consists of several steps. First, the histogram is calculated, after which the histogram of the image is created by counting the frequency of each brightness value in the image. Next, the cumulative distribution function (CDF) is calculated, after which the cumulative distribution function is calculated from the histogram by calculating the sum of the frequencies of all brightness values ​​up to a certain value. This creates a function that indicates how many pixels have a brightness value less than or equal to a certain value.Next, the histogram is transformed by normalizing the CDF and scaling it to the entire range of brightness values. This achieves an even distribution of the brightness values. Finally, the transformation is applied to the image. The transformation is then applied to each pixel in the image by replacing the pixel's brightness value with the corresponding value in the transformed CDF. Histogram equalization rescales the brightness values ​​in the image, thereby improving contrast. Areas of low brightness are brightened, while areas of high brightness are darkened. This makes details and structures in the image more visible.Alternatively or additionally, the internal processing may comprise further image processing steps that appear appropriate to a person skilled in the art and are particularly standardized, and / or the partial processing steps may have a different sequence of execution. Alternatively, it would also be conceivable for at least one adjustable partial processing step, for example, the perspective transformation step, to be non-adjustable, whereby the perspective transformation step may, for example, take image content into account and thus, in particular, involve increased computational complexity. It would be conceivable for at least one non-adjustable partial processing step to be adjustable and, in particular, dependent on at least one adjustable processing parameter.

[0027] It is further proposed that the further processing, in particular of the external processing unit, processes the operator-outputtable image into a machine-readable format in a processing branch. Flexibility can advantageously be increased, in particular by providing improved additional machine-based further processing, preferably automatic image recognition, based on a further-processed image in the machine-readable format. Furthermore, the reliability and / or speed of automatic image recognition can advantageously be increased based on the further-processed image in the machine-readable format. The further-processed image in the machine-readable format is preferably only suitable and / or intended for use in automatic, in particular machine-based, additional further processing, and in particular not for output to the operator.For example, the further processed image in the machine-readable format may appear unnatural to the operator, for example due to excessive sharpening, high brightness, high contrast, unnatural colors, in particular colors that do not correspond to the reality of the interior, and / or the like. The further processed image in the machine-readable format is preferably used in additional processing, in particular automatic image recognition. The external processing unit preferably performs the additional processing based on the further processed image in the machine-readable format.Preferably, the operator-outputtable image is further processed into the machine-readable format depending on a type of additional further processing, in particular automatic image recognition, for example object recognition and / or text recognition and / or QR code recognition and / or barcode recognition and / or the like. It is conceivable that only one further processed image in a machine-readable format is generated from the operator-outputtable image. Alternatively, it is conceivable that more than one further processed image in a machine-readable format is generated from the operator-outputtable image, preferably for various additional further processing, in particular for various types of recognition beyond automatic image recognition.The processing of the processing branch comprises, in particular, sharpening and / or brightness adjustment and / or contrast adjustment and / or color adjustment and / or the like. The processing of the processing branch can include noise adjustment. The processing of the processing branch can be based at least partially on at least one KL algorithm, for example the KL algorithm mentioned above. Alternatively or additionally, another algorithm, in particular a standard algorithm, would be conceivable for processing in the processing branch. Preferably, the additional further processing, in particular the automatic image recognition, is based on at least one KL algorithm.Alternatively, it is conceivable that the additional processing, in particular the automatic image recognition, is carried out directly on the basis of the image that can be output by the operator and / or is based on a further processed image that has been processed in a further processing branch mentioned below for output to the operator.

[0028] For example, for object recognition

[0029] It is also proposed that the further processing, in particular of the external processing unit, revises the operator-outputtable image for output to the operator in a further processing branch, in particular the one mentioned above. Advantageously, the processing branch and the further processing branch of the further processing can provide separate further processed images for output to the operator and for additional further processing, whereby these further processed images can each be improved in a particularly advantageous manner for the corresponding intended function. Advantageously, the operator-outputtable image can be revisited by means of the further processing branch into an additionally improved further processed image that can be output to the operator. In particular, the operator's ease of use can be advantageously increased by providing a particularly improved image.The revision of the further processing branch preferably includes noise reduction or is designed as such. The revision of the further processing branch is based, in particular, at least partially on the at least one Kl algorithm. The image further processed in the further processing branch is preferably output to the external output unit, in particular to the operator, particularly if the connection between the internal processing unit and the external processing unit is present.

[0030] Preferably, the processing operations of the processing branch and the further processing branch are carried out separately from one another and / or simultaneously with one another, in particular by means of the external processing unit. Alternatively, it is conceivable for the processing branch and the further processing branch to have at least one identical further processing sub-step. Preferably, the processing branch and the further processing branch have at least one different further processing sub-step. For the presence of the connection between the internal processing unit and the external processing unit, the further processing can have only the first processing thread or only the second processing thread or both processing threads. Preferably, the further processing has at least the processing thread for the presence of the connection between the internal processing unit and the external processing unit.For example, the further processing may comprise the further processing strand depending on the determined image quality of the operator-outputtable image, in particular for an image quality less than a threshold value.

[0031] Furthermore, a household appliance device, in particular the one mentioned above, is proposed for a household appliance, in particular the one mentioned above, which device has a processing unit, in particular the one mentioned above, internal to the household appliance, for processing an image, in particular the one mentioned above, of an interior of the household appliance, in particular the one mentioned above, into an image, in particular the one mentioned above, that can be output by the operator, and with a communication unit for transmitting the image that can be output by the operator to a processing unit, in particular the one mentioned above, that is external to the household appliance, in particular as a function of a connection, in particular the one mentioned above, between the internal processing unit and the external processing unit for further processing of the image that can be output by the operator, in particular the one mentioned above.The household appliance device is preferably designed to execute at least a portion, in particular only a portion, of the above-mentioned method for processing the data of the household appliance. Advantageously, efficiency, in particular computing efficiency and / or operating cost efficiency for executing the method and / or manufacturing efficiency of at least the household appliance, can be increased. Furthermore, flexibility can be increased, in particular by providing an operator-outputable image even when the connection between the internal processing unit and the external processing unit is missing.

[0032] Furthermore, a household appliance, in particular the one mentioned above, with a household appliance device, in particular the one mentioned above, is proposed. The efficiency and / or flexibility of the household appliance can be particularly advantageously increased.

[0033] Furthermore, an external processing unit, in particular the one mentioned above, is proposed, which is external to a household appliance, in particular the one mentioned above, and is provided, depending on a connection, in particular the one mentioned above, to a processing unit internal to the household appliance, in particular the one mentioned above, for a further processing, in particular the one mentioned above, of an operator-displayable image, in particular the one mentioned above, processed by the internal processing unit from an image, in particular the one mentioned above, of an interior of the household appliance, in particular the one mentioned above. The external processing unit is preferably provided for executing at least part, in particular only part, of the above-mentioned method for processing the data of the household appliance. Advantageously, efficiency and / or flexibility can be increased.In particular, operating costs for the external processing unit can be advantageously reduced by pre-processing the image of the interior by the internal processing unit. The external processing unit can provide an additionally enhanced image through further processing, particularly flexibly.

[0034] Furthermore, a household appliance system is proposed, comprising a household appliance device, in particular the one mentioned above, and an external processing unit, in particular the one mentioned above. The efficiency and / or flexibility of the household appliance system can be increased, particularly advantageously.

[0035] Furthermore, a computer program comprising instructions is proposed which, when executed by at least one computer, cause the computer program to carry out the above-mentioned method. The at least one computer is preferably at least part of the internal processing unit and / or the external processing unit and can be part of the external output unit. Advantageously, a particularly efficient and / or flexible computer program can be provided.

[0036] Furthermore, a computer-readable storage medium is proposed, comprising instructions which, when a computer program, in particular the above-mentioned one, is executed by at least one computer, in particular the above-mentioned one, cause the computer to carry out the above-mentioned method. The computer-readable storage medium preferably comprises the computer program. The computer-readable storage medium can be designed as part of the internal processing unit and / or the external processing unit and, in particular, can be permanently installed therein. The computer-readable storage medium can comprise the storage unit of the internal processing unit. Advantageously, storage of a particularly efficient and / or flexible computer program can be provided.

[0037] The method for processing data of the household appliance, the household appliance device, the household appliance, the external processing unit, the household appliance system, the computer program, and / or the computer-readable storage medium should not be limited to the application and embodiment described above. In particular, the method for processing data of the household appliance, the household appliance device, the household appliance, the external processing unit, the household appliance system, the computer program, and / or the computer-readable storage medium can have a number of individual method steps, elements, components, and units that differs from the number stated herein to fulfill a functionality described herein.

[0038] Further advantages will become apparent from the following description of the drawings. The drawings illustrate exemplary embodiments of the invention. The drawings, the description, and the claims contain numerous features in combination. Those skilled in the art will also conveniently consider the features individually and combine them into useful further combinations. They show:

[0039] Fig. 1 A household appliance system with a household appliance having an internal processing unit and with a processing unit external to the household appliance,

[0040] Fig. 2 is a flowchart for a method for processing data of the household appliance and

[0041] Fig. 3 is a flowchart for internal processing by means of the internal processing unit in the method.

[0042] Figure 1 shows a household appliance system 30 with a household appliance device 28 of a household appliance 10 and a processing unit 14 external to the household appliance 10. In this case, the household appliance system 30 comprises the household appliance 10. The household appliance 10 is designed as a household refrigeration appliance 20, in particular as a refrigerator and / or freezer. Alternatively, another configuration of the household appliance 10 that would be deemed appropriate by a person skilled in the art, for example, as a household cleaning appliance or a household cooking appliance, is conceivable.

[0043] The household appliance 10 has the household appliance device 28. The household appliance device 28 has a processing unit 12, in particular internal to the household appliance 10, for processing an image of an interior 16 of the household appliance 10 into an image that can be output by the operator. The internal processing unit 12 processes the image of the interior 16 in an internal processing step to produce the image that can be output by the operator. The image that can be output by the operator is output to the operator in at least one operating state.

[0044] The household appliance device 28 has a communication unit 22 for transmitting the operator-outputtable image to the processing unit 14 external to the household appliance 10, in particular, depending on a connection 18 between the internal processing unit 12 and the external processing unit 14 for further processing of the operator-outputtable image.

[0045] The communication unit 22 provides the connection 18 between the internal processing unit 12 and the external processing unit 14. The communication unit 22 connects the internal processing unit 12 and the external processing unit 14 to each other for data communication. The communication unit 22 connects the internal processing unit 12 and the external processing unit 14 to each other wirelessly.

[0046] The external processing unit 14 is provided, depending on the connection 18 to the processing unit 12 internal to the household appliance 10, for further processing the operator-output image processed by the internal processing unit 12 from the image of the interior 16 of the household appliance 10. The external processing unit 14 is designed as a cloud unit 34. The external processing unit 14 provides cloud computing. Alternatively, it would be conceivable for the external processing unit 14 to be designed, for example, as a smart device (not shown). The connection 18 is designed as a server connection.

[0047] The household appliance system 30 can have an output unit 36 ​​external to the household appliance 10. The external output unit 36 ​​is connected, in particular wirelessly, to the internal processing unit 12. The external output unit 36 ​​can be connected to the internal processing unit 12 via the communication unit 22 or via another communication unit (not shown). The external output unit 36 ​​can be connected to the external processing unit 14.

[0048] The external output unit 36 ​​is intended for the optical display of an image. The external output unit 36 ​​is designed as a smart device, in this case a smartphone. Alternatively, another configuration of the external output unit 36 ​​that would be deemed appropriate by a person skilled in the art is conceivable.

[0049] The household appliance 10 has an internal output unit 38. The internal output unit 38 is embedded or attached in or to a door 40 of the household appliance 10. Alternatively, another arrangement of the internal output unit 38 that would be deemed appropriate by a person skilled in the art is conceivable. The internal output unit 38 is provided for displaying the image that can be output by the operator. Alternatively, it is conceivable that the household appliance 10 is designed without the internal output unit 38.

[0050] The household appliance 10 has at least one camera sensor 32. The camera sensor 32 captures at least one image of the interior 16 of the household appliance 10. The camera sensor 32 transmits the at least one image of the interior 16 to the internal processing unit 12. The household appliance 10 has, for example, exactly two camera sensors 32. One camera sensor 32 is arranged, for example, on a ceiling 42 of the household appliance 10. Another camera sensor 32 is arranged, for example, on a door 40, in particular one of the doors 40, of the household appliance 10. Alternatively, a number of camera sensors 32 and / or another arrangement of the camera sensors 32 with respect to the interior 16 that appears appropriate to a person skilled in the art is conceivable.

[0051] Figure 2 shows a flowchart for a method for processing data of the household appliance 10, wherein the processing unit 12 internal to the household appliance 10 processes the at least one image of the interior 16 of the household appliance 10 into the operator-outputtable image, and wherein the operator-outputtable image is further processed by the external processing unit 14 depending on the connection 18 between the internal processing unit 12 and the processing unit 14 external to the household appliance 10.

[0052] The method comprises an internal processing step 100. The internal processing unit 12 processes the image of the interior 16 into the operator-outputable image in the internal processing step 100, in particular in the internal processing.

[0053] The operator-outputtable image is output to the internal output unit 38 in an internal output step 102. The operator-outputtable image is output to the internal output unit 38 independently of the connection 18 between the internal processing unit 12 and the external processing unit 14.

[0054] The method comprises a detection step 104. In the detection step 104, a state of the connection 18 between the internal processing unit 12 and the external processing unit 14 is detected. In the detection step 104, the presence or absence of the connection 18 between the internal processing unit 12 and the external processing unit 14 is detected.

[0055] The operator-outputtable image is transmitted in a transmission step 106 to the output unit 36 ​​external to the household appliance 10 depending on the connection 18 between the internal processing unit 12 and the external processing unit 14. In the transmission step 106, the operator-outputtable image is transmitted to the external output unit 36 ​​if the connection 18 between the internal processing unit 12 and the external processing unit 14 is absent. The method comprises an external output step 108, in particular if the transmission step 106 is present. In the external output step 108, the operator-outputtable image is output to the external output unit 36, in particular to the operator.

[0056] The operator-outputtable image is transmitted to the external processing unit 14 in a first transmission step 110 depending on the connection 18 between the internal processing unit 12 and the external processing unit 14. The operator-outputtable image is transmitted to the external processing unit 14 in the first transmission step 110, in particular by means of the communication unit 22, for the presence of the connection 18 between the internal processing unit 12 and the external processing unit 14. The method comprises either the transmission step 106 or the first transmission step 110 depending on the connection 18 between the internal processing unit 12 and the external processing unit 14.

[0057] The method comprises further processing, particularly after the first transmission step 110. The method comprises, particularly after the first transmission step 110, a further processing branch in a further processing step 112. In the further processing branch, particularly in the further processing step 112, the operator-outputable image is processed into a machine-readable format.

[0058] An image further processed by further processing step 112 is converted into the machine-readable format for additional processing, in particular in an additional processing step (not shown). The additional processing includes at least automatic image recognition, for example, object recognition and / or text recognition and / or QR code recognition and / or barcode recognition and / or the like. The operator-outputable image is processed in further processing step 112 depending on the type of additional processing.

[0059] The method comprises an additional transmission step 114, in particular after the further processing step 112. In the additional transmission step 114, the image further processed by the further processing step 112 is transmitted for additional processing. It is conceivable that the method comprises the additional further processing, in particular the additional further processing step. It is conceivable that the external processing unit 14 carries out the additional further processing. Alternatively, the additional processing could take place outside the external processing unit 14.

[0060] The method comprises a decision step 116, particularly after the first transmission step 110. In decision step 116, a decision is made regarding the execution of a further processing branch of the further processing. For example, decision step 116 may include determining the image quality of the operator-outputtable image, for example, based on an image quality metric. The external processing unit 14 may determine the image quality of the operator-outputtable image.

[0061] Depending on the decision step 116, the method is free of the further processing branch. Depending on the decision step 116, for example, for a determined image quality above a threshold value, the method comprises a second transmission step 118. In the second transmission step 118, the operator-outputtable image is transmitted from the external processing unit 14 to the external output unit 36.

[0062] Depending on the decision step 116, the method comprises the external output step 108, in particular after the second transmission step 118, in which the operator-outputable image is displayed on the external output unit 36.

[0063] Depending on the decision step 116, the method comprises a further processing branch in a further processing step 120. Depending on the decision step 116, for example, for a determined image quality below a threshold value in the decision step 116, the method comprises the further processing step 120.

[0064] In the further processing branch of the further processing, the operator-outputable image is revised for output to the operator. The further processing, in particular at least the further processing branch in the further processing step 120, comprises noise reduction. The noise reduction comprises upsampling. The further processing, in particular at least the further processing branch in the further processing step 120 and in particular at least the noise reduction, is based at least partially on a Kl algorithm. The Kl algorithm is preferably based on a neural network. The further processing, in particular at least the further processing branch in the further processing step 120 and in particular at least the noise reduction, is based at least partially on a GAN algorithm. The Kl algorithm is in particular at least partially a GAN algorithm.

[0065] The further processing, in particular at least the further processing branch in the further processing step 120 and in particular at least the noise reduction, is based at least in part on image information, in particular real image information. An image region further processed in the further processing can be replaced by a partial image generated by the K1 algorithm, in particular the GAN algorithm. The GAN algorithm can be implemented as a BSRGAN (Blind Super Resolution Generative Adversarial Network), SRGAN (Super Resolution Generative Adversarial Network), SWINIR (Shifted Window Image Restoration) or a comparable algorithm, or comprises at least one of these network types.

[0066] The method comprises a further second transmission step 122, in particular after the further processing step 120. In the further second transmission step 122, the image further processed in the further processing branch, in particular the further processing step 120, is transmitted to the external output unit 36.

[0067] The method comprises a further external output step 124, in particular for the presence of the further second transmission step 122. In the external output step 124, the image further processed, in particular in the further processing step 120 and in particular the further processing branch of the further processing, is output to the external output unit 36.

[0068] Alternatively, it is conceivable for the method to be free of at least one of the above-mentioned method steps. For example, the method could be free of the internal output step 102, in particular for one embodiment of the household appliance 10, free of the internal output unit 36. The method could, for example, be free of the decision step 116 and the further second transmission step 118, wherein the further processing could, for example, always comprise the further processing branch and in particular the further further processing step 120. It would also be conceivable for the method to be designed free of the further processing step 112 or the further further processing step 120 and, in particular, comprise only one of these two method steps.

[0069] The household appliance 10 and / or the household appliance device 28 comprise / comprises at least part of a computer-readable storage medium 26 comprising instructions which, when a computer program is executed by at least one computer 24, cause the computer 24 to carry out the above-mentioned method (see Figure 1). The household appliance 10 and / or the household appliance device 28 comprise / comprises the computer-readable storage medium 26 and / or the computer 24 in part. The external processing unit 14 comprises the computer-readable storage medium 26 and / or the computer 24 in part. The external output unit 36 ​​can comprise the computer-readable storage medium 26 and / or the computer 24 in part.

[0070] Figure 3 shows a flowchart of the internal processing, in particular of the internal processing step 100. The internal processing unit 12 processes the image, in particular in the internal processing and in particular in the internal processing step 100, depending on at least one adjustable processing parameter.

[0071] The internal processing comprises at least one partial processing, in particular at least one partial processing step, which depends on at least one respective processing parameter. The adjustable processing parameter can be adjusted for a respective household appliance type and / or a respective household appliance model. The adjustable processing parameter can preferably be changed independently of the image content. The internal processing comprises at least one non-adjustable partial processing, in particular one non-adjustable partial processing step.

[0072] The internal processing step 100 includes a pixel adjustment step 130 with pixel adjustment. In the pixel adjustment step 130, a number of bits per pixel in the image of the interior 16 is adjusted for the operator-outputtable image. The pixel adjustment is non-adjustable. The internal processing step 100 includes a pixel correction step 132 with pixel correction. In the pixel correction step 132, dead and / or defective pixels are detected and corrected. The pixel correction is non-adjustable.

[0073] The internal processing step 100 includes a white balance step 134 with a white balance. In the white balance step 134, a gray world white balance algorithm is executed. The white balance is not adjustable.

[0074] For an activated histogram, the internal processing step 100 may include a histogram equalization step 136 with a histogram equalization. The histogram equalization is non-adjustable.

[0075] The internal processing step 100 includes a color transformation step 138 with a color transformation. In the color transformation step 138, a Bayer color image encoding form is transformed into an RGB color image. The color transformation is non-adjustable. The color transformation step 138 can take place depending on the activation of the histogram after the white balance step 134 or the histogram equalization step 136.

[0076] The internal processing step 100 includes a color correction step 140 with a color correction according to a color correction matrix. The color correction depends on at least one adjustable processing parameter, in particular on at least one adjustable matrix entry.

[0077] The internal processing step 100 includes a perspective transformation step 142 with a perspective transformation. The perspective transformation depends on at least one adjustable processing parameter, in particular on at least one pixel coordinate to be transformed.

[0078] The internal processing step 100 includes a rotation step 144 involving a rotation of the image. The rotation depends on at least one adjustable processing parameter, in particular on at least one adjustable rotation angle.

[0079] The internal processing step 100 comprises a cropping step 146 involving cropping the image. A cropping window for cropping the image is preferably adjustable. The cropping window depends on at least one adjustable processing parameter, in particular on an adjustable pixel coordinate and / or a number of pixels to be removed.

[0080] The internal processing step 100 includes a gamma correction step 148 with a gamma correction. The gamma correction depends on at least one adjustable processing parameter, in particular on at least one adjustable gamma value.

[0081] The internal processing step 100 comprises a brightness and / or contrast step 150 with a brightness and / or contrast adjustment. The brightness and / or contrast adjustment depends on at least one adjustable processing parameter, in particular on at least one respective scaling parameter.

[0082] The internal processing step 100 includes a saturation step 152 with an adjustment of color saturation. The adjustment of color saturation depends on at least one adjustable processing parameter, in particular on at least one adjustable color saturation parameter.

[0083] The internal processing step 100 includes a sharpening step 154 ​​with a sharpening of the image. The sharpening depends on at least one adjustable processing parameter, in particular on at least one adjustable effect strength.

[0084] Internal processing step 100 includes a conversion step 156 for converting the image into an outputtable image format, for example, a JPEG image format. The image is compressed in conversion step 156. The conversion, in particular compression, depends on at least one adjustable processing parameter, in particular on at least one adjustable compression rate. Conversion step 156 is embodied as a final method step of internal processing step 100. The image generated by conversion step 156 is the image that can be output by the operator.

[0085] Alternatively or additionally, the internal processing may include further image processing steps that appear appropriate to the person skilled in the art and are particularly standardized, and / or the partial processing steps may have a different sequence of execution. Alternatively, it would also be conceivable for at least one non-adjustable partial processing step to be adjustable and / or for at least one adjustable partial processing step to be non-adjustable.

[0086] Of multiple objects, only one is provided with a reference symbol in the figures.

[0087] Reference symbol

[0088] 10 household appliances

[0089] 12 internal processing unit

[0090] 14 external processing unit

[0091] 16 Interior

[0092] 18 Connection

[0093] 20 household refrigeration appliances

[0094] 22 Communication unit

[0095] 24 computers

[0096] 26 computer-readable storage medium

[0097] 28 Household appliance device

[0098] 30 household appliance system

[0099] 32 camera sensor

[0100] 34 Cloud Unit

[0101] 36 external output unit

[0102] 38 internal output unit

[0103] 40 Door

[0104] 42 ceiling

[0105] 100 internal processing steps

[0106] 102 internal output step

[0107] 104 Recording step

[0108] 106 Transmission step

[0109] 108 external output step

[0110] 110 first transmission step

[0111] 112 Further processing step

[0112] 114 additional transmission step

[0113] 116 Decision step

[0114] 118 second transmission step

[0115] 120 further processing step further second transmission step external output step pixel adjustment step pixel correction step white balance step histogram equalization step color transformation step color correction step perspective transformation step rotation step cropping step gamma correction step

[0116] Brightness and / or contrast step Saturation step Sharpening step Conversion step

Claims

Claims 1. A method for processing data of a household appliance (10), wherein a processing unit (12) internal to the household appliance (10) processes at least one image of an interior (16) of the household appliance (10) into an image that can be output by the user, characterized in that the image that can be output by the user is further processed by the external processing unit (14) depending on a connection (18) between the internal processing unit (12) and a processing unit (14) external to the household appliance (10).

2. Method according to claim 1, characterized in that the further processing comprises noise reduction.

3. Method according to claim 1 or 2, characterized in that the further processing is based at least partially on a Kl algorithm, in particular a neural network.

4. Method according to claim 3, characterized in that the further processing is based at least partially on a GAN algorithm.

5. Method at least according to the preamble of claim 1, in particular according to one of the preceding claims, characterized in that the internal processing unit (12) processes the image as a function of at least one adjustable processing parameter.

6. Method according to one of the preceding claims, characterized in that the further processing in a processing branch processes the operator-outputtable image into a machine-readable format.

7. Method according to one of the preceding claims, characterized in that the further processing in a further processing branch revises the operator-outputable image for output to the operator.

8. Household appliance device (28) for a household appliance (10), in particular for carrying out at least part of the method according to one of claims 1 to 7, with a processing unit (12) internal to the household appliance (10) in particular for processing an image of an interior space (16) of the household appliance (10) into an image that can be output by the user and with a communication unit (22) for transmitting the image that can be output by the user to a processing unit (14) external to the household appliance (10) in accordance with a connection (18) between the internal processing unit (12) and the external processing unit (14) for further processing the image that can be output by the user.

9. Household appliance (10), in particular household refrigeration appliance (20), with a household appliance device (28) according to claim 8.

10. External processing unit (14), in particular for carrying out at least part of the method according to one of claims 1 to 7, which is external to a household appliance (10) and is provided in dependence on a connection (18) to a processing unit (12) internal to the household appliance (10) for further processing of an operator-outputtable image processed by means of the internal processing unit (12) from an image of an interior (16) of the household appliance (10).

11. A household appliance system (30) comprising a household appliance device (28) according to claim 8 and an external processing unit (14) according to claim 10.

12. A computer program comprising instructions which, when the computer program is executed by at least one computer (24), cause the computer (24) to carry out the method according to one of claims 1 to 7.

13. A computer-readable storage medium (26) comprising instructions which, when a computer program is executed by at least one computer (24), cause the computer (24) to carry out the method according to one of claims 1 to 7.

Images