Dishwasher which acquires image of dishes by using reflector and method for acquiring image

Abstract

The present invention relates to a dishwasher which acquires an image of dishes by using a reflector and a method for acquiring an image. The dishwasher, according to one embodiment of the present invention, comprises: a tub in which dishes are accommodated; one or more reflectors disposed above the tub; a camera for capturing an image reflected from the reflector; and a control unit for generating one image by aligning two or more images captured by the camera.

Description

Dishwasher for acquiring an image of tableware using a reflector and a method for acquiring an image

[0001] The present invention relates to a dishwasher that acquires an image of tableware using a reflector and a method for acquiring the image.

[0002] The content described in this section merely provides background information regarding the present invention and does not constitute prior art.

[0003] A dishwasher is a device that uses detergent and washing water to clean dirt, such as food residue, stuck to dishes or cooking utensils.

[0004] A typical dishwasher includes a tub that provides a washing space, a rack provided within the tub for holding dishes, a spray arm that sprays wash water onto the rack, a sump that stores wash water, and a pump that supplies wash water stored in the sump to the spray arm.

[0005] Since it is difficult to check the condition of the interior when the door of a dishwasher is closed, a camera can be placed inside. However, due to the size of the dishwasher, distortion may occur in the video captured by the camera, and technology to resolve this is required.

[0006] In order to solve the aforementioned problem, this specification aims to secure a sufficient shooting distance by placing a reflector inside a dishwasher and having a camera capture images of the dishes reflected by the reflector.

[0007] This specification aims to implement a method for acquiring images of tableware without distortion through the distance between a camera and a mirror, and a dishwasher.

[0008] The objects of the present invention are not limited to those mentioned above, and other unmentioned objects and advantages of the present invention may be understood from the following description and will be more clearly understood by the embodiments of the present invention. Furthermore, it will be readily apparent that the objects and advantages of the present invention can be realized by the means and combinations thereof set forth in the claims.

[0009] A dishwasher according to one embodiment of the present invention includes a tub for receiving dishes, one or more reflectors disposed on the upper side of the tub, a camera for capturing an image reflected by the reflectors, and a control unit for aligning two or more images captured by the camera to generate a single image.

[0010] A method for acquiring an image of tableware using a mirror according to an embodiment of the present invention comprises, in a dishwasher including a tub for receiving tableware, one or more racks disposed in the tub, one or more mirrors disposed on the upper side of the tub, a camera, and a control unit, wherein when storage begins after the rack is withdrawn, the control unit controls the camera to photograph an image reflected by the mirror; the control unit controls the camera to photograph the mirror until storage is finished; and the control unit aligns two or more images captured by the camera to generate a single image.

[0011] When applying the present invention, a mirror is placed inside the dishwasher and a camera captures images of the dishes reflected by the mirror, thereby securing a sufficient shooting distance.

[0012] When the present invention is applied, images of tableware can be obtained without distortion through the distance between the camera and the mirror.

[0013] The effects of the present invention are not limited to the effects described above, and those skilled in the art can easily derive various effects of the present invention from the configuration of the present invention.

[0014] FIG. 1 is a drawing showing the external shape of a dishwasher according to one embodiment of the present invention.

[0015] FIG. 2 is a cross-sectional view of a dishwasher according to one embodiment.

[0016] FIGS. 3 to 5 are drawings showing the process of a dishwasher according to one embodiment of the present invention being opened and then closed by a door.

[0017] FIG. 6 is a plan view of the configuration between a mirror and a camera according to one embodiment of the present invention.

[0018] FIG. 7 is a side view and a top view of a dishwasher with two reflectors arranged according to one embodiment of the present invention.

[0019] FIG. 8 is a side view and a top view of a dishwasher with two reflectors arranged according to another embodiment of the present invention.

[0020] FIG. 9 is a diagram showing how to set the field of view and shooting distance of a camera in correspondence with the inclination of dishes placed in a dishwasher according to one embodiment of the present invention.

[0021] FIGS. 10 and 11 are graphs comparing the object movement speed and the camera exposure time according to an embodiment of the present invention.

[0022] FIG. 12 is a graph showing the correlation between the movement speed of a camera and a rack according to one embodiment of the present invention.

[0023] FIGS. 13 and 14 are drawings showing the internal configuration of a dishwasher with two reflectors arranged according to another embodiment of the present invention.

[0024] FIG. 15 is a diagram summarizing the components of a dishwasher according to one embodiment of the present invention.

[0025] FIG. 16 is a diagram showing an image matching process according to one embodiment of the present invention.

[0026] FIG. 17 is a flowchart showing an image matching process according to one embodiment of the present invention.

[0027] FIG. 18 is a diagram showing the operation flow of an optical system of a dishwasher according to one embodiment of the present invention.

[0028] FIG. 19 is a diagram showing the process of determining the number of shooting frames per second of a camera in correspondence with the storage speed of a rack according to one embodiment of the present invention.

[0029] FIG. 20 is a drawing showing a process of selecting and aligning some of the images captured by a camera in correspondence with the storage speed of a rack according to one embodiment of the present invention.

[0030] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings so that those skilled in the art can easily implement the invention. The present invention may be embodied in various different forms and is not limited to the embodiments described herein.

[0031] To clearly explain the present invention, parts unrelated to the description have been omitted, and the same reference numerals are assigned to identical or similar components throughout the specification. Furthermore, some embodiments of the present invention are described in detail with reference to the exemplary drawings. In assigning reference numerals to the components of each drawing, identical components may have the same reference numeral whenever possible, even if they are shown in different drawings. Additionally, in describing the present invention, if it is determined that a detailed description of related known components or functions could obscure the essence of the present invention, such detailed description may be omitted.

[0032] In describing the components of the present invention, terms such as first, second, A, B, (a), (b), etc., may be used. These terms are intended only to distinguish the components from other components, and the nature, order, sequence, or number of the components are not limited by these terms. Where it is stated that a component is "connected," "combined," or "connected" to another component, it should be understood that the component may be directly connected or connected to the other component, but that other components may be "interposed" between each component, or that each component may be "connected," "combined," or "connected" through other components.

[0033] In addition, for convenience of explanation in implementing the present invention, the components may be described in detail; however, these components may be implemented within a single device or module, or a single component may be divided and implemented across multiple devices or modules.

[0034] This specification presents the configuration of a home appliance utilizing a mirror optical structure to effectively photograph objects within a narrow interior, and a method for the home appliance to align images. When a camera intends to acquire images through bird's-eye view shooting in a space where the distance to objects is narrow, such as in a dishwasher, refrigerator, oven, or washing machine, a structure using a mirror, which is an embodiment of the present invention, can be applied.

[0035] In addition, when applying embodiments of the present invention, three or more cameras are installed inside a home appliance, and images from different viewpoints can be obtained by varying the camera optical axis and the tilt of the reflector surface. This image data can be provided as data for a Nerf (Neural Radiance Fields) model that creates Novel View Synthesis (NVS), which provides images from multiple viewpoints of various objects placed inside the home appliance.

[0036] Next, we will briefly examine the external shape and internal structure of the dishwasher to which the embodiment of the present invention is applied.

[0037] FIG. 1 is a drawing showing the external shape of a dishwasher according to one embodiment of the present invention. An exemplary exterior of the dishwasher (1) may consist of a case (11), a door (20), and a control panel (5).

[0038] FIG. 2 is a cross-sectional view of a dishwasher according to one embodiment. A dishwasher (1) according to one embodiment may include a case (11) forming an exterior, a tub (12) in which dishes to be washed are received, a door (20) provided on the front of the tub (12) to open and close the tub (12), and a sump (100) provided on the lower side of the tub (12) in which washing water is stored.

[0039] Additionally, the dishwasher may include a plurality of spray arms (13, 14, 15) provided in a tub (12) for spraying washing water, a filter (200) provided in a sump (100) for filtering washing water sprayed from at least one of the plurality of spray arms (13, 14, 15) and recovered to the sump (100), a washing pump (150) for pressurizing washing water stored in the sump (100), and a switching valve (130) for flowing washing water pressurized by the washing pump (150) to at least one of the plurality of spray arms (13, 14, 15).

[0040] The tub (12) is formed in the shape of a cuboid with its front open by a door (20) to form a washing chamber (12a) inside. A communication hole is formed in the bottom (12b) of the tub (12) through which washing water flows into the sump (100). The washing chamber (12a) is equipped with a plurality of racks (16, 17a, 17b) for storing items to be washed. The plurality of racks (16, 17a, 17b) may include a first rack (16) placed at the bottom of the washing chamber (12a), a second rack (17a) placed above it, and a third rack (17b) placed above it. The first rack (16), the second rack (17a), and the third rack (17b) are spaced apart vertically and can be slid out toward the front of the tub (12). Each rack is a dish basket in which dishes, etc. are placed. Although 17a and 17b are shown in the drawing, only one of 17a or 17b may be placed depending on the structure of the dishwasher (1). Alternatively, according to another embodiment, only one rack (16) may be placed.

[0041] A plurality of spray arms (13, 14, 15) are arranged in an up-and-down direction. The plurality of spray arms (13, 14, 15) may include a lower spray arm (13) positioned at the bottom and spraying washing water from the bottom to the top toward the first rack (16), an upper spray arm (14) positioned above the lower spray arm (13) and spraying washing water from the bottom to the top toward the second rack (17a), and a tower spray arm (15) positioned at the top of the washing room (12a) above the upper spray arm (14) and spraying washing water from the top toward the third rack (17b).

[0042] A plurality of spray arms (13, 14, 15) receive washing water from a washing pump (150) through a plurality of spray arm connecting channels (18, 19, 21). The plurality of spray arm connecting channels (18, 19, 21) may include a lower spray arm connecting channel (18) connected to a lower spray arm (13), an upper spray arm connecting channel (19) connected to an upper spray arm (14), and a tower spray arm connecting channel (21) connected to a tower spray arm (15).

[0043] The lower sandstone (13), upper sandstone (14) and tower sandstone (15) can each receive washing water from the washing pump (150) through the upper sandstone connecting channel (18), upper sandstone connecting channel (19), and tower sandstone connecting channel (21).

[0044] A sump (100) is positioned below the bottom (12b) of a tub (12) to collect wash water. A filter (200) can filter contaminants from the wash water moving from the tub (12) to the sump (100).

[0045] Washing water sprayed through multiple spray arms (13, 14, 15) falls to the bottom (12b) of the tub (12) along with contaminants adhering to the object to be washed. Accordingly, contaminants adhering to the dishes are filtered as they pass through a filter (200) arranged to communicate with the bottom (12b) of the tub (12) and can be stored in the sump (100).

[0046] During the washing operation, the washing water can wash dishes contained in racks (16, 17a, 17b) while circulating through the sump (100), spray arms (13, 14, 15), tub (12), and filter (200).

[0047] The washing pump (150) supplies washing water stored in the sump (100) to at least one of a plurality of spray arms (13, 14, 15). The washing pump (150) may include a washing motor that generates rotational force and an impeller that is rotated by the washing motor to pump washing water. The washing pump (150) may be connected to a switching valve (130) and a washing water supply path (180).

[0048] When the washing pump (150) is driven, the washing water stored in the sump (100) flows into the washing pump (150) through the collection channel (170) and can then be pumped to the switching valve (130) through the washing water supply channel (180).

[0049] The switching valve (130) selectively supplies washing water, which is pressurized by the washing pump (150), to at least one of the lower part sand arm (13), the upper part sand arm (14), and the top part sand arm (15). The switching valve (130) can selectively connect at least one of the washing water supply path (180) and the plurality of sand arm connection paths (18, 19, 21).

[0050] The sump (100) is connected to a water supply channel (23) through which washing water supplied from an external water source flows. The water supply channel (23) may be equipped with a water supply valve (22) that controls the washing water supplied from the external water source. The water supply valve (22) can supply washing water from the external water source to the sump (100). When the water supply valve (22) is opened, the washing water supplied from the external water source can flow into the sump (100) through the water supply channel (23).

[0051] A sump (100) may be connected to a drain channel (24) that drains washing water to the outside of the dishwasher. A drain pump (25) that drains washing water within the sump (100) through the drain channel (24) may be provided in the drain channel (24). When the drain pump (25) is operated, washing water stored in the sump (100) can be drained to the outside of the case (11) through the drain channel (24).

[0052] A heater (160) for heating the washing water may be provided inside the sump (100) or in the washing pump (150). In FIG. 2, a heater (160) attached to the washing pump (150) is shown as an example.

[0053] With a dishwasher (1) configured as shown in FIGS. 1 and 2, it is difficult to see what dishes are placed inside when the door (20) is closed. However, if a camera is placed on the top of the dishwasher (1), image distortion may occur due to the short distance between the camera and the dishes.

[0054] Accordingly, the present specification allows for the placement of a reflector to photograph the dishes inside the dishwasher.

[0055] The present specification provides a structure in which a camera can photograph a rack inside a dishwasher (1) by placing a reflector inside the dishwasher (1) through a door, and a method for combining the photographed images.

[0056] Additionally, a control unit that processes video captured by a camera, checks the movement speed of a rack, outputs messages in the event of an error, and checks or controls the operation of the dishwasher (1) may be placed within the dishwasher (1). The control unit may be implemented as a microprocessor, a specific hardware chip, or software.

[0057] Below, we examine the configuration of a dishwasher equipped with a reflector. A dishwasher according to one embodiment of the present invention comprises a tub for receiving dishes and one or more reflectors positioned above the tub to reflect the dishes in the tub. A camera captures an image reflected by the reflector. The control unit of the dishwasher aligns two or more images captured by the camera to generate a single image.

[0058] In this specification, the upper side of the tub (12) may include various embodiments.

[0059] The upper side of the tub (12), which is a space where dishes are placed and washed, is a space above the rack, and since dishes are not placed in this space, various components such as cameras and mirrors can be placed in this space.

[0060] The upper side of the tub (12) indicates a space within the tub close to the tub ceiling wall. Additionally, a partition wall (35) may be optionally placed on the upper side of the tub (12). FIGS. 3 to 5 and FIGS. 7 and 8 are embodiments in which the partition wall (35) is placed. FIGS. 13 and 14 are embodiments in which the partition wall is not placed.

[0061] Additionally, the upper side of the tub (12) may include an embodiment in which a reflector (210, 220) and a camera module (300) are positioned above the ceiling wall (inner surface of the case (11)) of the tub. This refers to the embodiment of FIGS. 13 and FIGS. 14.

[0062] Additionally, the upper side of the tub may refer to the side above the case. In this case, both the case and the tub may be provided with through holes, and each may also be provided with a transparent cover. Alternatively, a separate camera accessory module that is detachable from the case may be mounted in the dishwasher. It may take the form shown in FIG. 13, which will be described later. In this embodiment as well, it is advantageous for securing internal space in the dishwasher and maintenance may be relatively easy.

[0063] First, an embodiment in which a partition wall (35) is arranged is shown in FIGS. 3 to 5.

[0064] FIGS. 3 to 5 are drawings showing the process of a dishwasher according to an embodiment of the present invention being opened and then closed by a door. For convenience of explanation, the racks are shown in a simplified structure with only the first and third racks arranged.

[0065] A mirror (210) is placed inside the dishwasher (1), and a camera (300) that photographs the mirror (210) is placed at a distance of C_Dist from the mirror (210). That is, the respective components can be arranged so that the distance between the mirror (210) and the camera (300) is greater than the working distance of the camera (300).

[0066] In the tub, a space containing a camera (300) and a washing space equipped with racks (16, 17b) can be separated by a partition (35). Additionally, the partition (35) forms a gap at the bottom of the reflector (210). Since light can pass through the gap, the camera (300) can observe the dishes. A transparent cover may be placed in the gap, and the transparent cover prevents foreign substances, water, and moisture from penetrating into the space where the camera (300) is placed.

[0067] As shown in FIG. 3, when the door (20) is pulled forward, the racks (16, 17b) are pulled out forward. Alternatively, after the door (20) is opened, the user can pull out the racks (16, 17b) by pulling them forward, respectively.

[0068] As illustrated in FIG. 4, and after placing tableware on the rack (16, 17b), when the user pushes the door (20) or the rack (16, 17b), the camera (300) captures an image reflected on the mirror (210).

[0069] The area reflected by the mirror (210) as the rack (16, 17b) moves is the C_Range (Camera Range). The camera (300) photographs the tableware placed in the C_Range through the mirror (210). That is, the camera (300) photographs the image of the tableware reflected by the mirror (210).

[0070] C_Range can be varied depending on the size of the mirror (210) or the distance between the mirror (210) and the rack (17b, 16). Since C_Range does not include the entire rack (17b, 16), the camera (300) continuously captures images reflected on the mirror (210) while the rack moves.

[0071] In Fig. 4, the image captured by the camera once is within the range of C_Range, but since the camera (300) continues to capture images as the rack moves, the entire rack (dish basket) can be captured even when its length is long (rack length Rack_Depth).

[0072] That is, when the camera (300) repeatedly captures and aligns images of size C_Range, it can obtain an image corresponding to the total length of the rack (Rack_Depth).

[0073]

[0074] FIG. 5 is a drawing showing the case where the door (20) is closed or the rack is completely stored inside the dishwasher (1). In the state of FIG. 5, the control unit of the dishwasher (1) can combine the images that the camera (300) has continuously captured to create a single image or video.

[0075] Since the camera (300) is separated from the mirror (210) by C_Dist, the distance between the camera (300) and the rack (17b, 16) is greater than C_Dist. Therefore, the dishwasher (1) can secure a sufficient shooting distance (working distance) from the camera (300) to obtain an image of the rack (17b, 16) without distortion.

[0076] As shown in FIG. 5, to confirm whether the door (20) is closed or the rack is fully stored inside the dishwasher (1), the control unit of the dishwasher (500 in FIG. 15) can confirm or detect the following in an image captured by the camera (300), or confirm that the following state has occurred: i) the end part of the rack (e.g., the handle part) or ii) a change in light when the door is closed, thereby determining that the rack is fully stored. Alternatively, the control unit of the dishwasher (1) (500 in FIG. 15) can determine that the rack is fully stored by using sensor values ​​detected by the rack sensor (340 in FIG. 15) and the door sensor (350 in FIG. 15) in the image captured by the camera (300). If a specific pattern, mark, or label is placed on the rack, the control unit of the dishwasher (1) (500 in FIG. 15) can confirm the corresponding pattern, mark, or label in the image.

[0077] The embodiments of FIGS. 3 to 5 are embodiments in which the position where the reflector (210) is placed is the upper side of the direction in which the rack placed in the tub is pulled out, and is placed at a first angle with respect to the Y-axis. And the camera (300) is positioned to photograph the reflector (210) by being spaced apart along the Z-axis with respect to the reflector (210).

[0078] For example, the first angle can be 45 degrees. The first angle is the optimal angle for the reflector (210) to reflect images of tableware, and the first angle can be changed depending on the height of the camera (300). Therefore, the first angle can be determined according to the arrangement of the camera (300), the distance between the camera (300) and the reflector (210), the height of the reflector (210), etc.

[0079]

[0080] When applying the embodiments of FIGS. 3 to 5, both the camera and the mirror are placed inside the dishwasher. Therefore, the camera device is not exposed on the exterior of the dishwasher, allowing the camera device to be configured without compromising the aesthetics. In order to minimize the occlusion caused by the stacking of dishes inside the dishwasher, it is necessary to take a bird-eye view (top-down) shot with the camera. Since the distance between the mirror (210) and the camera (300) and the distance between the mirror (210) and the rack are secured, a sufficient shooting distance can be secured.

[0081] In other words, because the working distance (distance to the object) between the dish basket and the camera is short due to the narrow internal environment of the dishwasher, the image of the object is distorted when using a wide-angle camera; however, the problem of image distortion of the object can be solved by using the mirror and camera of the present invention. That is, with an optical structure using a mirror, even a narrow-angle camera can secure a sufficient working distance.

[0082] A dishwasher (1) with a reflector optical structure according to an embodiment of the present invention does not require the use of a wide-angle lens for internal shooting, so stored items can be captured without image distortion.

[0083] Since the vertical field of view is limited by the height of the mirror when using a mirror, the dishwasher of this specification can scan the entire area of ​​the dishwasher basket by applying image stitching, a technology that combines images to capture the entire area of ​​the basket (rack) where dishes are placed inside.

[0084] The dishwasher (1) measures the movement speed of the basket (rack) for image stitching and sets the ROI by the distance traveled to align the image.

[0085] A method for measuring the basket movement speed can be applied using image analysis algorithms, computer vision (optical flow, template matching), and deep learning algorithms without using separate sensors. Additionally, the type, location, and condition of the dishware tools inside the dishwasher (1) can be recognized and determined using the scanned dishwasher basket image.

[0086] FIG. 6 is a plan view of the configuration between a mirror and a camera according to an embodiment of the present invention. It is a plan view based on the structure of FIG. 3 to 5. The distance (C_Dist) between the mirror (210) and the camera (300) can be adjusted. The distance can be adjusted according to the field of view of the camera (300) and the size of the mirror (210), and if the camera (300) is placed at the point (P) furthest from the mirror, the widest field of view can be secured.

[0087]

[0088] Next, we examine an embodiment that secures the camera's shooting distance by using two or more mirrors. For the sake of convenience of explanation, the detailed configuration of the lower part of the dishwasher is omitted.

[0089] FIG. 7 is a side view and a top view of a dishwasher with two mirrors arranged according to an embodiment of the present invention. Two mirrors (210, 220) are arranged parallel to each other on the top of the dishwasher (1), and the dotted line shows the trajectory of the rack as it passes through the two mirrors (210, 220) and is captured by the camera (300). Compared to FIG. 3 to 5, FIG. 7 increases the distance due to the two mirrors. Therefore, the shooting distance of the camera (300) becomes 2*C_Dist.

[0090] FIG. 7 is an embodiment having two mirrors. The position of the first mirror (210) can be arranged in the same way as in FIG. 3 to FIG. 5. Meanwhile, the second mirror (220) is arranged spaced apart from the first mirror (210) with respect to the Z-axis. At this time, the second mirror (220) can be arranged at a second angle with respect to the first mirror with respect to the X-axis. The camera (300) is arranged between the first mirror (210) and the second mirror (220) with respect to the Z-axis.

[0091] FIG. 7 is an embodiment in which the second angle is 0 degrees and the first reflector (210) and the second reflector (220) are arranged parallel to the X-axis.

[0092]

[0093] FIG. 8 is a side view and a top view of a dishwasher with two mirrors arranged according to another embodiment of the present invention. Two mirrors (210, 220) are arranged on the top of the dishwasher (1) so as not to be parallel with respect to the X-axis, and the dotted line shows the trajectory of the rack as it passes through the two mirrors (210, 220) and is captured by the camera (300). FIG. 8 also increases the distance due to the two mirrors compared to FIG. 3 to 5. Therefore, the shooting distance of the camera (300) becomes C_Dist1 + C_Dist2.

[0094] In FIG. 8, the position of the first mirror (210) can be arranged in the same way as in FIG. 3 to FIG. 5. Meanwhile, the second mirror (220) is arranged spaced apart from the first mirror (210) with respect to the Z-axis. At this time, the second mirror (220) can be arranged at a second angle with respect to the first mirror with respect to the X-axis. The camera (300) is arranged between the first mirror (210) and the second mirror (220) with respect to the Z-axis.

[0095] Figure 8 shows that the second angle exceeds 0 degrees, and the camera (300) is positioned to photograph the second mirror (220) at a distance from the second mirror (220).

[0096] In the embodiments of FIGS. 7 and 8, the respective components may be arranged such that the sum of the distance between the first mirror (210) and the second mirror (220) (C_Dist in FIG. 7, C_Dist1 in FIG. 8) and the distance between the camera (300) and the second mirror (220) (C_Dist in FIG. 7, C_Dist2 in FIG. 8) is greater than or equal to the shooting distance of the camera (300).

[0097]

[0098] When the above-described embodiments are applied, the distance between the camera (300) and the tableware is increased, so the lens of the camera (300) can be configured as a narrow-angle lens.

[0099] Wide-angle lenses can capture a wide field of view from a short distance, but they have the problem of obtaining distorted images of objects. That is, when a wide-angle lens camera is placed in the narrow interior space of a dishwasher, the distance to the objects is short, so the top surface of the object in the center of the camera is visible, but the side of the object at the outer edge of the camera is captured.

[0100] On the other hand, while narrow-angle lenses do not cause distortion when photographing objects, they suffer from the problem of a narrowed field of view. In order for a narrow-angle lens to secure the same field of view as a wide-angle lens, the distance between the camera and the object (shooting distance or working distance) must be increased.

[0101] Therefore, when a reflector is positioned as in the embodiments of the present invention, the image of an object is reflected through the reflector, so if the distance between the reflector and the camera is sufficiently increased, the dishware can be recognized without distortion of the image of the object even in the narrow interior space of the dishwasher.

[0102] In addition, since the reflector is placed on the upper side of the tub, the camera (300) can minimize the occlusion between the tableware by capturing the reflected image in a bird's-eye view.

[0103] As illustrated in FIGS. 3 to 8, by placing one or more optical modules, such as reflectors (mirrors) or prisms, on the top of the rack of the dishwasher (1), the camera (300) can secure a longer working distance. As a result, a narrow-angle camera (300) with a narrow field of view can be placed inside the dishwasher (1), allowing for the capture of internal images without image distortion. In other words, an optical system that uses less space can be implemented inside the dishwasher (1).

[0104] The length of the mirror can be determined inversely proportional to the working distance at which the camera (300) can take photos. Referring to the embodiment of FIGS. 7 and FIGS. 8 in which two or more mirrors are arranged, the first mirror (210) may have the longest length, and the subsequent mirrors (220) may be arranged to become progressively smaller.

[0105] The higher the height of the first mirror, the greater the VFOV (Vertical Field of View) for the rack. Additionally, the first mirror (210) can be positioned so as to be tilted 45° (45 degrees) with respect to the surface of the tableware basket, such as the rack, so that the camera (300) can take a bird's-eye view shot.

[0106] When a reflector is placed inside to capture an internal image of a home appliance as shown in FIGS. 3 to 8, a top-down view capture becomes possible. In other words, even when capturing from a low height without changing the height inside the home appliance, the phenomenon of small dishes being obscured by large dishes can be minimized.

[0107] In addition, by placing the reflector on the upper side for a top-down view, the camera's shooting distance can be extended. As a result, even a narrow-angle camera without image distortion can secure a sufficient Horizontal Field of View (HFOV).

[0108] In addition, when the length of the rack (dish basket) is long (for example, when it is long in the Z-axis direction of Fig. 1, or the length of the rack in Fig. 4, Rack_Depth), it is necessary to align the images to capture the entire thing. Therefore, when aligning parts of the dish basket reflected in the mirror to create a single image, a vertical field of view can also be secured.

[0109]

[0110] We examine an example of installing an optical system using a reflector.

[0111] FIG. 9 is a diagram showing how to set the field of view and shooting distance of a camera in correspondence with the inclination of dishes placed in a dishwasher according to one embodiment of the present invention.

[0112] The conditions for photographing tableware placed on the rack closest to the camera (the top rack, e.g., 17b in FIGS. 2 to 8) may vary depending on the size of the rack and the height of the tableware to be recognized.

[0113] For example, assume that the rack closest to the camera has a width of 51 cm (length in the X-axis direction) and a depth of 47 cm (length in the Z-axis direction), and that the height of the tableware is 5 to 30 cm. If the plate is the tableware, the diameter of the plate can be the maximum height when the plate is placed upright on the rack.

[0114] The shape of the bowl must be captured without distortion regardless of its position within the camera's field of view, and it must be possible to distinguish in the video based on the tilt value at which water begins to accumulate in the bowl. For example, if water may accumulate inside the bowl after washing when the bowl is tilted at 25 degrees or more, it must be possible to distinguish at least between cases where the bowl is tilted at 25 degrees or more and cases where it is not.

[0115] Therefore, assuming the tilt (target tilt) of the exemplary tableware in FIG. 9 is 25, the camera's field of view can be 50 degrees. And if a pinhole camera is used as an example, the camera's field of view and shooting distance can be calculated so that the tableware can be distinguished based on the perspective projection of the camera.

[0116] That is, the camera's HFOV (maximum horizontal field of view) can be calculated by multiplying the tilt value of the tableware you want to distinguish by 2. Refer to Equation 1.

[0117]

[0118] [Mathematical Formula 1]

[0119] HFOV = (Target slope of the table) x 2

[0120]

[0121] Meanwhile, the working distance (WD) of the camera for photographing tableware can be calculated as shown in Equation 2.

[0122]

[0123] [Mathematical Formula 2]

[0124]

[0125] Here, the High-FoV (HFoV) is the field of view of the camera lens, based on the horizontal field of view. The working distance corresponds to the distance between the camera and the subject (the top of the dish basket) or the sensor resolution; when a mirror is positioned by applying an embodiment of the present invention, the distance between the mirror and the camera may be an example of the working distance. Alternatively, the sum of the distance between the mirror and the rack and the distance between the mirror and the camera may be an example of the working distance.

[0126] In addition, the camera's Vertical Field of View (VFOV) is a value that influences the movement speed of the rack (dish basket) and the selection of the mirror height. To cope with the rack's fast movement speed, the camera requires a wider mirror and a VFOV.

[0127] Therefore, VFOV can be calculated as in Equation 3.

[0128] [Mathematical Formula 3]

[0129]

[0130] S can be the size of an object, for example, the width of a tableware basket (rack) to be photographed (e.g., 51 cm) as an example.

[0131] In addition, to increase precision, the sensor resolution (R) and the vertical length (Sh) of the tableware basket may be taken into account. The sensor resolution (R) is the resolution of the camera sensor, and a value such as 1920x1080 pixels is used as an example. The vertical length (Sh) of the tableware basket is the vertical length of the tableware basket to be photographed, and a value of approximately 5 cm is used as an example.

[0132] When applying these values, the shooting distance for each HFOV is calculated as shown in Table 1.

[0133] Table 1 is a diagram showing the camera's high field of view (HFOV) and the corresponding shooting distance (working distance) according to one embodiment of the present invention.

[0134] HFOV Angle of View HFOV Star Working Distance (cm) 30 9 5.17 35 80.88 40 70.06 45 61.56 50 54.68 55 48.99 60 44.17 65 40.03

[0135] This corresponds to the minimum distance between the camera and the mirror that must be secured depending on the angle of view of the camera. If there is only one mirror, the distance between the camera and the mirror must be greater than or equal to the operating distance. If there are two or more mirrors, the sum of the distance between the first mirror and the second mirror (e.g., C_Dist1 in Fig. 8) and the distance between the camera and the second mirror (e.g., C_Dist2 in Fig. 8) must be greater than or equal to the operating distance. In the structure of Fig. 7, the value of C_Dist*2 must be greater than or equal to the operating distance.

[0136] The size and position of the mirrors included in the dishwasher are determined according to the aforementioned shooting distance. The installation angle of the mirror that first reflects the rack can be set to 45 degrees with respect to the Y-axis. If two or more mirrors are arranged, the second mirror can be installed to maintain an angle parallel to the optical axis of the camera. Additionally, the height of the first mirror can be determined to a size that ensures a maximum VFOV greater than that of the camera.

[0137] In addition, the camera's FPS (Frames Per Second) can be determined to correspond to the movement speed of the rack. For example, the control unit of the dishwasher can control the camera so that the number of frames captured per second is proportional to the movement speed of the rack.

[0138] When the camera's FPS is high, the number of frames captured per second increases. When the maximum movement speed of an object is denoted as Vmax, the physical distance corresponding to 1px (pixel) is denoted as d, and the shooting time is denoted as t, the relationship between them is given by Equation 4.

[0139] [Mathematical Formula 4]

[0140] Vmax = d / t

[0141]

[0142]

[0143] The higher the camera's FPS, that is, the shorter the camera's shooting time (exposure time), the more it can respond to the fast movement speed of the cutlery basket.

[0144] FIGS. 10 and 11 are graphs comparing the object movement speed and the camera exposure time according to an embodiment of the present invention. In both FIGS. 10 and 11, the camera exposure time (seconds) (Exposure Time, s) corresponds to the x-axis.

[0145] In addition, Fig. 10 corresponds to the Y-axis when the maximum object speed is mm / s as the actual distance of 3 mm is displayed as 1 pixel, and Fig. 11 corresponds to the Y-axis when the maximum object speed is mm / s as the actual distance of 5 mm is displayed as 1 pixel. Fig. 10 is based on 100 mm / s and 30 FPS, and Fig. 11 is based on 166 mm / s and 30 FPS.

[0146] If the dish rack moves rapidly, the camera may acquire a blurry image; therefore, in order for the camera to acquire a clear image, it is necessary to limit the maximum speed of the dish rack in correspondence with the camera's shooting speed.

[0147] The maximum speed of the dish basket affects the camera's VFOV, and if there is no overlapping area between the frames generated by the camera capturing images continuously, the control unit may not be able to find an overlapping area during the process of aligning the images. Therefore, the control unit can check the movement speed of the rack and, if the rack is moving too fast, request the user to move the rack or dish basket slowly by outputting a voice message or a visual message.

[0148] FIG. 12 is a graph showing the correlation between the movement speed of a camera and a rack according to one embodiment of the present invention.

[0149] When the camera's FPS is 120, the maximum movement speed of the object is 36.14 (cm / s). When the camera's FPS is 60, the maximum movement speed of the object is 18.07 (cm / s). When the camera's FPS is 30, the maximum movement speed of the object is 9.00 (cm / s). These figures are exemplary and the present invention is not limited thereto.

[0150] When applying FIGS. 10 to 12, the control unit can check the movement speed of the rack and determine the shooting speed or interval of the camera. Alternatively, if the movement speed of the rack is fast based on the shooting speed of the camera, the control unit can output a voice message or a visual message requesting the rack's movement speed to be lowered.

[0151] FIGS. 13 and 14 are drawings showing the internal configuration of a dishwasher with two reflectors arranged according to another embodiment of the present invention.

[0152] FIG. 13 is a structure in which a transparent cover (190) is placed on the upper surface of the door (20) and the closing part, and an optical system is installed outside the upper surface of the case (11). It is a structure in which two reflectors (210, 220) and a camera (300) are placed.

[0153] FIG. 14 is a structure in which an optical system is installed within a tub (12), and the upper surface of the case (11) may be double-layered. The upper surface may be double-layered, and a transparent cover (190) may be placed on the lower side. It is a structure in which two mirrors (210, 220) and a camera (300) are arranged.

[0154] In both FIG. 13 and FIG. 14, the position of the transparent cover (190) can be such that the reflector (210) is positioned close to the door (20) so that the top rack (e.g., 17b) is stored in the reflector (210).

[0155] As illustrated in FIGS. 13 and 14, an embodiment of the upper side of the tub (12) may mean that the reflector (210, 220) and the camera module (300) are positioned above the ceiling wall of the tub (inner surface of the case (11)). A through hole is provided in the ceiling wall of the tub so that an image inside the tub can reach the camera (300). At this time, a transparent cover (190 in FIGS. 13 and 14) may be placed in the through hole.

[0156] In the case of FIGS. 13 and 14, the camera (300) may be positioned between the ceiling wall of the tub and the case. In this embodiment, the cleaning space inside the tub can be maximized, and there is an advantage of relatively simple manufacturing. Maintenance may be more advantageous in the case of a built-in dishwasher without a top cover.

[0157] In another embodiment of the present invention, the reflector (210, 220) is positioned at the top, and the camera (300) may be positioned below the top of the rack (17b) positioned at the top or on the rear wall of the tub (12).

[0158] FIG. 15 is a diagram summarizing the components of a dishwasher according to one embodiment of the present invention.

[0159] The dishwasher (1) includes a control unit (500), a camera (300), an interface unit (330), one or more mirrors (210, ..., 230), a rack sensor (340), a door sensor (350), and a functional unit (400) for performing dishwashing. For the detailed configuration of the functional unit (400), refer to FIG. 2 above.

[0160] A camera (300), one or more mirrors (210, ..., 230), and a control unit (500) may be components of an optical system (410). Depending on the embodiment, a rack sensor (340) may be included in the optical system (410) or in the functional unit (400).

[0161] The control unit (500) aligns the images captured by the camera (300) to generate video or static images of the dishes placed in the dishwasher (1).

[0162] The rack sensor (340) detects the movement speed of the racks (16, 17a, ..., 17m) and determines whether the rack is fully stored. If the rack sensor (340) is not present, the control unit (500) can calculate the movement speed or check whether the rack is stored based on the shape of the rack captured by the camera (300). To do this, a specific mark may be placed on each rack (16, 17a, ..., 17m), and the control unit (500) can calculate the movement speed of the rack from the image captured by the camera (300).

[0163] Alternatively, if the area where tableware of the racks (16, 17a, ..., 17m) is placed has a specific structure such as a mesh shape, square, or circle, the control unit (500) can check this pattern and calculate the movement speed.

[0164] The interface unit (330) outputs voice information or visual information. For example, the interface unit (330) may output a message requesting that the rack be moved slowly, a message indicating that storage is not taking place, etc. Examples of these messages include various types of messages, such as voice messages and visual messages.

[0165] One or more reflectors (210, ..., 230) may be placed inside the dishwasher (1).

[0166] The control unit (500) can detect whether storage begins after the rack is withdrawn by using a rack sensor (340) or a door sensor (350). When storage begins after the rack is withdrawn, the control unit (500) controls the camera (300) to start shooting.

[0167]

[0168] FIG. 16 is a diagram showing an image matching process according to an embodiment of the present invention. In the embodiments of FIG. 3 to 5, when racks are stored and an image reflected on a mirror (210) is captured by a camera (300), the image captured by the camera corresponds to a part of the rack (17b).

[0169] That is, when stored in the direction of the arrow, the sections captured by the camera are a, b, c, ..., k, respectively. The areas captured in each section are indicated as a1, b1, c1, ..., k1, respectively, and can be verified as corresponding rectangles. Adjacent sections overlap to a certain extent. Therefore, the control unit (500) can identify the overlapping parts and perform image stitching.

[0170]

[0171] FIG. 17 is a flowchart showing an image matching process according to one embodiment of the present invention.

[0172] The shooting ROI (Region of Interest) on the rack (dish basket) is determined by the height of the first mirror reflecting the rack. As a result, the camera has a narrow ROI, as shown in a1, b1, and c1 in Fig. 16. Therefore, to capture the entire area of ​​the rack, multiple images must be aligned.

[0173] That is, while the user takes out the rack (dish basket), places the dishes on the rack, and then stores the rack back into the tub (washing room), the camera takes pictures and sets the ROI based on the distance moved for each frame, and the control unit (500) can combine the set ROIs to create a single image. In this process, the control unit (500) can analyze the frames of the continuous video or calculate the speed of movement of each frame through a sensor (e.g., a rack sensor).

[0174] The control unit (500) may use computer vision and deep learning algorithms to analyze video frames. For more accurate speed measurement, markers for feature point extraction may be placed in specific areas of the rack. The markers may be displayed in various forms (characters, shapes) on the side of the dish basket or the handle of the basket, and can be applied in various forms.

[0175] When using computer vision, the control unit (500) can apply techniques such as optical flow and template matching.

[0176] One embodiment of the method using a sensor is that the control unit (500) obtains the movement speed or distance, etc., from an encoder installed on the wheel of the dish basket. Alternatively, the method using a sensor may obtain distance information from the rack through a distance measuring sensor (ultrasonic, ToF, Lidar, etc.). In this case, both the encoder installed on the wheel of the rack and the distance measuring sensor are embodiments of a rack sensor.

[0177] Let's examine the flow of Fig. 17 in detail. S301 to S303 correspond to the pre-processing steps. This is an image pre-processing process to increase accuracy when matching images.

[0178] The control unit (500) detects the edges of the tableware in the image through preprocessing (S303), and then performs frame differentencing and template matching to align the images (S304 to S306). Then, the control unit (500) can align all images captured by the camera (300), detect an end point, reset the area, and save the image (S308 to S310).

[0179] Each step is examined in detail. When an image captured by the camera (300) is input (S301), the control unit (500) performs correction and conversion operations (S302). Correction includes an undeistort operation to remove distortion. Conversion includes applying a specific filter for accuracy in aligning the image.

[0180] The control unit (500) can apply a filter suitable for extracting visual components of tableware from among various filters. For example, the control unit (500) can apply a Robinson compass mask to the image. In this case, the image can be converted to gray. This operation enhances the edge components of the tableware in various directions (e.g., 8 directions) to increase the accuracy of image matching.

[0181] The preprocessed image is aligned with the previously acquired image.

[0182] The image stitching process corresponds to processes S304 through S307.

[0183] To perform the image matching process, the control unit (500) calculates the movement distance of the rack for each frame, sets an area suitable for image matching, and joins them together. In this process, the control unit (500) obtains a differential image that minimizes matching errors that may occur due to changes in lighting, light reflection, or shadows, searches for matching points based on this image, and performs matching using the matching points.

[0184] That is, during the frame differencing (S304) process, the control unit (500) calculates a difference image between the previous image (previous frame) and the input new image (new frame). Then, the control unit (500) performs template matching (S305) to identify the overlapping area. After identifying the overlapping area, the control unit (500) performs image alignment (image stitching) through blending (S306).

[0185] Then, the control unit (500) checks whether it is the last image (S307). Whether it is the last image can be checked by whether the rack storage is complete or whether the door is closed, etc.

[0186] If it is not the last image, receive a new image as input and perform processes S301 through S307.

[0187] In the case of the last image (S307), the control unit (500) performs post-processing. Processes S308 through S310 are performed. The control unit (500) adjusts the ROI of the aligned image to finally generate a bird's-eye view, i.e., a top-down view, of the rack. Then, the control unit (500) recognizes the shape of the dish basket and detects the handle area to complete the image alignment work for the area of ​​the dish basket (i.e., the area where the dishes are placed).

[0188] More specifically, the control unit (500) detects an end point in the entire aligned image (S308), resets the area (S309), and saves the image (S310).

[0189] In the process of FIG. 17, the control unit (500) can sequentially perform alignment on a number of images acquired after the rack is stored. For example, in the example of FIG. 16 above, when the camera captures sections a, b, and c and finally completes capturing up to section k, a total of k images are acquired.

[0190] The control unit (500) can sequentially apply the process of FIG. 17 to the image for section a and the image for section b, and then finally perform alignment on all images up to section k. Here, the image of section k corresponds to the last image of S307.

[0191] Alternatively, the control unit (500) may perform preprocessing S301 to S303 for each image, and then perform image matching S304 to S307.

[0192] The control unit (500) can match images by searching for matching points for consecutive images.

[0193]

[0194] FIG. 18 is a diagram showing the operation flow of an optical system of a dishwasher according to one embodiment of the present invention.

[0195] In the entire process, when storage begins after the rack is withdrawn, the control unit (500) controls the camera (300) to capture an image reflected by the mirror (210). Then, the control unit (500) controls the camera (300) to capture the mirror (210) until storage is finished, and then combines two or more images captured by the camera (300) to create a single image.

[0196] We will examine this in detail.

[0197] The door sensor (350) detects the opening of the door (20) (S311). The control unit (500) turns on the power of the camera (300) (S312). Turning on the power includes the camera (300) entering a ready state so that it can take pictures. Then, the withdrawal of the rack (dish basket) begins (S313). The withdrawal of the rack can be detected by the rack sensor (340).

[0198] The control unit (500) checks whether the rack has been pulled out completely (S314). If the rack has not been pulled out completely, the control unit (500) instructs the user to pull out the rack (dish basket) through the interface unit (330) after a certain amount of time has elapsed (S315).

[0199] Meanwhile, if the rack is fully withdrawn in S314, wait until the storage of the rack begins (S316).

[0200] When storage of the rack begins, the control unit (500) controls the camera (300) to start shooting (S317). When storage of the rack begins, the camera (300) captures an image reflected by a mirror. The control unit (500) checks whether storage of the rack is completed (S318). The control unit (500) can check whether storage of the rack is completed through a rack sensor (340) or a door sensor (350), etc. If storage of the rack is not completed, the control unit (500) generates a warning message through the interface unit (330) to instruct the user to retrieve and store the rack (dish basket) again (S321).

[0201] When storage of the rack is completed, the control unit (500) checks whether the number of frames captured by the camera (300) during the storage process meets a reference value (S320). The reference value is determined based on the section that the camera (300) can capture and the total length of the rack. If the length of the rack is 50 cm and the section that the camera (300) can capture is 10 cm, but the number of captured frames is 4, then a problem has occurred where the entire rack was not captured.

[0202] Therefore, if the standard value is not met, the control unit (500) generates a warning message through the interface unit (330) to instruct the user to retrieve and store the rack (dish basket) again (S321).

[0203] When the number of frames captured by the camera (300) in S320 meets the threshold value, the control unit (500) terminates camera shooting (S322).

[0204] When multiple racks are placed in the dishwasher (1), the control unit (500) checks whether image collection is completed for both the upper rack and the lower rack (S323). If image collection is completed, the control unit (500) starts an algorithm operation for image matching (S326). The control unit (500) performs the image matching operation of FIG. 17.

[0205] If image collection for some racks is not completed, the control unit (500) generates a guidance message to the user through the interface unit (330) regarding the racks (dish baskets) that have not been collected (S324). Even in this case, if the user performs a washing operation (S325), the control unit (500) starts an algorithm operation for image alignment (S326).

[0206] In the process of Fig. 18, some steps may be omitted, and some steps may be added or repeated during the operation of the dishwasher (1).

[0207] As illustrated in FIG. 18, the control unit (500) may output a message requesting the rack to be withdrawn if the rack is not withdrawn after the operation of the dishwasher (1) is completed or after a storage error occurs (S315, S321). The message output is provided by the interface unit (330) via voice / text, etc.

[0208] In addition, the control unit (500) can output a message requesting to withdraw the rack even if the number of captured frames of the image acquired after storage is completed does not meet the reference value (S320) (S320, S321).

[0209]

[0210] Next, we examine the process in which the control unit (500) adjusts the FPS of the camera (300) according to the storage speed of the rack, or selects and aligns only some of the images captured by the camera (300).

[0211] FIG. 19 is a diagram showing the process of determining the number of shooting frames per second of a camera in correspondence with the storage speed of a rack according to one embodiment of the present invention.

[0212] The control unit (500) can check the storage speed of the rack through the rack sensor (340) or the door sensor (350). Alternatively, the control unit (500) can check the movement speed of the rack from images captured by the camera (300).

[0213] That is, the control unit (500) checks the storage speed of the rack (S371) and then determines the number of frames per second (FPS) of the camera (300) corresponding to the storage speed of the rack (S372).

[0214] For example, if the rack storage speed is fast, the control unit (300) increases the number of frames per second (FPS) of the camera (300). If the rack storage speed is slow, the control unit (300) decreases the number of frames per second (FPS) of the camera (300).

[0215] And the control unit (300) controls the shooting speed of the camera according to the determined number of shooting frames per second (S373).

[0216] FIG. 19 controls the shooting speed of the camera during the process of storing the rack, so that the camera (300) can secure an image corresponding to the storage speed of the rack.

[0217]

[0218] FIG. 20 is a drawing showing a process of selecting and aligning some of the images captured by a camera in correspondence with the storage speed of a rack according to one embodiment of the present invention.

[0219] The control unit (500) can check the storage speed of the rack through the rack sensor (340) or the door sensor (350). Alternatively, the control unit (500) can check the movement speed of the rack from images captured by the camera (300).

[0220] That is, the control unit (500) checks the storage speed of the rack (S375) and selects some of the images captured by the camera (300) in correspondence with the storage speed of the rack (S376).

[0221] For example, if the camera (300) captures at 50 FPS per second, but the rack storage speed is slow and 20 FPS is sufficient to match the rack storage speed, the control unit (500) selects only some of the 50 frames secured per second. For example, the control unit (500) can select 35 images, which is more than 20 FPS. Then, the control unit (500) performs an image matching operation on the selected images (S377).

[0222] When applying the embodiment of FIG. 20, if the camera (300) captures an excessive number of images compared to the movement speed of the rack, it is possible to prevent the time required to align them.

[0223] Summarizing FIGS. 19 and 20, the control unit (500) can quickly obtain accurate images by determining the number of frames per second of the camera (300) according to the storage speed of the rack or by selecting only some of the images captured by the camera (300).

[0224] Although it has been described that all components constituting an embodiment of the present invention are combined or operate as a single unit, the present invention is not necessarily limited to such an embodiment, and within the scope of the purpose of the present invention, all components may be selectively combined in one or more ways to operate. Furthermore, while all components may each be implemented as a single independent piece of hardware, some or all of the components may be selectively combined to be implemented as a computer program having a program module that performs some or all of the combined functions on one or more pieces of hardware. The codes and code segments constituting the computer program can be easily inferred by those skilled in the art of the present invention. An embodiment of the present invention may be implemented by storing such a computer-readable storage medium and reading and executing it by a computer. The storage medium for the computer program includes a magnetic recording medium, an optical recording medium, and a storage medium including a semiconductor recording element. Additionally, a computer program implementing an embodiment of the present invention includes a program module that is transmitted in real time through an external device.

[0225] Although the present invention has been described above with reference to embodiments, various changes and modifications can be made by those skilled in the art. Therefore, it should be understood that such changes and modifications are included within the scope of the present invention as long as they do not depart from the scope of the invention.

[0226]

[0227] -Explanation of the symbols-

[0228] 1: Dishwasher 12: Tub

[0229] 210, 220: Reflector 300: Camera

[0230] 340: Rack sensor 500: Control unit

[0231]

[0232]

Claims

1. A tub for accommodating tableware; One or more reflectors positioned on the upper side of the above tub; A camera that captures an image reflected by the above mirror; and A dishwasher comprising a control unit that combines two or more images captured by the above camera to generate a single image.

2. In Paragraph 1, The above reflector is positioned at a first angle with respect to the Y-axis on the upper side of the direction in which the rack placed in the tub is withdrawn, and A dishwasher in which the camera is positioned to photograph the mirror along the Z-axis relative to the mirror.

3. In Paragraph 2, A dishwasher characterized in that the distance between the camera and the reflector is greater than or equal to the shooting distance of the camera.

4. In Paragraph 1, The above reflectors are two or more, and Among the above reflectors, the first reflector is positioned at a first angle with respect to the Y-axis on the upper side of the direction in which the rack placed in the tub is pulled out, and Among the above mirrors, the second mirror is spaced apart from the first mirror with respect to the Z-axis, and the second mirror is positioned at a second angle with respect to the X-axis. A dishwasher in which the camera is positioned between the first mirror and the second mirror with respect to the Z-axis according to the second angle to photograph the second mirror.

5. In Paragraph 4, A dishwasher characterized in that the sum of the distances between the first mirror and the second mirror and the distance between the camera and the second mirror is greater than or equal to the shooting distance of the camera.

6. In Paragraph 4, If the above second angle is 0 degrees, A dishwasher in which the first reflector and the second reflector are arranged parallel to each other with respect to the X-axis.

7. In Paragraph 4, If the above second angle is greater than 0 degrees, A dishwasher in which the camera is positioned to photograph the second mirror at a distance from the second mirror.

8. In Paragraph 1, One or more racks are placed in the above tub, and A dishwasher in which the control unit controls the camera to start shooting when storage begins after the rack is pulled out.

9. In Paragraph 8, When storage begins after the above rack is withdrawn, A dishwasher in which the control unit determines the number of frames per second of the camera according to the storage speed of the rack, or selects only some of the images captured by the camera.

10. In Paragraph 8, If the rack is not withdrawn after the operation of the dishwasher is completed or after a storage error occurs, or if the number of captured frames of the image acquired after the storage is completed does not meet the standard value, A dishwasher in which the above control unit outputs a message requesting the removal of the rack.

11. In Paragraph 8, A dishwasher in which the control unit performs preprocessing on a plurality of images captured by the camera and searches for matching points on consecutive images to join the images.

12. In Paragraph 11, The control unit detects the edges of the tableware in the image through the preprocessing, and then aligns the image by performing frame differentiation and template matching. A dishwasher in which the control unit aligns all images captured by the camera, detects an endpoint, resets the area, and saves the image.

13. A dishwasher comprising a tub for receiving dishes, one or more racks disposed in the tub, one or more reflectors, a camera, and a control unit disposed on the upper side of the tub, When storage begins after the rack is withdrawn, the control unit controls the camera to capture an image reflected by the mirror; The above control unit controls the camera to photograph the reflector until the storage is completed; and A method for acquiring an image of tableware using a mirror, wherein the control unit comprises the step of aligning two or more images captured by a camera to generate a single image.

14. In Paragraph 13, When storage begins after the above rack is withdrawn, A method for acquiring an image of tableware using a reflector, wherein the control unit further includes the step of determining the number of frames per second of the camera according to the storage speed of the rack or selecting only some of the images captured by the camera.

15. In Paragraph 13, If the rack is not withdrawn after the operation of the dishwasher is completed or after a storage error occurs, or if the number of captured frames of the image acquired after the storage is completed does not meet the standard value, A method for acquiring an image of tableware using a reflector, wherein the control unit further includes the step of outputting a message requesting the rack to be withdrawn.

Images