Clothes processing device, reminding method after clothes processing, equipment and storage medium

Abstract

The invention provides a clothes processing device, a reminding method after clothes processing, equipment and a storage medium, and the method comprises the steps: obtaining an in-cylinder space image of the clothes processing device, determining an in-cylinder state through the in-cylinder space image, and determining whether to output reminding information or not according to the in-cylinder state. The actual implementation of the clothes processing device does not depend on the matching of the door body state of the clothes processing device, the application possibility is expanded, and the clothes processing device is more favorable for being popularized and used in a large range. The state in the drum can be recognized through the image of the space in the drum, left clothes can be recognized, foreign matter can be recognized, a user can be reminded in time, the problems of wrinkles, peculiar smells, molds and the like caused by the fact that the clothes are left in the clothes treatment device can be avoided, the clothes can be prevented from being polluted, and the washing machine and the clothes can be prevented from being damaged; and the use experience of the user is greatly improved.

Description

Technical Field

[0001] This invention relates to the field of clothing processing device control technology, and particularly to clothing processing devices and methods, equipment and storage media for reminding users after clothing processing. Background Technology

[0002] Washing machines are an essential appliance in modern households. When the washing machine finishes washing, it will sound an alarm to remind the user to remove the clothes.

[0003] If users forget to take clothes out of the washing machine or leave some clothes inside, and the washing machine cannot remind them, the clothes will remain in the washing machine for too long. Because the washing machine is a tightly sealed container, clothes that are not dried in time or left for a long time will not only easily develop wrinkles, but will also easily develop odors and mold in the sealed environment.

[0004] To implement a drum-clothes reminder, Chinese patent application number CN202010582339.2 discloses a method, device, and clothing handling device for drum-clothes reminders. This method monitors image information inside the drum to confirm whether clothing is left inside, and then triggers an alarm via a linked mobile terminal to alert the user upon confirmation of clothing. However, this solution relies on the coordination of the door status of the clothing handling device, which has significant limitations in practical applications. Furthermore, this solution can only detect leftover clothing and cannot identify foreign objects inside the drum, potentially leading to clothing contamination or even damage to the washing machine and the clothes. Summary of the Invention

[0005] To achieve the above-mentioned objectives and other advantages of the present invention, a first objective of the present invention is to provide a reminder method after clothing processing, configured in a clothing processing device after clothing processing has ended, comprising the following steps:

[0006] Acquire an image of the space inside the garment processing device's drum;

[0007] The internal state of the cylinder is determined by the image of the internal space.

[0008] The output reminder message is determined based on the state inside the cylinder.

[0009] Furthermore, the internal space of the cylinder is configured to include at least the internal wall of the cylinder.

[0010] Furthermore, the image of the space inside the cylinder is a depth image, which is an image in which the depth values ​​of each point in the space inside the cylinder are collected by a depth image acquisition device as pixel values.

[0011] Furthermore, the step of determining the internal state of the cylinder through the internal space image includes:

[0012] The depth image is used to identify whether there are clothing and / or foreign objects inside the cylinder;

[0013] The state inside the tube is determined based on the identification results; wherein, the state inside the tube is an empty tube or a tube with clothing left inside and / or a tube with foreign objects inside.

[0014] Further, the step of identifying whether there are clothing and / or foreign objects inside the cylinder through the depth image includes:

[0015] The contours of objects inside the cylinder are identified using the depth image;

[0016] The identified contours are compared with preset contours to determine the category to which the object belongs.

[0017] Further, the step of recognizing the outline of the object inside the cylinder through the depth image includes:

[0018] Obtain the background of the space inside the cylinder;

[0019] The object outline region is obtained by subtracting the background of the space inside the cylinder from the current depth image;

[0020] The object's outline is obtained by segmenting the object's contour region using depth values.

[0021] Furthermore, the step of obtaining the background of the space inside the cylinder includes:

[0022] Control the inner cylinder to stop at a preset position so that the current inner cylinder space background is the pre-stored inner cylinder space background;

[0023] or,

[0024] Obtain the current depth image of the space inside the cylinder;

[0025] After controlling the inner cylinder to rotate, it returns to the position where the previous image of the depth of the inner cylinder was acquired, so as to re-acquire the image of the depth of the inner cylinder.

[0026] The background of the internal space was determined by taking two images of the depth inside the cylinder.

[0027] Furthermore, the step of determining the background of the internal space through two consecutive acquisitions of the internal space depth images includes:

[0028] The depth values ​​of each point in the two consecutive images of the depth of the space inside the cylinder are subtracted.

[0029] Compare the difference to the threshold;

[0030] The background region inside the cylinder was determined by comparing the results.

[0031] Furthermore, the step of segmenting the object contour region using depth values ​​includes:

[0032] Scan the outline region of the object line by line;

[0033] Detect the four neighbors of each pixel in each row;

[0034] Determine if there are any pixels in the four neighboring regions whose depth value difference is greater than a preset value;

[0035] Based on the judgment result, the edge points of the object contour are determined to segment the object contour region and obtain the object contour.

[0036] Furthermore, the step of comparing the identified contour with a preset contour includes:

[0037] Calculate the similarity between the identified contour and the preset contour;

[0038] The category to which an object belongs is determined by the similarity calculation results.

[0039] Furthermore, the reminder information includes reminders about clothing left in the container and / or reminders about foreign objects in the container.

[0040] Furthermore, it also includes the following steps:

[0041] After a preset time interval, another image of the space inside the clothing processing device drum is acquired to determine whether clothing and / or foreign objects are still present inside the drum.

[0042] If it is determined that there are still clothes and / or foreign objects inside the container, a corresponding reminder message will be output.

[0043] Furthermore, it also includes the following steps:

[0044] After confirming that the clothes in the drum have been removed, another image of the space inside the clothes processing device drum is acquired to determine if there are any foreign objects inside the drum.

[0045] When a foreign object is detected inside the cylinder, an alert message indicating that a foreign object is present inside the cylinder is output.

[0046] A second objective of the present invention is to provide a garment processing apparatus that applies the above-described method.

[0047] A third object of the present invention is to provide an electronic device comprising: a memory having program code stored thereon; and a processor connected to the memory, wherein the above-described method is implemented when the program code is executed by the processor.

[0048] A fourth objective of this invention is to provide a computer-readable storage medium having program instructions stored thereon, which, when executed, implement the method described above.

[0049] Compared with the prior art, the beneficial effects of the present invention are:

[0050] This invention provides a clothing processing device and a method, apparatus, and storage medium for reminding users after clothing processing. It acquires an image of the space inside the drum of the clothing processing device, determines the state inside the drum based on the image, and determines whether to output a reminder message based on the state. The actual implementation of this invention does not rely on the door state of the clothing processing device, expanding its application possibilities and facilitating wider adoption. This invention can identify the state inside the drum through an image of the space, not only identifying leftover clothing but also foreign objects, and promptly reminding the user. This avoids problems such as wrinkles, odors, and mold caused by clothing left inside the clothing processing device, and also prevents clothing contamination and damage to the washing machine and clothing, significantly improving the user experience.

[0051] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it according to the contents of the specification, the preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. Specific embodiments of the present invention are given in detail below with reference to the accompanying drawings. Attached Figure Description

[0052] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0053] Figure 1 This is a flowchart of the reminder method after clothing treatment in Example 1;

[0054] Figure 2 This is a flowchart of Example 1 for determining the internal state of the cylinder using the internal space image;

[0055] Figure 3 This is a flowchart illustrating the process of identifying whether there are clothes and / or foreign objects inside the cylinder using the depth image in Example 1.

[0056] Figure 4 This is a flowchart of the process for recognizing the outline of an object inside a cylinder using the depth image in Example 1;

[0057] Figure 5 This is a flowchart of the process for obtaining the background of the space inside the cylinder in Example 1;

[0058] Figure 6 This is a flowchart of Example 1 showing the process of determining the background of the internal space by taking two images of the depth inside the cylinder.

[0059] Figure 7 This is a flowchart of Example 1 showing the segmentation of the object's contour region using depth values;

[0060] Figure 8This is a flowchart of Example 1 showing the comparison between the identified contour and the preset contour;

[0061] Figure 9 This is a flowchart of the clothing and / or foreign object removal detection process in Example 1;

[0062] Figure 10 This is a flowchart of the foreign object detection process after removing clothing in Example 1;

[0063] Figure 11 A schematic diagram of a depth image acquisition device capturing images of the inner wall of the bottom cylinder.

[0064] Figure 12 A schematic diagram of a depth image acquisition device capturing images of the inner wall of the side cylinder.

[0065] Figure 13 This is a schematic diagram of the electronic device in Example 3;

[0066] Figure 14 This is a schematic diagram of the storage medium in Example 4.

[0067] In the diagram: 1. Inner cylinder; 2. Image acquisition device. Detailed Implementation

[0068] The present invention will now be further described with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.

[0069] Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this invention.

[0070] The drawing numbers in this application are only used to distinguish the steps in the scheme and are not used to limit the execution order of the steps. The specific execution order is as described in the specification.

[0071] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0072] After the garment processing device processes the clothes, users may forget to remove them, or they may forget to remove the remaining clothes after removing some. Clothes may also stick to the inner wall of the processing drum, making them unnoticed. In daily use, besides clothes, items left inside the garment processing device, such as coins, keys, and crumpled paper, can also be left inside. Coins and keys can damage the clothes and the processing device, while crumpled paper dissolves in water and sticks to the clothes, causing secondary contamination. The garment processing reminder method provided in Embodiment 1 of this invention can not only detect clothes left in the drum but also foreign objects. This avoids problems such as wrinkles, odors, and mold caused by clothes left in the processing device, and also prevents clothing contamination and damage to the washing machine and clothes, significantly improving the user experience.

[0073] Figure 1 This is a flowchart of a reminder method after clothing processing provided in Embodiment 1 of the present invention. Embodiment 1 of the present invention is applicable to a clothing processing device that, after processing clothing, detects whether there are clothes and / or foreign objects in the drum and issues a reminder based on the detection result. The clothing processing by the clothing processing device includes, but is not limited to, washing, drying, and care of clothing. Taking a washing machine as an example, the end of clothing processing by the clothing processing device means the washing machine has completed the washing process; taking a dryer as an example, the end of clothing processing by the clothing processing device means the dryer has completed the drying process.

[0074] This method can be executed by the main control unit of the garment processing device. The main control unit can be implemented in the form of software and / or hardware, and is generally integrated into any electronic device with network communication capabilities, such as a mobile terminal, PC, or server.

[0075] Example 1

[0076] A method for providing a reminder after clothing processing is completed by a clothing processing device. The clothing processing device performs actions on the clothing, including but not limited to washing, drying, and care. For example, if the clothing processing device is configured as a washing machine, the completion of clothing processing by the device signifies the washing machine has finished the washing process; if the clothing processing device is configured as a dryer, the completion of clothing processing by the device signifies the dryer has finished the drying process. Figure 1 As shown, the method includes the following steps:

[0077] S1. Obtain an image of the space inside the garment processing device drum;

[0078] In this embodiment, an image acquisition device is installed inside the garment processing device, facing inwards from the inner drum used for loading, washing, drying, and caring for garments. Optionally, the image acquisition device includes a camera, a light sensor, etc. This image acquisition device is used to acquire images of the space inside the garment processing device's drum. Specifically, after the garment processing device finishes processing the garments, the image acquisition device is activated to acquire images of the space inside the drum. One or more image acquisition devices can be deployed depending on their properties, installation location, and actual needs; the specific number can be adjusted according to the actual application.

[0079] In practical use, after the user has taken all or part of the clothes, and after the clothes processing device has completed the processing, the door of the clothes processing device may not be closed. By collecting images of the space inside the tube, the state inside the tube can be detected, which expands the possibilities of application.

[0080] This garment processing device features image recognition capabilities. It uses image recognition technology to detect whether any objects are left inside the garment processing drum. If an object is found, it outputs a reminder message to the user. Optionally, this image recognition process can be integrated into an image acquisition unit or performed by the garment processing device's controller.

[0081] When acquiring images of the space inside the garment processing device, this image acquisition device can capture either an overall image of the inner drum or a partial image of the inner drum. The specific acquisition range is determined based on the design and installation location of the garment processing device and the image acquisition device, the computational load of image processing, and actual requirements.

[0082] In some preferred embodiments, the image acquisition device is fixedly installed inside the door glass of the clothing handling device.

[0083] In some alternative embodiments, the image acquisition device is fixedly installed inside the garment processing device drum or at a door seal, etc.

[0084] This invention does not limit the specific installation location of the image acquisition device. When using the image acquisition device to acquire images of the space inside the tube, this invention does not rely on the status of the door of the clothing handling device; that is, regardless of whether the door is closed or not, the state inside the tube can be detected through the acquired images of the space inside the tube, thus expanding the possibilities for application.

[0085] After the clothes are processed by the garment processing device, most of the clothes may be at the bottom of the inner drum, while a small portion of the clothes may be stuck to the inner wall of the garment processing device.

[0086] In some embodiments, the internal space of the cylinder is configured to include at least an internal wall. This internal wall includes the overall internal wall of the inner cylinder, the internal wall located in the bottom region of the inner cylinder, the internal wall located in the side region of the inner cylinder, etc. Figure 11 As shown, image acquisition device 2 acquires images of the inner wall of the inner cylinder 1 located in the area indicated by the arrow. Figure 12 As shown, image acquisition device 2 acquires images of the inner wall of the inner cylinder 1 located in the area indicated by the arrow. The specific location and size of the inner wall to be acquired can be selected according to actual needs or circumstances.

[0087] To accurately identify foreign objects inside the drum and small garments adhering to the inner wall of the garment handling device, high-precision measurements of the internal space image are required. In some embodiments, the internal space image is a depth image. Specifically, the depth image is an image where the depth values ​​of each point in the internal space are captured by a depth image acquisition device as pixel values. It directly reflects the geometry of the visible surface of the object.

[0088] Alternatively, techniques for acquiring depth images include Time-of-Flight (ToF), binocular vision, and structured light.

[0089] To accurately detect objects in a short time, unaffected by humidity, air pressure, and temperature, and suitable for use in garment handling devices, as well as for detecting near and far objects of various shapes and sizes without depending on the door status of the garment handling device, a ToF sensor is employed in some preferred embodiments. This sensor offers high accuracy, fast measurement speed, long measurement distance, and low cost.

[0090] ToF (Time-of-Flight) sensors use tiny emitters to emit infrared light or laser light, which bounces off any object and returns to the sensor. Based on the time difference between the emission of light and its return to the sensor, the sensor can measure the distance between the object and the sensor.

[0091] ToF sensors can provide depth information, which, when combined with a camera, can produce 3D images, enabling the detection of objects inside the garment processing device.

[0092] S2. Determine the internal state of the cylinder using the internal space image;

[0093] In some embodiments, such as Figure 2 As shown, the step of determining the internal state of the cylinder through the internal space image includes:

[0094] S20. Identify whether there are clothes and / or foreign objects inside the cylinder using the depth image;

[0095] Foreign objects refer to objects other than clothing and objects that are paired with clothing (such as zippers, buttons, etc., which are not detached from the clothing body; when they are detached from the clothing body, they can be considered foreign objects). This embodiment does not specifically limit the type of clothing left inside the tube.

[0096] Optionally, depth images can be used to first identify whether there are any objects inside the cylinder. If an object is identified, it can then be determined whether the object is clothing. If it is not clothing, it can be directly identified as a foreign object.

[0097] Optionally, depth images can be used to first identify whether there are objects inside the cylinder. If an object is identified, it can then be determined whether it is clothing or a foreign object. The final identification result of the object can be determined based on the judgment result.

[0098] The specific depth image processing procedure can be set according to the actual image processing capabilities and actual needs.

[0099] S21. Determine the state inside the cylinder based on the identification results.

[0100] Based on the depth image recognition results, the state inside the tube is determined to be either empty, containing clothing, or containing foreign objects. Specifically, if no object is detected inside the tube by the depth image, the tube is determined to be empty; if clothing is detected inside the tube by the depth image, the tube contains clothing; if foreign objects are detected inside the tube by the depth image, the tube contains foreign objects; and if both clothing and foreign objects are detected inside the tube by the depth image, the tube contains both clothing and foreign objects.

[0101] In some embodiments, such as Figure 3 As shown, the step of identifying whether there are clothes and / or foreign objects inside the cylinder through the depth image includes:

[0102] S200: Identify the outline of the object inside the cylinder using the depth image;

[0103] Since clothing and foreign objects both have specific outlines inside the tube, the outline of the object inside the tube can be identified through depth image, and then the specific type of the object can be determined based on the identified outline, that is, whether it is clothing or foreign object.

[0104] S201. Compare the identified contour with the preset contour to obtain the category to which the object belongs.

[0105] Optionally, a preset outline, including clothing outlines, can be pre-stored in the memory of the clothing processing device. The clothing outlines can be outlines of different types of clothing. The identified outline is compared with the preset clothing outlines. If the outlines match, the object is determined to be clothing; if the outlines do not match, the object is determined to be a foreign object.

[0106] To improve the accuracy of foreign object identification, the clothing processing device optionally stores preset contours in its memory, including foreign object contours. These contours can be those of objects commonly placed inside clothing. The identified contour is compared with the preset clothing contours. If they match, the object is identified as clothing. If they do not match, the device further compares the identified contour with the preset foreign object contours. If they match, the corresponding foreign object is output; otherwise, the object is identified as a foreign object. Contours that do not match the clothing contour are then matched with the foreign object contour to further determine their category, thus making the identification more accurate.

[0107] Alternatively, the identified contour can be compared with preset clothing and foreign object contours, and the category of the object can be determined based on the matching results. This embodiment does not specifically limit the specific comparison process of the contours.

[0108] In some embodiments, such as Figure 4 As shown, the step of recognizing the outline of an object inside the cylinder using the depth image includes:

[0109] S210, Obtain the background of the space inside the cylinder;

[0110] To accurately identify the outline of the object inside the cylinder, it is necessary to distinguish between the internal space background and the object outline region in the depth image. This internal space background can be a pre-stored internal space background. This pre-stored internal space background is the background captured by the image acquisition device when the inner cylinder is stationary in a certain position and the cylinder is empty. At this point, the inner cylinder must be stationary in the same position as when the pre-stored internal space background was captured, so that the internal space background in the depth image captured by the image acquisition device at this time is consistent with the pre-stored internal space background.

[0111] The background of the space inside the cylinder can also be identified from the depth images acquired before and after the inner cylinder is rotated to the same position by the image acquisition device.

[0112] S220. Subtract the background of the space inside the cylinder from the current depth image to obtain the object outline region;

[0113] Object detection is performed by distinguishing the background space inside the cylinder from the object contour region through background subtraction. Optionally, a background subtraction method based on median filtering is used to subtract the background space inside the cylinder from the current depth image to obtain the object contour region.

[0114] S230. The object contour region is segmented by depth value to obtain the object contour.

[0115] Because the depth values ​​of the object and the background space inside the cylinder differ, the object's outline area can be segmented by using the depth values ​​to obtain the boundary line between the background space inside the cylinder and the object, thus obtaining the object's outline.

[0116] In some optional embodiments, the step of obtaining the background of the space inside the cylinder includes:

[0117] Control the inner cylinder to stop at a preset position so that the current inner cylinder space background is the pre-stored inner cylinder space background;

[0118] Because the inner drum of a garment processing device typically contains components such as lifting ribs, different inner drum spatial backgrounds are generated depending on the position the inner drum is in. In this embodiment, the inner drum spatial background is a pre-stored inner drum spatial background. This pre-stored inner drum spatial background is the inner drum spatial background captured by the image acquisition device when the inner drum is in an empty state at a certain position. At this time, it is required that the inner drum is in the same position when the pre-stored inner drum spatial background is captured, so that the inner drum spatial background in the depth image captured by the image acquisition device at this time is consistent with the pre-stored inner drum spatial background, avoiding recognition errors caused by different backgrounds.

[0119] When the image acquisition device is installed on the door glass, in actual use, the door may not be closed after the clothing handling device has finished processing the clothing, and the degree of opening of the door is not fixed. To adapt to different usage scenarios, the background of the space inside the cylinder can also be identified from the depth images captured by the image acquisition device before and after the inner cylinder is rotated to the same position. In some optional embodiments, such as Figure 5 As shown, the step of obtaining the background of the space inside the cylinder includes:

[0120] S211. Obtain the current depth image of the space inside the cylinder;

[0121] S212. Control the inner cylinder to rotate and return to the position where the previous image of the depth of the inner cylinder was acquired, so as to re-acquire the image of the depth of the inner cylinder.

[0122] Optionally, the inner drum can be controlled to rotate at a preset speed to prevent damage to clothes, the garment handling device, and the foreign objects themselves from objects inside the drum. This preset speed is relatively slow, lower than the rotation speed during normal washing.

[0123] S213. Determine the background of the space inside the cylinder by using the depth images of the space inside the cylinder acquired in two separate acquisitions.

[0124] Since the pixel values ​​of the background space inside the cylinder usually do not change, that is, they are relatively stable over a long period of time, the background space inside the cylinder in the depth image can be distinguished by the unchanging pixel values ​​in the two depth images acquired before and after.

[0125] In some embodiments, such as Figure 6 As shown, the step of determining the background of the internal space by acquiring two images of the internal space depth from the cylinder includes:

[0126] S221. Subtract the depth values ​​of each point in the two consecutive depth images of the space inside the cylinder.

[0127] Since the pixel value of each point in the depth image is the depth value, and the pixel value of the background inside the cylinder usually does not change, the background inside the cylinder and the object outline can be distinguished by the change in depth value in the depth image of the same location under different states.

[0128] S222. Compare the difference with the threshold;

[0129] To avoid errors in depth image acquisition by the image acquisition unit affecting the accuracy of distinguishing between the background and object contour areas inside the tube, a threshold greater than zero is set and compared with the calculated difference. This threshold can be set according to actual needs.

[0130] S223. Determine the background area inside the cylinder by comparing the results.

[0131] Specifically, if the difference is not greater than the threshold, the point is determined to be a background point in the space inside the cylinder;

[0132] If the difference is greater than the threshold, the point is determined to be an object contour point;

[0133] All background points within the tube form the background area within the tube.

[0134] In some embodiments, such as Figure 7 As shown, the step of segmenting the object contour region using depth values ​​includes:

[0135] S231. Scan the outline region of the object line by line;

[0136] It should be noted that the outline area of ​​an object can also be scanned in other ways, such as scanning column by column, scanning multiple rows or columns at the same time, etc.

[0137] S232. Detect the four neighborhoods of each pixel in each row;

[0138] In this context, the four-neighborhood of a pixel refers to the set of four horizontally and vertically adjacent pixels of pixel p.

[0139] It should be noted that, depending on the requirements, the eight-neighborhood, diagonal neighborhood, and m-adjacency of a pixel can also be detected.

[0140] S233. Determine whether there are pixels in the four neighboring regions whose depth value difference is greater than a preset value;

[0141] Since depth images are composed of basic unit pixels, there are certain relationships between pixels, including pixel neighborhood, adjacency, connectivity, and distance between pixels. By analyzing the differences between four neighboring pixels, the edge points of an object can be identified, that is, the boundaries between the object and the background space within the image.

[0142] S234. Determine the object contour edge points based on the judgment result to segment the object contour region and obtain the object contour.

[0143] Specifically, if there is a pixel in the four neighboring regions whose depth value difference is greater than a preset value, then the pixel is the edge point of the object contour.

[0144] If there are no pixels in the four neighboring regions whose depth difference is greater than the preset value, then there are no edge points in the four neighboring pixels. Then the four neighboring regions of the next pixel are detected, and the above steps are repeated until all pixels have been detected.

[0145] In some embodiments, such as Figure 8 As shown, the step of comparing the identified contour with the preset contour includes:

[0146] S240. Calculate the similarity between the identified contour and the preset contour;

[0147] Optionally, the similarity between the identified contours can be obtained by calculating the Hausdorff distance between the identified contours and the preset contours. The identified contours and the preset contours are treated as two point sets, X and Y. The maximum distance between each point x in the X point set and the nearest point in the Y point set is found. The smaller this distance is, the more similar and overlapping the two point sets are.

[0148] S241. Determine the category to which the object belongs based on the similarity calculation results.

[0149] Specifically, if one of the calculated similarity results is greater than the similarity threshold, then the category corresponding to that similarity result is the category to which the object belongs.

[0150] Considering the possibility that all similarity results may not exceed the similarity threshold, in order to reduce the error rate of object recognition inside the cylinder, we can first calculate the similarity between the recognized outline and the preset clothing outline. If all similarity results are not greater than the similarity threshold, then we determine whether there is a preset foreign object outline.

[0151] If there is no preset outline of the foreign object, the object is determined to be a foreign object;

[0152] If a foreign object outline is preset, the similarity between the identified outline and the preset foreign object outline is calculated. If one of the calculated similarity results is greater than the similarity threshold, the category corresponding to that similarity result is the category to which the object belongs. If all similarity results are not greater than the similarity threshold, the object is determined to be a foreign object or an empty cylinder. At this time, the final result is determined based on the difference between the similarity result and the similarity threshold. If the difference is small, the object can be determined to be a foreign object. If the difference is large, the object can be determined to be an empty cylinder, and no warning information needs to be output accordingly.

[0153] S3. Determine the output reminder information based on the state inside the cylinder. The reminder information is used to alert the user that an object exists in the clothing handling device, including but not limited to clothing, foreign objects, etc.

[0154] In some embodiments, the reminder information includes a reminder about clothing left in the canister and / or a reminder about foreign objects in the canister. That is, when clothing is detected in the canister, a reminder about clothing left in the canister is output; when foreign objects are detected in the canister, a reminder about foreign objects in the canister is output; when both clothing and foreign objects are detected in the canister, a reminder about both clothing and foreign objects in the canister is output.

[0155] Optionally, after outputting a reminder message, the clothing processing device pushes the message to the user to remind them to remove the clothing and / or foreign objects from the device promptly. The reminder message can be implemented in various ways. For example, the clothing processing device can control a flashing indicator light, play the reminder message, or send the message to a user terminal for display and / or playback. The user terminal refers to a terminal device that is communicatively connected to the clothing processing device, such as a mobile phone, tablet, or computer. This embodiment does not specifically limit the method of implementing the reminder message.

[0156] Optionally, the notification message can include the type of object, allowing the user to determine whether the object needs to be removed.

[0157] The notification message may also include images of the interior space of the garment processing device, allowing users to see the contents of the device more intuitively and make their next decision.

[0158] After the garment handling device outputs an alert, it can either stop monitoring the status inside the drum directly or continue monitoring to output the corresponding alert again. In actual use, a suitable processing flow can be set according to actual needs.

[0159] Accordingly, after receiving the reminder information, if the user determines that the object does not need to be removed based on the type of object and the image of the space inside the tube in the reminder information, the user can use a feedback command to instruct the clothing handling device not to continue reminding. The clothing handling device will then end the subsequent tube status monitoring program based on the received feedback command.

[0160] Optionally, after the user removes the object from the garment handling device, an object removal command can be sent to the garment handling device through the control panel of the garment handling device or the application on the user terminal, so that the garment handling device can end the subsequent in-drum status monitoring program.

[0161] After the user removes the object from the garment handling device, the user can choose not to perform any feedback operation. The garment handling device can then execute a subsequent internal status monitoring program to continue monitoring the internal status and output corresponding reminder information.

[0162] After allowing the user time to process the objects inside the cylinder, the state inside the cylinder needs to be checked again to prevent the user from forgetting to process them. In some optional embodiments, such as... Figure 9 As shown, it also includes the following steps:

[0163] S4. After a preset time interval, acquire the image of the space inside the clothing processing device again to determine whether there are still clothes and / or foreign objects inside the device; specifically, the outline of the object inside the device can also be identified by the depth image of the space inside the device, and the identified outline can be compared with the preset outline to determine the category of the object, etc. The specific method is the same as the depth image processing method mentioned above, and will not be repeated here.

[0164] The preset time can be a built-in preset time of the clothing processing device or a time set by the user. This embodiment does not specify the specific setting method or duration of the preset time.

[0165] S5. If it is determined that there are still clothes and / or foreign objects inside the tube, output a corresponding reminder message. The specific reminder method and content are the same as those of the inner tube, and will not be repeated here.

[0166] Optionally, a threshold for outputting reminder messages can be set. When the number of times reminder messages are output reaches the threshold, the monitoring of the internal state of the cylinder and the output of reminder messages will stop.

[0167] The processing procedure after the user receives the reminder information, and the subsequent operations performed by the clothing processing device based on the user's processing results, are the same as the processing principle in the above embodiments, and will not be repeated here.

[0168] Since foreign objects are usually small, they can easily be obscured when there are clothes inside the garment handling device's drum. Therefore, after the clothes are removed, the drum can be checked again for any foreign objects to prevent damage to the clothes and the garment handling device. In some alternative embodiments, such as... Figure 10 As shown, it also includes the following steps:

[0169] S6. After confirming that the clothes in the drum have been removed, acquire the image of the space inside the drum of the clothes processing device again to determine whether there are foreign objects inside the drum; specifically, the outline of the object inside the drum can also be identified by the depth image of the space inside the drum, and the identified outline is compared with the preset outline to determine the category of the object, etc. The specific method is the same as the depth image processing method mentioned above, and will not be repeated here.

[0170] Step S6 assumes that clothing has already been identified inside the drum and a corresponding alert has been issued. After confirming that the clothing has been removed, the system can then monitor for any foreign objects inside the drum. The determination of clothing removal can be obtained through user feedback or by identifying the depth of the space inside the drum using a captured image. The specific principle is the same as the method for determining object removal in the drum described in the previous embodiment, and will not be repeated here.

[0171] S7. When a foreign object is detected inside the cylinder, output a foreign object detection alert message. The specific alert method and content are the same as those for the inner cylinder, and will not be repeated here.

[0172] Optionally, a threshold for outputting reminder messages can be set. When the number of times reminder messages are output reaches the threshold, the monitoring of the internal state of the cylinder and the output of reminder messages will stop.

[0173] This embodiment provides a method for providing reminders after clothing processing. It acquires an image of the space inside the drum of a clothing processing device, determines the state inside the drum based on the image, and determines whether to output a reminder message based on the state. The actual implementation of this invention does not rely on the door state of the clothing processing device, expanding its application possibilities and facilitating wider adoption. This invention can identify the state inside the drum through an image of the space, not only identifying leftover clothing but also foreign objects, and promptly reminding the user. This avoids problems such as wrinkles, odors, and mold caused by clothing left inside the clothing processing device, and also prevents clothing contamination and damage to the washing machine and clothing, significantly improving the user experience.

[0174] Example 2

[0175] A clothing processing apparatus is provided, employing the method described above. For a detailed description of the method, please refer to the corresponding descriptions in the above method embodiments, which will not be repeated here. In some embodiments, the clothing processing apparatus may be used as a washing machine, in other embodiments as a washer-dryer combo, or as a clothes dryer.

[0176] The clothing processing device of this embodiment may include the following process:

[0177] The internal space image acquisition module is used to acquire images of the internal space of the clothing processing device.

[0178] The cylinder interior state determination module is used to determine the cylinder interior state through the cylinder interior space image;

[0179] The reminder module is used to determine the reminder information to be output based on the state inside the cylinder.

[0180] In this embodiment, an image acquisition device is installed inside the garment processing device, facing inwards from the inner drum used for loading, washing, drying, and caring for garments. Optionally, the image acquisition device includes a camera, a light sensor, etc. This image acquisition device is used to acquire images of the space inside the garment processing device's drum. Specifically, after the garment processing device finishes processing the garments, the image acquisition device is activated to acquire images of the space inside the drum. One or more image acquisition devices can be deployed depending on their properties, installation location, and actual needs; the specific number can be adjusted according to the actual application.

[0181] In practical use, after the user has taken all or part of the clothes, and after the clothes processing device has completed the processing, the door of the clothes processing device may not be closed. By collecting images of the space inside the tube, the state inside the tube can be detected, which expands the possibilities of application.

[0182] This garment processing device features image recognition capabilities. It uses image recognition technology to detect whether any objects are left inside the garment processing drum. If an object is found, it outputs a reminder message to the user. Optionally, this image recognition process can be integrated into an image acquisition unit or performed by the garment processing device's controller.

[0183] When acquiring images of the space inside the garment processing device, this image acquisition device can capture either an overall image of the inner drum or a partial image of the inner drum. The specific acquisition range is determined based on the design and installation location of the garment processing device and the image acquisition device, the computational load of image processing, and actual requirements.

[0184] In some preferred embodiments, the image acquisition device is fixedly installed inside the door glass of the clothing handling device.

[0185] In some alternative embodiments, the image acquisition device is fixedly installed inside the garment processing device drum or at a door seal, etc.

[0186] This invention does not limit the specific installation location of the image acquisition device. When using the image acquisition device to acquire images of the space inside the tube, this invention does not rely on the status of the door of the clothing handling device; that is, regardless of whether the door is closed or not, the state inside the tube can be detected through the acquired images of the space inside the tube, thus expanding the possibilities for application.

[0187] Based on the technical solutions of the above embodiments, optionally, the internal space of the cylinder is configured to include at least the internal space of the cylinder inner wall.

[0188] Optionally, based on the technical solution of the above embodiments, the image of the space inside the cylinder is a depth image.

[0189] Based on the technical solution of the above embodiments, optionally, the depth image is an image in which the depth values ​​of each point in the cylindrical space are collected by a depth image acquisition device as pixel values.

[0190] Based on the technical solutions of the above embodiments, optionally, the step of determining the internal state of the cylinder through the internal space image includes:

[0191] The depth image is used to identify whether there are clothing and / or foreign objects inside the cylinder;

[0192] The state inside the cylinder is determined based on the identification results.

[0193] Based on the technical solutions of the above embodiments, optionally, the state inside the tube is an empty tube or a tube with clothing left inside and / or a tube with foreign objects inside.

[0194] Based on the technical solution of the above embodiments, optionally, the step of identifying whether there are clothes and / or foreign objects inside the cylinder through the depth image includes:

[0195] The contours of objects inside the cylinder are identified using the depth image;

[0196] The identified contours are compared with preset contours to determine the category to which the object belongs.

[0197] Based on the technical solutions of the above embodiments, optionally, the step of recognizing the outline of the object inside the cylinder through the depth image includes:

[0198] Obtain the background of the space inside the cylinder;

[0199] The object outline region is obtained by subtracting the background of the space inside the cylinder from the current depth image;

[0200] The object's outline is obtained by segmenting the object's contour region using depth values.

[0201] Based on the technical solutions of the above embodiments, optionally, the step of obtaining the background of the space inside the cylinder includes:

[0202] Control the inner cylinder to stop at a preset position so that the background of the space inside the cylinder is the pre-stored background of the space inside the cylinder;

[0203] or,

[0204] Obtain the current depth image of the space inside the cylinder;

[0205] After controlling the inner cylinder to rotate, it returns to the position where the previous image of the depth of the inner cylinder was acquired, so as to re-acquire the image of the depth of the inner cylinder.

[0206] The background of the internal space was determined by taking two images of the depth inside the cylinder.

[0207] Based on the technical solution of the above embodiments, optionally, the step of determining the background of the inner space of the cylinder by acquiring the inner space depth images of the cylinder twice includes:

[0208] The depth values ​​of each point in the two consecutive images of the depth of the space inside the cylinder are subtracted.

[0209] Compare the difference to the threshold;

[0210] The background region inside the cylinder was determined by comparing the results.

[0211] Based on the technical solution of the above embodiments, optionally, the step of segmenting the object contour region by depth value includes:

[0212] Scan the outline region of the object line by line;

[0213] Detect the four neighbors of each pixel in each row;

[0214] Determine if there are any pixels in the four neighboring regions whose depth value difference is greater than a preset value;

[0215] Based on the judgment result, the edge points of the object contour are determined to segment the object contour region and obtain the object contour.

[0216] Based on the technical solution of the above embodiments, optionally, the step of comparing the identified contour with the preset contour includes:

[0217] Calculate the similarity between the identified contour and the preset contour;

[0218] The category to which an object belongs is determined by the similarity calculation results.

[0219] Based on the technical solutions of the above embodiments, optionally, the reminder information includes reminder information about clothing left in the drum and / or reminder information about foreign objects in the drum.

[0220] Optionally, based on the technical solution of the above embodiments, the method further includes the following steps:

[0221] After a preset time interval, another image of the space inside the clothing processing device drum is acquired to determine whether clothing and / or foreign objects are still present inside the drum.

[0222] If it is determined that there are still clothes and / or foreign objects inside the container, a corresponding reminder message will be output.

[0223] Optionally, based on the technical solution of the above embodiments, the method further includes the following steps:

[0224] After confirming that the clothes in the drum have been removed, another image of the space inside the clothes processing device drum is acquired to determine if there are any foreign objects inside the drum.

[0225] When a foreign object is detected inside the cylinder, an alert message indicating that a foreign object is present inside the cylinder is output.

[0226] This embodiment provides a clothing handling device that acquires an image of the space inside the drum, determines the state inside the drum based on the image, and determines whether to output a reminder message based on the state. The actual implementation of this invention does not rely on the door state of the clothing handling device, expanding its application possibilities and facilitating wider adoption. This invention can identify the state inside the drum through an image of the space inside, not only identifying leftover clothing but also foreign objects, and promptly reminding the user. This avoids problems such as wrinkles, odors, and mold caused by clothing left inside the clothing handling device, and also prevents clothing contamination and damage to the washing machine and clothes, significantly improving the user experience.

[0227] Example 3

[0228] An electronic device, such as Figure 13 As shown, it includes: a memory storing program code; and a processor connected to the memory, which, when executed by the processor, implements a method for reminding users after clothing processing. For a detailed description of the method, please refer to the corresponding description in the above method embodiments, which will not be repeated here.

[0229] Example 4

[0230] A computer-readable storage medium, such as Figure 14 As shown, it stores program instructions, which, when executed, implement a reminder method for clothing processing. For a detailed description of the method, please refer to the corresponding description in the above method embodiments; it will not be repeated here.

[0231] The number of devices and processing scale described herein are for the purpose of simplifying the description of the invention. Applications, modifications, and variations of the invention will be readily apparent to those skilled in the art.

[0232] Although embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. For those skilled in the art, other modifications can be easily made. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and illustrations shown and described herein.

[0233] The apparatus, electronic device, and non-volatile computer storage medium and method provided in the embodiments of this specification are corresponding. Therefore, the apparatus, electronic device, and non-volatile computer storage medium also have similar beneficial technical effects as the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the corresponding apparatus, electronic device, and non-volatile computer storage medium will not be repeated here.

[0234] Those skilled in the art will also know that, besides implementing the controller in the form of purely computer-readable program code, the same functions can be achieved by logically programming the method steps, making the controller take the form of logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers (PLCs), and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the devices included within it for implementing various functions can also be considered structures within that hardware component. Alternatively, the devices for implementing various functions can be considered as both software units implementing the method and structures within a hardware component.

[0235] The systems, apparatuses, or units described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. For ease of description, the above apparatuses are described separately as various units based on their functions. Of course, when implementing one or more embodiments of this specification, the functions of each unit can be implemented in one or more software and / or hardware.

[0236] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the embodiments of this specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0237] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0238] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0239] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0240] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0241] This specification may be described in the general context of computer-executable instructions, such as program units, that are executed by a computer. Generally, program units include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification may also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program units may reside in local and remote computer storage media, including storage devices.

[0242] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0243] The above description is merely an embodiment of this specification and is not intended to limit the scope of one or more embodiments of this specification. Various modifications and variations can be made to one or more embodiments of this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of one or more embodiments of this specification should be included within the scope of the claims of one or more embodiments of this specification.

Claims

1. A method for reminding customers after clothing treatment, characterized in that, The configuration includes the following steps after the garment processing device finishes processing the garments: Acquire an image of the space inside the garment processing device's drum; The internal state of the cylinder is determined by the image of the internal space. The output reminder message is determined based on the state inside the cylinder.

2. The reminder method after clothing treatment as described in claim 1, characterized in that: The internal space of the cylinder is configured to include at least the internal wall of the cylinder.

3. The reminder method after clothing treatment as described in claim 1, characterized in that: The image of the space inside the cylinder is a depth image, which is an image in which the depth values ​​of each point in the space inside the cylinder are collected by a depth image acquisition device as pixel values.

4. The reminder method after clothing treatment as described in claim 3, characterized in that, The step of determining the internal state of the cylinder through the internal space image includes: The depth image is used to identify whether there are clothing and / or foreign objects inside the cylinder; The state inside the tube is determined based on the identification results; wherein, the state inside the tube is an empty tube or a tube with clothing left inside and / or a tube with foreign objects inside.

5. The reminder method after clothing treatment as described in claim 4, characterized in that, The step of identifying whether there are clothes and / or foreign objects inside the cylinder through the depth image includes: The contours of objects inside the cylinder are identified using the depth image; The identified contours are compared with preset contours to determine the category to which the object belongs.

6. The reminder method after clothing treatment as described in claim 5, characterized in that, The step of identifying the outline of the object inside the cylinder through the depth image includes: Obtain the background of the space inside the cylinder; The object outline region is obtained by subtracting the background of the space inside the cylinder from the current depth image; The object's outline is obtained by segmenting the object's contour region using depth values.

7. The reminder method after clothing treatment as described in claim 6, characterized in that, The step of obtaining the background of the space inside the cylinder includes: Control the inner cylinder to stop at a preset position so that the current inner cylinder space background is the pre-stored inner cylinder space background; or, Obtain the current depth image of the space inside the cylinder; After controlling the inner cylinder to rotate, it returns to the position where the previous image of the depth of the inner cylinder was acquired, so as to re-acquire the image of the depth of the inner cylinder. The background of the internal space was determined by taking two images of the depth inside the cylinder.

8. The reminder method after clothing treatment as described in claim 7, characterized in that, The step of determining the background of the cylinder space by acquiring two consecutive images of the cylinder space depth includes: The depth values ​​of each point in the two consecutive images of the depth of the space inside the cylinder are subtracted. Compare the difference to the threshold; The background region inside the cylinder was determined by comparing the results.

9. The reminder method after clothing treatment as described in claim 6, characterized in that, The step of segmenting the object contour region using depth values ​​includes: Scan the outline region of the object line by line; Detect the four neighbors of each pixel in each row; Determine if there are any pixels in the four neighboring regions whose depth value difference is greater than a preset value; Based on the judgment result, the edge points of the object contour are determined to segment the object contour region and obtain the object contour.

10. The reminder method after clothing treatment as described in claim 5, characterized in that, The step of comparing the identified contour with the preset contour includes: Calculate the similarity between the identified contour and the preset contour; The category to which an object belongs is determined by the similarity calculation results.

11. The reminder method after clothing treatment as described in claim 4, characterized in that: The reminder information includes reminders of clothing left in the container and / or reminders of foreign objects in the container.

12. The reminder method after clothing treatment as described in claim 4, characterized in that, It also includes the following steps: After a preset time interval, another image of the space inside the clothing processing device drum is acquired to determine whether clothing and / or foreign objects are still present inside the drum. If it is determined that there are still clothes and / or foreign objects inside the container, a corresponding reminder message will be output.

13. The reminder method after clothing treatment as described in claim 12, characterized in that, It also includes the following steps: After confirming that the clothes in the drum have been removed, another image of the space inside the clothes processing device drum is acquired to determine if there are any foreign objects inside the drum. When a foreign object is detected inside the cylinder, an alert message indicating that a foreign object is present inside the cylinder is output.

14. A garment processing device, characterized in that, The method described in any one of claims 1 to 13 is applied.

15. An electronic device, characterized in that, include: A memory that stores program code; A processor connected to the memory, which, when the program code is executed by the processor, implements the method as described in any one of claims 1 to 13.

16. A computer-readable storage medium, characterized in that, It stores program instructions that, when executed, implement the method as described in any one of claims 1 to 13.

Images