Control methods, devices, and equipment for clothing processing equipment based on image recognition

Abstract

This invention relates to a control method, apparatus, medium, and garment processing device based on image recognition. The garment processing device includes a live animal detection program, which comprises: continuously acquiring multiple images of the garment processing drum at different times; processing the multiple images to obtain multiple grayscale value matrices corresponding to the multiple images; and determining whether there is a live animal in the garment processing drum based on the multiple grayscale value matrices. This invention can accurately identify whether there is a live animal in the garment processing drum, thus preventing safety accidents.

Description

Technical Field

[0001] This invention relates to the field of clothing processing equipment, and more specifically, to a control method, apparatus, and device for clothing processing equipment based on image recognition. Background Technology

[0002] If the door of the garment processing equipment is not closed properly, children or small animals may crawl into the garment processing drum. If the user starts the garment processing equipment without knowing this, it will cause a serious accident.

[0003] In related technologies, garment processing equipment uses several methods to determine the presence of living organisms inside the garment processing drum. The first method compares a preset image of a living organism with a photograph to determine its presence. However, due to the polymorphism and multiple colors of living organisms, this method may be inaccurate if the preset sample is not comprehensive enough. The second method uses three infrared range sensors to detect the approach and departure of living organisms to determine if they have entered the garment processing drum. However, these range sensors detect straight-line distances and obstructions, resulting in many blind spots where living organisms cannot be detected. The third method detects living organisms by detecting displacement within the garment processing drum. This method fails when there is external interference or when the living organism is stationary or dormant. The fourth method uses a thermopile infrared sensor to detect temperature data within the garment processing drum and filters the temperature data based on the temperature characteristics of the living organism to detect it. However, for garment processing equipment that has just finished drying or has been washed at a high temperature, the temperature inside the garment processing drum is still high, which can easily affect the accuracy of this method. Summary of the Invention

[0004] This invention provides a control method, apparatus, and device for a garment processing equipment based on image recognition, so as to at least solve the technical problem that it is currently impossible to accurately determine whether there are living creatures inside the garment processing drum.

[0005] The first aspect of this invention proposes a control method for a garment processing device based on image recognition, the garment processing device including a live animal detection program, the live animal detection program including:

[0006] Continuously acquire multiple images of the garment processing drum at different times;

[0007] The multiple images are processed to obtain multiple grayscale value matrices corresponding to the multiple images;

[0008] The presence or absence of living organisms inside the clothing processing drum is determined based on the multiple grayscale value matrices.

[0009] In some embodiments, determining whether there are living organisms inside the clothing processing drum based on the plurality of grayscale value matrices includes:

[0010] Determine the changes in the grayscale value matrix between every two adjacent images in the plurality of images;

[0011] The presence or absence of living organisms in the clothing processing tube is determined based on the changes in the grayscale value matrix of each pair of adjacent images.

[0012] In some implementations, determining the change in the grayscale value matrix between every two adjacent images in the plurality of images includes:

[0013] Determine multiple gray values ​​arranged in the gray value matrix array;

[0014] Determine the difference between the gray value at each position in the Nth gray value matrix and the gray value at the corresponding position in the (N-1)th gray value matrix;

[0015] Determine the sum of gray value differences at all locations, and determine the change in the gray value matrix between each pair of adjacent images based on the sum of gray value differences at all locations.

[0016] Where: N is an integer greater than or equal to 2.

[0017] In some embodiments, determining whether there are living organisms inside the clothing treatment drum based on the sum of the differences between a plurality of calculated grayscale value matrices includes:

[0018] Determine the sum of the differences between multiple grayscale value matrices and the value of the set grayscale value;

[0019] If the sum of the differences between the multiple grayscale value matrices is less than the set grayscale value, it is determined that there are no living creatures in the clothing processing drum.

[0020] If at least one of the sums of the differences between the plurality of grayscale value matrices is greater than or equal to the set grayscale value, it is determined that there is a living organism in the clothing processing drum.

[0021] In some embodiments, before performing the step of continuously acquiring multiple image information of the clothing processing drum, the liveness detection method further includes:

[0022] First, control the garment processing drum to rotate at a first angle in a first direction;

[0023] Then control the garment processing drum to rotate in a second direction at a second angle, the first direction being opposite to the second direction.

[0024] In some embodiments, the live animal detection procedure is performed before the garment processing equipment is started.

[0025] Alternatively, during the operation of the clothing processing equipment, if the opening and closing of the door is detected and it is determined that the clothing processing process has not ended, the live animal detection procedure may be executed.

[0026] A second aspect of the present invention provides a control device comprising one or more processors and a non-transitory computer-readable storage medium storing program instructions, wherein when the one or more processors execute the program instructions, the one or more processors are used to implement any of the image recognition-based clothing processing device control methods proposed in the first aspect of the present invention.

[0027] In some embodiments, the control device includes:

[0028] The image acquisition module is used to continuously acquire multiple images of the clothing processing drum at different times.

[0029] An image processing module is used to process the multiple images to obtain multiple grayscale value matrices corresponding to the multiple images;

[0030] The analysis module is used to determine whether there are any living organisms inside the clothing processing drum based on the multiple grayscale value matrices.

[0031] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a control method for any of the image recognition-based clothing processing devices proposed in the first aspect of the present invention.

[0032] The fourth aspect of the present invention provides a garment processing device that operates according to any of the image recognition-based garment processing device control methods proposed in the first aspect of the present invention, or includes the control device proposed in the second aspect of the present invention, or includes the computer-readable storage medium proposed in the third aspect of the present invention.

[0033] The technical solution of the present invention can include the following beneficial effects: The present invention continuously acquires multiple images of the clothing processing tube at different times. After a living creature enters the clothing processing tube, the images of the clothing processing tube at different times are different when the living creature moves. The multiple gray value matrices corresponding to the multiple images at different times are also different. Therefore, the presence or absence of a living creature in the clothing processing tube can be accurately identified based on the changes in the multiple gray value matrices, thus avoiding the occurrence of safety accidents. Attached Figure Description

[0034] Figure 1 This is a flowchart illustrating a live animal detection process for a garment processing device according to an exemplary embodiment.

[0035] Figure 2 It is a grayscale matrix of a first image shown according to an exemplary embodiment.

[0036] Figure 3 It is a grayscale matrix of a second image shown according to an exemplary embodiment.

[0037] Figure 4 It is a grayscale matrix of a third image shown according to an exemplary embodiment.

[0038] Figure 5 This is a formula for calculating the sum of differences in grayscale value matrices between adjacent images, as illustrated in an exemplary embodiment.

[0039] Figure 6 This is a control flowchart of a garment processing device according to an exemplary embodiment. Detailed Implementation

[0040] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0041] According to an exemplary embodiment, this embodiment proposes a control method for a garment processing device based on image recognition. The garment processing device is, for example, a washing machine or a washer-dryer, and includes a live animal detection program. Figure 1 This is a flowchart illustrating a live animal detection process for a garment processing device according to an exemplary embodiment. The live animal detection procedure includes the following steps:

[0042] S11. Continuously acquire multiple images of the clothing processing tube at different times.

[0043] In this embodiment, the garment processing device includes at least one photographic device for capturing images of the interior of the garment processing drum. One photographic device may be used, and its imaging range covers the interior of the garment processing drum. Multiple photographic devices may be used, and the sum of their imaging ranges covers the garment processing drum. The location of the photographic devices can be set according to actual conditions and is not limited here. For example, at least one photographic device may be located at the door of the garment processing drum, and / or on the rear wall of the garment processing drum. In one example, when acquiring images of the interior of the garment processing drum, the photographic device continuously captures multiple images at predetermined intervals within a predetermined time period. The predetermined time period is, for example, greater than 1 minute and less than or equal to 2 minutes, and the predetermined interval is, for example, 3 to 5 seconds. After capturing multiple images of the interior of the garment processing drum at different times, the photographic device provides subsequent steps with the grayscale matrix of the multiple images to identify whether there are living organisms inside the garment processing drum.

[0044] S12. Process multiple images to obtain multiple grayscale value matrices corresponding to the multiple images.

[0045] In this embodiment, after acquiring multiple images of the clothing processing drum at different times, the images are processed. For example, the processing method involves calling the `cv2.imread()` function from the OpenCV software package.

[0046] `cv2.IMREAD_GRAYSCALE` reads multiple images in grayscale format, thus obtaining a grayscale value matrix corresponding to the multiple images. Of course, other grayscale value recognition software can also be used. In this embodiment, after determining the grayscale value matrix of multiple images, subsequent steps use the grayscale value matrix to determine whether there are living organisms inside the clothing processing drum.

[0047] S13. Determine whether there are any living creatures inside the clothing processing drum based on multiple grayscale value matrices.

[0048] In this embodiment, since the images captured at different times inside the clothing processing drum after a living creature enters the drum are different, the grayscale value matrices corresponding to these images at different times are also different. Therefore, the presence or absence of a living creature inside the clothing processing drum can be accurately identified based on the multiple grayscale value matrices corresponding to the images captured at different times, effectively preventing safety accidents.

[0049] In some embodiments, determining whether there are living creatures in the clothing processing drum based on multiple grayscale value matrices includes: determining the changes in the grayscale value matrix of every two adjacent images in multiple images; and determining whether there are living creatures in the clothing processing drum based on the changes in the grayscale value matrix of every two adjacent images.

[0050] In this embodiment, since the grayscale value matrix of the images changes at different times after a living creature enters the clothing processing drum, the presence or absence of a living creature in the clothing processing drum can be determined by determining the changes in the grayscale value matrix of each pair of adjacent images. In one example, determining the changes in the grayscale value matrix of each pair of adjacent images includes: determining multiple grayscale values ​​arranged in the grayscale value matrix array; determining the difference between the grayscale value at each position in the Nth grayscale value matrix and the corresponding grayscale value at the (N-1)th grayscale value matrix, where N is an integer greater than or equal to 2; determining the sum of the grayscale value differences at all positions; and determining the changes in the grayscale value matrix of each pair of adjacent images based on the sum of the grayscale value differences at all positions.

[0051] After determining the sum of the differences between the grayscale value matrices of every two adjacent images, it is possible to determine whether there is a living being in the laundry drum based on the sum of the differences between the grayscale value matrices of every two adjacent images. Specifically: determine the magnitudes of the sums of the differences between multiple grayscale value matrices and a set grayscale value respectively. When the sums of the differences between multiple grayscale value matrices are all less than the set grayscale value, it is determined that there is no living being in the laundry drum. When at least one of the sums of the differences between multiple grayscale value matrices is greater than or equal to the set grayscale value, it is determined that there is a living being in the laundry drum.

[0052] In a specific example, as Figures 2-5 shown, multiple grayscale value matrices, for example, include successively adjacent first, second, and third grayscale value matrices. In the first grayscale value matrix, A11 to Ann respectively represent a grayscale value of a picture, and the arrangement positions of A11 to Ann are also the same as the positions in the picture. The number of grayscale values and the array arrangement method in the second and third grayscale value matrices are the same as those in the first grayscale value matrix, and the multiple grayscale values in these three grayscale value matrices correspond one by one. Then, when calculating the sum of the differences between the grayscale value matrices of the second picture and the first picture, subtract the grayscale value at each position of the grayscale value matrix of the second picture from the corresponding matrix grayscale value of the first picture and then sum, that is, SAB = ∑(Bij - Aij), (i, j = 1, 2…n). Similarly, to find the sum of the differences between the grayscale value matrices of the third picture and the second picture, subtract the grayscale value at each position of the grayscale value matrix of the third picture from the corresponding matrix grayscale value of the second picture and then sum, that is, SBC = ∑(Cij - Bij), (i, j = 1, 2…n). The subsequent differences in the grayscale values of the pictures are also calculated in this way. After obtaining the sum of the differences in the grayscale values of adjacent pictures, compare it with the set value, that is, determine whether SAB < SS and SBC < SS are satisfied. If both SAB and SBC are less than the set value SS, it is determined that there is no living being in the laundry drum. If SAB ≥ SS or SBC ≥ SS, there is a living being in the laundry drum.

[0053] In some embodiments, after determining that there is a living being in the laundry drum, control the laundry treatment device to stop running and send an alarm message to the user so that the user can check the situation in the laundry drum in time and eliminate potential safety hazards in time. If it is determined that there is no living being in the laundry drum, the corresponding process runs normally.

[0054] In some embodiments, before performing the step of continuously acquiring multiple image information from within the clothing processing drum, the liveness detection method further includes: first controlling the clothing processing drum to rotate in a first direction by a first angle, then controlling the clothing processing drum to rotate in a second direction by a second angle, and remaining stationary for a second predetermined duration, wherein the first direction and the second direction are opposite. This embodiment, by controlling the clothing processing drum to rotate in different directions, prevents live animals from remaining stationary or asleep within the clothing processing drum, ensuring the safety of the live animals while facilitating the triggering of their activity, thus aiding in image detection in subsequent steps. The ranges of the first angle and the second angle can be the same or different; for example, the range of the first angle and the second angle is less than 45 degrees, and the second predetermined duration is, for example, 20s to 30s.

[0055] In some embodiments, a live animal detection procedure is performed before the garment processing equipment is started, or during the operation of the garment processing equipment, when the opening and closing of the door is detected, and it is determined that the garment processing process has not ended, a live animal detection procedure is performed to ensure that there are no live animals during the rotation of the garment processing drum and to avoid safety accidents.

[0056] Figure 6 This is a control flowchart of a garment processing device according to an exemplary embodiment, with reference to... Figure 6 The control process of the garment processing equipment includes the following steps:

[0057] S601, The user starts the washing machine;

[0058] S602, Enter the live animal detection procedure;

[0059] S603. Control the garment processing drum to rotate in a first direction by a first angle, then control the garment processing drum to rotate in a second direction by a second angle and then stop. The first direction and the second direction are opposite.

[0060] S604. Take multiple images at set intervals within a set time period, and convert the grayscale values ​​of the multiple images into multiple grayscale value matrices.

[0061] S605. Determine whether the sum of the differences between the grayscale value matrices of each adjacent image is less than the set grayscale value. If the determination result is yes, proceed to S606; if the determination result is no, proceed to S607.

[0062] S606. If a living creature is found inside the clothing processing drum, an alarm is issued to the user.

[0063] S607. Ensure there are no living creatures in the clothing processing drum and proceed with the clothing processing flow normally.

[0064] S608. Determine whether the user has opened the door of the clothing processing equipment during the process. If the result is yes, stop the current process and return to S602. If the result is no, proceed to S609.

[0065] S609. Determine whether the clothing processing process has ended. If the result is yes, end the clothing processing process. If the result is no, return to S608.

[0066] According to an exemplary embodiment, this embodiment proposes a control device, which includes one or more processors and a non-transitory computer-readable storage medium storing program instructions. When the one or more processors execute the program instructions, the one or more processors are used to implement any of the image recognition-based clothing processing device control methods proposed in the above embodiments.

[0067] Specifically, the control device includes: an image acquisition module for continuously acquiring multiple images of the clothing processing drum at different times; an image processing module for processing the multiple images to obtain multiple grayscale value matrices corresponding to the multiple images; and an analysis module for determining whether there are any living organisms in the clothing processing drum based on the multiple grayscale value matrices.

[0068] According to an exemplary embodiment, this embodiment proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a control method for any of the image recognition-based clothing processing devices proposed in the above embodiments.

[0069] According to an exemplary embodiment, this embodiment proposes a garment processing device that operates according to any of the image recognition-based garment processing device control methods proposed in the above embodiments, or includes the control device proposed in the above embodiments, or includes the computer-readable storage medium proposed in the third aspect of the present invention.

[0070] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0071] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0072] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0073] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0074] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0075] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0076] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0077] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A control method for a garment processing device based on image recognition, characterized in that, The garment processing equipment includes a live animal detection program, which includes: Continuously acquire multiple images of the garment processing drum at different times; The multiple images are processed to obtain multiple grayscale value matrices corresponding to the multiple images; The presence of any living organisms inside the clothing processing drum is determined based on the multiple grayscale value matrices. The step of determining whether there are living organisms inside the clothing processing drum based on the multiple grayscale value matrices includes: Determine the changes in the grayscale value matrix between every two adjacent images in the plurality of images; The presence or absence of living organisms in the clothing processing drum is determined based on the changes in the grayscale value matrix of each pair of adjacent images. The step of determining whether there are living creatures in the clothing processing drum based on the changes in the grayscale value matrix of each pair of adjacent images includes: Determine the sum of the differences between multiple grayscale value matrices and the value of the set grayscale value; If the sum of the differences between the multiple grayscale value matrices is less than the set grayscale value, it is determined that there are no living creatures in the clothing processing drum. If at least one of the sums of the differences in the plurality of grayscale value matrices is greater than or equal to the set grayscale value, it is determined that there is a living creature in the clothing processing drum.

2. The control method for the clothing processing equipment based on image recognition according to claim 1, characterized in that, Determining the change in the grayscale value matrix between every two adjacent images in the plurality of images includes: Determine multiple gray values ​​for each of the gray value matrix array arrangements; Determine the difference between the gray value at each position in the Nth gray value matrix and the gray value at the corresponding position in the (N-1)th gray value matrix; Determine the sum of gray value differences at all locations, and determine the change in the gray value matrix between each pair of adjacent images based on the sum of gray value differences at all locations. Where N is an integer greater than or equal to 2.

3. The control method for the clothing processing equipment based on image recognition according to claim 1, characterized in that, Before performing the step of continuously acquiring multiple images of the clothing processing drum at different times, the liveness detection procedure further includes: First, control the garment processing drum to rotate at a first angle in a first direction; Then control the garment processing drum to rotate in the second direction at the second angle, and keep it stationary for a set time. The first direction is opposite to the second direction.

4. The control method for the clothing processing equipment based on image recognition according to claim 1, characterized in that, The live animal detection procedure is executed before the clothing processing equipment is started and operated. Alternatively, during the operation of the clothing processing equipment, if the opening and closing of the door is detected and it is determined that the clothing processing process has not ended, the live animal detection procedure may be executed.

5. A control device, characterized in that, It includes one or more processors and a non-transitory computer-readable storage medium storing program instructions, wherein when the one or more processors execute the program instructions, the one or more processors are used to implement the control method of the image recognition-based clothing processing device according to any one of claims 1-4.

6. The control device according to claim 5, characterized in that, The control device includes: The image acquisition module is used to continuously acquire multiple images of the clothing processing drum at different times. An image processing module is used to process the multiple images to obtain multiple grayscale value matrices corresponding to the multiple images; The analysis module is used to determine whether there are any living organisms inside the clothing processing drum based on the multiple grayscale value matrices.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the control method for the image recognition-based clothing processing device as described in any one of claims 1-4.

8. A garment processing device, characterized in that, The garment processing device operates according to the control method of the image recognition-based garment processing device according to any one of claims 1-4, or includes the control device according to claim 5 or 6, or includes the computer-readable storage medium according to claim 7.

Images