Method and device for identifying foreign matters in washing machine, storage medium and electronic equipment

Abstract

The invention provides an identification method and device for foreign matters in a washing machine, a storage medium and electronic equipment, and the method comprises the steps: a positioning step: obtaining first image data of an object in the washing machine, and employing a target detection algorithm to position a first region on the object in the washing machine; a first acquisition step: acquiring second image data of the first area, and determining the probability of existence of foreign matters in the first area as a first prediction probability according to the second image data and a dichotomy algorithm; a second acquisition step: acquiring infrared sensing data of the first area, and determining the probability that foreign matters exist in the first area as a second prediction probability according to the infrared sensing data and the deep neural network; and a determination step: determining the final prediction probability of the foreign matters in the first region according to the first prediction probability and the second prediction probability. According to the method, the paper foreign matters are detected before washing and during washing, so that the paper foreign matters are prevented from being damaged in the washing process and polluting clothes and the environment in the washing barrel.

Description

Technical Field

[0001] The present application relates to the field of foreign object recognition, and in particular, to a method for recognizing foreign objects in a washing machine, a device for recognizing foreign objects in a washing machine, a computer-readable storage medium, and an electronic device. Background Art

[0002] It is inevitable that users will make mistakes when using washing machines, such as putting foreign objects in the pockets of clothes into the washing drum for washing. Foreign objects may be some important paper objects, such as banknotes, invoices, documents, etc. These paper objects are at risk of being damaged during the washing process, and the paper scraps they produce will be stained on the clothes and the inner drum of the washing machine and are difficult to clean, resulting in a bad user experience. Summary of the invention

[0003] The main purpose of the present application is to provide a method for identifying foreign objects in a washing machine, a device for identifying foreign objects in a washing machine, a computer-readable storage medium and an electronic device, so as to at least solve the problem that foreign objects in clothes in an existing washing machine cannot be identified.

[0004] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a method for identifying foreign objects in a washing machine is provided, comprising: a positioning step, acquiring first image data of an object in the washing machine, and using a target detection algorithm to locate a first area on the object in the washing machine according to the first image data; a first acquisition step, acquiring second image data of the first area, and determining the probability of the existence of foreign objects in the first area as a first predicted probability based on the second image data and a binary classification algorithm; a second acquisition step, acquiring infrared sensor data of the first area, and determining the probability of the existence of foreign objects in the first area as a second predicted probability based on the infrared sensor data and a deep neural network; a determination step, determining the final predicted probability of the existence of foreign objects in the first area based on the first predicted probability and the second predicted probability.

[0005] Optionally, first image data of an object in a washing machine is obtained, and a target detection algorithm is used to locate a first area on the object in the washing machine according to the first image data, including: controlling the washing machine to enter a waterless rotation mode, and obtaining the first image data of the object in the washing machine; constructing a YOLO target detection algorithm based on a backbone network, a feature enhancement network and a target detection head, wherein the YOLO target detection algorithm is trained using multiple sets of training data, each set of the multiple sets of training data includes: picture data of historical objects in the washing machine and label data of the historical objects corresponding to the picture data of the historical objects acquired within a historical time period; the first image data is input into the YOLO target detection algorithm to obtain a positioning result of the first area.

[0006] Optionally, second image data of the first area is obtained, and the probability of the presence of foreign matter in the first area is determined as a first predicted probability based on the second image data and a binary classification algorithm, including: intercepting the second image data from the first image data; obtaining a binary classification algorithm, the binary classification algorithm is one of the following: support vector machine, random forest algorithm, decision tree algorithm; using the binary classification algorithm to determine the first predicted probability based on the second image data.

[0007] Optionally, infrared sensing data of the first area is obtained, and the probability of the presence of foreign matter in the first area is determined as a second predicted probability based on the infrared sensing data and a deep neural network, including: using active infrared technology to actively emit a first infrared light to the object in the washing machine, and receiving a second infrared light reflected by the object through a detector, wherein the active infrared technology is a technology in which an infrared light source actively emits infrared light to detect an object; converting the second infrared light into an electrical signal to obtain the infrared sensing data, and amplifying and filtering the infrared sensing data to obtain a processed digital signal; using image processing technology to generate an infrared image of the object based on the processed digital signal; and inputting the infrared image into the deep neural network to obtain the second predicted probability.

[0008] Optionally, determining the final predicted probability of the presence of a foreign object in the first region according to the first predicted probability and the second predicted probability includes: obtaining a weighted average formula A=α SVM ×0.5+α Infrared ×0.5, where A is the final predicted probability, α SVM is the first predicted probability, α Infrared is the second predicted probability; and according to the weighted average formula, the final predicted probability of the existence of foreign matter in the first area is determined.

[0009] Optionally, after determining the final predicted probability of the presence of foreign matter in the first area based on the first predicted probability and the second predicted probability, the method further includes: when the final predicted probability of the presence of foreign matter in the first area is greater than or equal to a preset probability, stopping the washing program of the washing machine and issuing a warning message; when the final predicted probability of the presence of foreign matter in the first area is less than the preset probability, entering the washing program of the washing machine.

[0010] Optionally, when the final predicted probability of the presence of foreign matter in the first area is less than the preset probability, after entering the washing program of the washing machine, the method further includes: repeating the positioning step, the first acquisition step, the second acquisition step and the determination step until the final predicted probability is greater than or equal to the preset probability or the washing of the washing machine is completed.

[0011] According to another aspect of the present application, a device for identifying foreign objects in a washing machine is provided, comprising: a positioning unit, used to locate a first area on the object in the washing machine using a target detection algorithm; a first acquisition unit, used to acquire real-time image data of the first area, and determine the probability of the existence of foreign objects in the first area as a first predicted probability based on the real-time image data and a support vector machine model; a second acquisition unit, used to acquire infrared sensor data of the first area, and determine the probability of the existence of foreign objects in the first area as a second predicted probability based on the infrared sensor data and a deep neural network; and a determination unit, used to determine a final predicted probability of the existence of foreign objects in the first area based on the first predicted probability and the second predicted probability.

[0012] According to another aspect of the present application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute any one of the methods for identifying foreign objects in a washing machine.

[0013] According to another aspect of the present application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing any one of the methods for identifying foreign objects in a washing machine.

[0014] Applying the technical solution of the present application, the above-mentioned method for identifying foreign objects in the washing machine first has a positioning step, obtaining the first image data of the object in the washing machine, and using the target detection algorithm to locate the first area on the object in the washing machine according to the first image data; followed by a first acquisition step, obtaining the second image data of the first area, and determining the probability of the existence of foreign objects in the first area as the first predicted probability according to the second image data and the binary classification algorithm; then a second acquisition step, obtaining the infrared sensor data of the first area, and determining the probability of the existence of foreign objects in the first area as the second predicted probability according to the infrared sensor data and the deep neural network; finally, a determination step, determining the final predicted probability of the existence of foreign objects in the first area according to the first predicted probability and the second predicted probability. This method can detect paper foreign objects in the first time, stop the washing program before they are diffused and damaged, and analyze the collected missed and false alarm information data, optimize the detection algorithm online, continuously optimize the detection performance, improve the user experience, and solve the problem that foreign objects in the clothes in the existing washing machine cannot be distinguished. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings constituting part of the present application are used to provide a further understanding of the present application. The exemplary embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0016] Figure 1 A hardware structure block diagram of a mobile terminal for executing a method for identifying foreign matter in a washing machine provided in an embodiment of the present application is shown;

[0017] Figure 2 A schematic flow chart of a method for identifying foreign matter in a washing machine provided according to an embodiment of the present application is shown;

[0018] Figure 3 A schematic diagram of a network structure of a YOLOv5 target detection algorithm provided according to an embodiment of the present application is shown;

[0019] Figure 4 A schematic flow chart of another method for identifying foreign matter in a washing machine provided according to an embodiment of the present application is shown;

[0020] Figure 5 A structural block diagram of a device for identifying foreign matter in a washing machine provided according to an embodiment of the present application is shown.

[0021] The above drawings include the following reference numerals:

[0022] 102, processor; 104, memory; 106, transmission device; 108, input and output devices. DETAILED DESCRIPTION

[0023] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0024] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] As introduced in the background technology, users are bound to make mistakes when using washing machines, such as throwing foreign objects in clothing pockets into the washing drum for washing without taking them out. Foreign objects may be some important paper objects, such as banknotes, invoices, documents, etc. These paper objects are at risk of being damaged during the washing process, and the paper scraps they produce will be contaminated on clothing and the inner drum of the washing machine and are difficult to clean, resulting in a bad user experience. In order to solve the problem that foreign objects cannot be identified in clothing in existing washing machines, the embodiments of the present application provide a method for identifying foreign objects in a washing machine, a device for identifying foreign objects in a washing machine, a computer-readable storage medium, and an electronic device.

[0027] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0028] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal for a method for identifying foreign matter in a washing machine according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0029] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for identifying foreign matter in a washing machine in an embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof. The transmission device 106 is used to receive or send data via a network. The above network specific examples may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0030] In this embodiment, a method for identifying foreign objects in a washing machine that runs on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0031] Figure 2 FIG. 1 is a flow chart of a method for identifying foreign matter in a washing machine according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0032] Step S201 (positioning step), obtaining first image data of an object in the washing machine, and using a target detection algorithm to position a first area on the object in the washing machine according to the first image data;

[0033] Specifically, visual sensors are used to monitor the laundry environment, and clothing pockets are located and clothing pocket areas are captured based on the YOLO target detection algorithm, effectively reducing the detection area for identifying foreign objects.

[0034] The method of acquiring first image data of an object in a washing machine and locating a first area on the object in the washing machine according to the first image data using a target detection algorithm comprises the following steps:

[0035] Step S2011, controlling the washing machine to enter a waterless rotation mode, and acquiring first image data of the object in the washing machine;

[0036] Step S2012, constructing a YOLO target detection algorithm based on the backbone network, the feature enhancement network and the target detection head, wherein the YOLO target detection algorithm is trained using multiple sets of training data, and each set of training data in the multiple sets of training data includes: image data of the historical objects in the washing machine and label data of the historical objects corresponding to the image data of the historical objects acquired in a historical time period;

[0037] Step S2013: input the first image data into the YOLO target detection algorithm to obtain a positioning result of the first area.

[0038] Specifically, the YOLO algorithm is a fast and efficient real-time target detection algorithm that can automatically and accurately locate the target area, improving the intelligence and automation level of the washing machine. A sensor device is installed on the washing machine to monitor the washing screen and obtain image data of the object. Before washing, the washing machine is rotated for a certain period of time without adding clean water. During the waterless rotation process, the YOLO target detection algorithm with basic training is used to detect pockets on the washing screen image to locate the pocket area.

[0039] In an optional solution, the YOLO algorithm has multiple versions, generally including YOLOv1 to YOLOv7. In this embodiment, the YOLOv5 target detection algorithm is used to locate the pocket area. Figure 3The network structure of the YOLOv5 target detection algorithm is given, which is mainly composed of a backbone network, a feature enhancement network, and a target detection head. Among them, Conv represents the convolution operation, C3 represents the modular combination of convolutions, n represents the number of cycles, k represents the size of the convolution kernel, s represents the convolution step, and SPPF is a feature fusion module. Based on the backbone network, the feature map is obtained, and the image features are further enhanced using the feature enhancement network. The shallow feature map has rich spatial location information, and the deep feature map has rich semantic information. Feature enhancement is achieved by fusing the upper and lower layers and horizontal connections. The feature enhancement network outputs three feature maps of different scales, and three detection heads are used to realize target detection of different pocket sizes. The output result of the target detection is the position of the circumscribed rectangle of the pocket area in the monitoring image.

[0040] The training process of the YOLOv5 target detection algorithm requires image data and label data, which correspond to each other. The information in the label data is the location information of the circumscribed rectangle of the pocket area in the corresponding image data. The model weight is obtained by making a pocket target detection data set for algorithm training. In the application stage, the image is input into the model for inference to obtain the pocket target detection result.

[0041] Step S202 (first acquisition step), acquiring second image data of the first region, and determining the probability of the presence of foreign matter in the first region as a first predicted probability based on the second image data and a binary classification algorithm;

[0042] Specifically, the pocket area is classified into two categories: whether there are paper foreign objects or not. If paper foreign objects are found, an alarm is triggered and the washing program is stopped to prevent paper scraps from contaminating clothes and making them difficult to clean, thereby improving the user experience.

[0043] The method of obtaining the second image data of the first area and determining the probability of the existence of a foreign object in the first area as a first predicted probability according to the second image data and a binary classification algorithm comprises the following steps:

[0044] Step S2021, extracting the second image data from the first image data;

[0045] Step S2022, obtaining a binary classification algorithm, wherein the binary classification algorithm is one of the following: support vector machine, random forest algorithm, decision tree algorithm;

[0046] Step S2023: Determine the first prediction probability according to the second image data using the binary classification algorithm.

[0047] Specifically, after locating the pocket area using the YOLO target detection algorithm, a picture of the area is captured, and the picture is further classified into two categories to determine whether it contains paper foreign matter. Image classification can be implemented using specific algorithms such as support vector machines, random forests, and decision trees.

[0048] In an optional solution, a support vector machine is used for image binary classification. The goal of the support vector machine is to find an optimal hyperplane that maximizes the interval between the two types of data points. The mathematical representation of the hyperplane is w×x+b=0, where w is the normal vector, x is the input feature vector, and b is the bias term. The minimum distance between the data points of two categories and the hyperplane is the interval, and the goal is to find a hyperplane that maximizes this interval. First, the feature vector is extracted from the intercepted pocket area image using the gradient direction histogram, and the feature vector is normalized to eliminate the dimensional difference between different features; the support vector machine model is trained using the extracted feature vector and the corresponding label, that is, the category of whether the image has paper foreign matter; the support vector is the closest data point to the hyperplane, which determines the position and direction of the hyperplane. During the training process, only the support vector has an effect on the position of the hyperplane; finally, the feature vector is extracted from the new pocket area image, and the trained support vector machine model is used to perform binary classification of whether there is paper foreign matter, and the probability of having paper foreign matter is output.

[0049] Step S203 (second acquisition step), acquiring infrared sensor data of the first area, and determining the probability of the presence of foreign matter in the first area as a second predicted probability based on the infrared sensor data and the deep neural network;

[0050] Specifically, an infrared sensing detection branch is introduced at the same time to improve the accuracy of judgment, reduce false alarms, reduce losses, and improve user experience.

[0051] The infrared sensing data of the first area is obtained, and the probability of the existence of a foreign object in the first area is determined as a second predicted probability according to the infrared sensing data and the deep neural network, including the following steps:

[0052] Step S2031, using active infrared technology to actively emit a first infrared light to the object in the washing machine, and receiving a second infrared light reflected by the object through a detector, wherein the active infrared technology is a technology in which an infrared light source actively emits infrared light to detect the object;

[0053] Step S2032, converting the second infrared light into an electrical signal to obtain the infrared sensing data, and amplifying and filtering the infrared sensing data to obtain a processed digital signal;

[0054] Step S2033, generating an infrared image of the object using image processing technology according to the processed digital signal;

[0055] Step S2034: input the infrared image into the deep neural network to obtain the second prediction probability.

[0056] Specifically, considering that the detection performance of paper foreign matter binary classification detection using image methods alone will be affected by uncertain factors such as lighting and the size of paper foreign matter, this application also introduces an infrared sensing detection branch. Infrared sensors are mainly divided into active and passive forms.

[0057] In an optional solution, active infrared technology with higher precision is used, using an infrared light source to actively emit infrared light, and then the detector receives the reflected infrared signal to detect the target. Based on the results of pocket positioning, the infrared light source emits infrared light to the pocket area at a predetermined frequency and intensity. The emitted infrared light irradiates the pocket area, and the target reflects part of the infrared light. The infrared detector receives the reflected infrared light and converts it into an electrical signal, which is amplified and filtered. The processed digital signal generates an infrared image through image processing technology, and the reflection characteristics of the infrared image are analyzed. The difference in the absorption and reflection ability of paper foreign matter and clothing to infrared energy is used to detect whether there is a paper foreign matter. By collecting a large amount of infrared image measured data as a data set, a deep neural network is trained using a deep learning method to predict the probability of having a paper foreign matter.

[0058] Step S204 (determination step), determining the final predicted probability of the existence of foreign matter in the first area according to the first predicted probability and the second predicted probability.

[0059] Specifically, the use of binary classification and infrared scanning weighted prediction of whether there is a paper foreign object in the pocket area is more accurate than judging based on only one condition, reducing false alarms, reducing losses, and improving user experience. At the same time, feedback is provided on false alarm data to continuously optimize the paper foreign object recognition performance.

[0060] Wherein, determining the final predicted probability that a foreign object exists in the first area according to the first predicted probability and the second predicted probability comprises the following steps:

[0061] Step S301, obtain the weighted average formula A=α SVM ×0.5+α Infrared ×0.5, where A is the above final prediction probability, α SVM is the first predicted probability, α Infrared is the second predicted probability mentioned above;

[0062] Step S302: determining the final predicted probability of the existence of foreign matter in the first region according to the weighted average formula.

[0063] Specifically, for the same pocket area, both the visual image solution and the infrared image solution output the probability of having paper foreign matter. The two probabilities are weighted to obtain the final prediction result, which improves the accuracy of the prediction result. When paper foreign matter is detected, an alarm is immediately issued to remind the user and the washing process is stopped. If no paper foreign matter is detected, the washing process is started.

[0064] After determining the final predicted probability that a foreign object exists in the first area according to the first predicted probability and the second predicted probability, the method further comprises the following steps:

[0065] Step S401, when the final predicted probability of the presence of foreign matter in the first area is greater than or equal to a preset probability, stopping the washing program of the washing machine and issuing a warning message;

[0066] Step S402: When the final predicted probability of the existence of foreign matter in the first area is less than the preset probability, the washing program of the washing machine is entered.

[0067] Specifically, the preset probability in the present application is set to 0.6, that is, when A is greater than 0.6, it is considered that paper foreign matter exists.

[0068] Among them, when the final predicted probability of the existence of foreign matter in the above-mentioned first area is less than the above-mentioned preset probability, after entering the washing program of the above-mentioned washing machine, the above-mentioned method also includes: repeatedly executing the above-mentioned positioning step, the above-mentioned first acquisition step, the above-mentioned second acquisition step and the above-mentioned determination step until the above-mentioned final predicted probability is greater than or equal to the above-mentioned preset probability or the washing of the above-mentioned washing machine is completed.

[0069] Specifically, after entering the washing program, the washing process is still monitored in real time, and the target detection pocket positioning, paper foreign body image classification, infrared sensor paper detection, prediction weighting, and foreign body judgment process are repeated until the foreign body is found or the washing program is completed. When the user finds that the washing machine has missed or falsely reported paper foreign bodies, the washing machine will send the image data and infrared sensor data collected during the washing process to the server for analysis. The service background optimizes the detection algorithm and continuously enhances the washing machine's paper foreign body recognition ability by upgrading the detection algorithm online, thereby improving the user experience.

[0070] The above-mentioned method for identifying foreign objects in the washing machine of the present application first has a positioning step, which is to obtain the first image data of the object in the washing machine, and use the target detection algorithm to locate the first area on the object in the washing machine according to the first image data; then there is a first acquisition step, which is to obtain the second image data of the first area, and determine the probability of the existence of foreign objects in the first area as the first predicted probability according to the second image data and the binary classification algorithm; then there is a second acquisition step, which is to obtain the infrared sensor data of the first area, and determine the probability of the existence of foreign objects in the first area as the second predicted probability according to the infrared sensor data and the deep neural network; finally there is a determination step, which is to determine the final predicted probability of the existence of foreign objects in the first area according to the first predicted probability and the second predicted probability. The method can detect paper foreign objects in the first time, stop the washing program before they are diffused and damaged, and analyze the collected missed and false alarm information data, optimize the detection algorithm online, continuously optimize the detection performance, improve the user experience, and solve the problem that foreign objects in the clothes in the existing washing machine cannot be distinguished.

[0071] The complete process of the method for identifying foreign matter in the washing machine of the present application is as follows: Figure 4 As shown, the steps include: rotating for a certain period of time without adding clean water before washing, and performing pocket detection on the washing screen image during the waterless rotation process using the trained YOLOv5 target detection algorithm to locate the pocket area; performing binary classification and infrared sensing paper detection to determine whether there are paper foreign objects in the positioning area, and performing weighted processing on the two probabilities to obtain the final prediction result to determine whether there are paper foreign objects; when paper foreign objects are detected, an alarm is immediately issued to remind the user and stop the washing process, and if no paper foreign objects are detected, the washing program is started; when the user finds that the washing machine has missed or falsely reported paper foreign objects, the service background will optimize the detection algorithm and upgrade the detection algorithm online.

[0072] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0073] The embodiment of the present application also provides a device for identifying foreign objects in a washing machine. It should be noted that the device for identifying foreign objects in a washing machine in the embodiment of the present application can be used to execute the method for identifying foreign objects in a washing machine provided in the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions thereof will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0074] The following is an introduction to the foreign matter identification device in the washing machine provided in the embodiment of the present application.

[0075] Figure 5 Schematic diagram of a device for identifying foreign matter in a washing machine according to an embodiment of the present application. Figure 5 As shown, the device includes: a positioning unit 10, a first acquisition unit 20, a second acquisition unit 30 and a determination unit 40, the positioning unit 10 is used to use a target detection algorithm to locate the first area on the object in the above-mentioned washing machine; the first acquisition unit 20 is used to acquire real-time image data of the above-mentioned first area, and determine the probability of the existence of foreign matter in the above-mentioned first area as a first predicted probability according to the above-mentioned real-time image data and a support vector machine model; the second acquisition unit 30 is used to acquire infrared sensor data of the above-mentioned first area, and determine the probability of the existence of foreign matter in the above-mentioned first area as a second predicted probability according to the above-mentioned infrared sensor data and a deep neural network; the determination unit 40 is used to determine the final predicted probability of the existence of foreign matter in the above-mentioned first area according to the above-mentioned first predicted probability and the above-mentioned second predicted probability.

[0076] The above-mentioned foreign body identification device in the washing machine of the present application includes: a positioning unit, a first acquisition unit, a second acquisition unit and a determination unit. The positioning unit is used to locate the first area on the object in the washing machine by using a target detection algorithm; the first acquisition unit is used to acquire real-time image data of the first area, and determine the probability of the existence of foreign bodies in the first area as a first predicted probability according to the real-time image data and the support vector machine model; the second acquisition unit is used to acquire infrared sensor data of the first area, and determine the probability of the existence of foreign bodies in the first area as a second predicted probability according to the infrared sensor data and the deep neural network; the determination unit is used to determine the final predicted probability of the existence of foreign bodies in the first area according to the first predicted probability and the second predicted probability. The device can detect paper foreign bodies at the first time, stop the washing program before they are diffused and damaged, and analyze the collected missed and false alarm information data, optimize the detection algorithm online, continuously optimize the detection performance, improve the user experience, and solve the problem that foreign bodies in clothes in the existing washing machine cannot be distinguished.

[0077] In some examples, the positioning unit includes a first positioning module, a second positioning module and a third positioning module, wherein the first positioning module is used to control the washing machine to enter the waterless rotation mode and obtain the first image data of the object in the washing machine; the second positioning module is used to construct a YOLO target detection algorithm based on a backbone network, a feature enhancement network and a target detection head, wherein the YOLO target detection algorithm is trained using multiple sets of training data, and each set of training data in the multiple sets of training data includes: image data of historical objects in the washing machine and label data of the historical objects corresponding to the image data of the historical objects acquired during a historical time period; the third positioning module is used to input the first image data into the YOLO target detection algorithm to obtain the positioning result of the first area. The clothing pocket is located based on the YOLO target detection algorithm and the clothing pocket area is intercepted, so as to effectively reduce the detection area for identifying foreign objects.

[0078] In some examples, the first acquisition unit includes a first acquisition module, a second acquisition module and a third acquisition module, the first acquisition module is used to intercept the second image data from the first image data; the second acquisition module is used to acquire a binary classification algorithm, and the binary classification algorithm is one of the following: support vector machine, random forest algorithm, decision tree algorithm; the third acquisition module is used to use the binary classification algorithm to determine the first prediction probability according to the second image data. Use the trained support vector machine model to perform binary classification of whether there is paper foreign matter, and output the probability of having paper foreign matter.

[0079] In some examples, the second acquisition unit includes a fourth acquisition module, a fifth acquisition module, a sixth acquisition module and a seventh acquisition module. The fourth acquisition module is used to actively emit a first infrared light to the object in the washing machine using active infrared technology, and receive the second infrared light reflected by the object through a detector. The active infrared technology is a technology in which an infrared light source actively emits infrared light to detect an object; the fifth acquisition module is used to convert the second infrared light into an electrical signal to obtain the infrared sensing data, and amplify and filter the infrared sensing data to obtain a processed digital signal; the sixth acquisition module is used to generate an infrared image of the object using image processing technology according to the processed digital signal; the seventh acquisition module is used to input the infrared image into the deep neural network to obtain the second prediction probability. Infrared sensing detection branches are introduced to improve the accuracy of judgment, reduce false alarms, reduce losses, and improve user experience.

[0080] In some examples, the determination unit includes a first determination module and a second determination module, the first determination module is used to obtain the weighted average formula A=α SVM ×0.5+α Infrared ×0.5, where A is the above final prediction probability, αSVM is the first predicted probability, α Infrared is the second predicted probability; the second determination module is used to determine the final predicted probability of the existence of foreign matter in the first area according to the weighted average formula. Compared with judging based on only one condition, the accuracy of the prediction result is improved and false alarms are reduced.

[0081] In some examples, the determination unit further includes a third determination module and a fourth determination module, the third determination module is used to stop the washing program of the washing machine and issue a warning message when the final predicted probability of the existence of foreign matter in the first area is greater than or equal to the preset probability; the fourth determination module is used to enter the washing program of the washing machine when the final predicted probability of the existence of foreign matter in the first area is less than the preset probability. When A is greater than 0.6, it is considered that paper foreign matter exists.

[0082] In some examples, the fourth determination module is further used to repeatedly perform the above positioning step, the above first acquisition step, the above second acquisition step and the above determination step until the above final prediction probability is greater than or equal to the above preset probability or the above washing machine completes washing. By upgrading the detection algorithm online, the ability of the washing machine to identify paper foreign objects is continuously enhanced to improve the user experience.

[0083] The foreign body identification device in the washing machine comprises a processor and a memory, the positioning unit and the like are stored in the memory as program units, and the processor executes the program units stored in the memory to realize the corresponding functions. The modules are all located in the same processor; or, the modules are located in different processors in any combination.

[0084] The processor includes a kernel, and the kernel retrieves the corresponding program unit from the memory. One or more kernels can be set, and the problem of foreign matter indistinguishable in clothes in the existing washing machine can be solved by adjusting kernel parameters.

[0085] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0086] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method for identifying foreign objects in the washing machine.

[0087] Specifically, the method for identifying foreign matter in a washing machine includes:

[0088] Step S201 (positioning step), obtaining first image data of an object in the washing machine, and using a target detection algorithm to position a first area on the object in the washing machine according to the first image data;

[0089] Specifically, visual sensors are used to monitor the laundry environment, and clothing pockets are located and clothing pocket areas are captured based on the YOLO target detection algorithm, effectively reducing the detection area for identifying foreign objects.

[0090] Step S202 (first acquisition step), acquiring second image data of the first region, and determining the probability of the presence of foreign matter in the first region as a first predicted probability based on the second image data and a binary classification algorithm;

[0091] Specifically, the pocket area is classified into two categories: whether there are paper foreign objects or not. If paper foreign objects are found, an alarm is triggered and the washing program is stopped to prevent paper scraps from contaminating clothes and making them difficult to clean, thereby improving the user experience.

[0092] Step S203 (second acquisition step), acquiring infrared sensor data of the first area, and determining the probability of the presence of foreign matter in the first area as a second predicted probability based on the infrared sensor data and the deep neural network;

[0093] Specifically, an infrared sensing detection branch is introduced at the same time to improve the accuracy of judgment, reduce false alarms, reduce losses, and improve user experience.

[0094] Step S204 (determination step), determining the final predicted probability of the existence of foreign matter in the first area according to the first predicted probability and the second predicted probability.

[0095] Specifically, the use of binary classification and infrared scanning weighted prediction of whether there is a paper foreign object in the pocket area is more accurate than judging based on only one condition, reducing false alarms, reducing losses, and improving user experience. At the same time, feedback is provided on false alarm data to continuously optimize the paper foreign object recognition performance.

[0096] Optionally, first image data of an object in a washing machine is obtained, and a target detection algorithm is used to locate a first area on the object in the washing machine according to the first image data, including: controlling the washing machine to enter a waterless rotation mode, and obtaining the first image data of the object in the washing machine; constructing a YOLO target detection algorithm based on a backbone network, a feature enhancement network and a target detection head, wherein the YOLO target detection algorithm is trained using multiple sets of training data, and each set of training data in the multiple sets of training data includes: picture data of historical objects in the washing machine and label data of the historical objects corresponding to the picture data of the historical objects acquired within a historical time period; the first image data is input into the YOLO target detection algorithm to obtain a positioning result of the first area.

[0097] Optionally, second image data of the first area is obtained, and the probability of the presence of foreign matter in the first area is determined as a first predicted probability based on the second image data and a binary classification algorithm, including: intercepting the second image data from the first image data; obtaining a binary classification algorithm, the binary classification algorithm is one of the following: support vector machine, random forest algorithm, decision tree algorithm; using the binary classification algorithm to determine the first predicted probability based on the second image data.

[0098] Optionally, infrared sensing data of the first area is obtained, and the probability of the presence of foreign matter in the first area is determined as a second predicted probability based on the infrared sensing data and the deep neural network, including: using active infrared technology to actively emit a first infrared light to the object in the washing machine, and receiving a second infrared light reflected by the object through a detector, wherein the active infrared technology is a technology in which an infrared light source actively emits infrared light to detect an object; converting the second infrared light into an electrical signal to obtain the infrared sensing data, and amplifying and filtering the infrared sensing data to obtain a processed digital signal; using image processing technology to generate an infrared image of the object based on the processed digital signal; and inputting the infrared image into the deep neural network to obtain the second predicted probability.

[0099] Optionally, according to the first predicted probability and the second predicted probability, determining the final predicted probability that a foreign object exists in the first region includes: obtaining a weighted average formula A=α SVM ×0.5+α Infrared ×0.5, where A is the final predicted probability, α SVM is the first predicted probability, α Infrared is the second predicted probability mentioned above; according to the weighted average formula mentioned above, the final predicted probability of the existence of foreign matter in the first area mentioned above is determined.

[0100] Optionally, after determining the final predicted probability of the presence of foreign matter in the first area based on the first predicted probability and the second predicted probability, the method further includes: when the final predicted probability of the presence of foreign matter in the first area is greater than or equal to a preset probability, stopping the washing program of the washing machine and issuing a warning message; when the final predicted probability of the presence of foreign matter in the first area is less than the preset probability, entering the washing program of the washing machine.

[0101] Optionally, when the final predicted probability of the presence of foreign matter in the first area is less than the preset probability, after entering the washing program of the washing machine, the method further includes: repeating the positioning step, the first acquisition step, the second acquisition step and the determination step until the final predicted probability is greater than or equal to the preset probability or the washing of the washing machine is completed.

[0102] An embodiment of the present invention provides a processor, and the processor is used to run a program, wherein the method for identifying foreign matter in a washing machine is executed when the program is run.

[0103] Specifically, the method for identifying foreign matter in a washing machine includes:

[0104] Step S201 (positioning step), obtaining first image data of an object in the washing machine, and using a target detection algorithm to position a first area on the object in the washing machine according to the first image data;

[0105] Specifically, visual sensors are used to monitor the laundry environment, and clothing pockets are located and clothing pocket areas are captured based on the YOLO target detection algorithm, effectively reducing the detection area for identifying foreign objects.

[0106] Step S202 (first acquisition step), acquiring second image data of the first region, and determining the probability of the presence of foreign matter in the first region as a first predicted probability based on the second image data and a binary classification algorithm;

[0107] Specifically, the pocket area is classified into two categories: whether there are paper foreign objects or not. If paper foreign objects are found, an alarm is triggered and the washing program is stopped to prevent paper scraps from contaminating clothes and making them difficult to clean, thereby improving the user experience.

[0108] Step S203 (second acquisition step), acquiring infrared sensor data of the first area, and determining the probability of the presence of foreign matter in the first area as a second predicted probability based on the infrared sensor data and the deep neural network;

[0109] Specifically, an infrared sensing detection branch is introduced at the same time to improve the accuracy of judgment, reduce false alarms, reduce losses, and improve user experience.

[0110] Step S204 (determination step), determining the final predicted probability of the existence of foreign matter in the first area according to the first predicted probability and the second predicted probability.

[0111] Specifically, the use of binary classification and infrared scanning weighted prediction of whether there is a paper foreign object in the pocket area is more accurate than judging based on only one condition, reducing false alarms, reducing losses, and improving user experience. At the same time, feedback is provided on false alarm data to continuously optimize the paper foreign object recognition performance.

[0112] An embodiment of the present invention provides a device, the device including a processor, a memory, and a program stored in the memory and executable on the processor, and when the processor executes the program, at least the following steps are implemented:

[0113] Step S201 (positioning step), obtaining first image data of an object in the washing machine, and using a target detection algorithm to position a first area on the object in the washing machine according to the first image data;

[0114] Specifically, visual sensors are used to monitor the laundry environment, and clothing pockets are located and clothing pocket areas are captured based on the YOLO target detection algorithm, effectively reducing the detection area for identifying foreign objects.

[0115] Step S202 (first acquisition step), acquiring second image data of the first region, and determining the probability of the presence of foreign matter in the first region as a first predicted probability based on the second image data and a binary classification algorithm;

[0116] Specifically, the pocket area is classified into two categories: whether there are paper foreign objects or not. If paper foreign objects are found, an alarm is triggered and the washing program is stopped to prevent paper scraps from contaminating clothes and making them difficult to clean, thereby improving the user experience.

[0117] Step S203 (second acquisition step), acquiring infrared sensor data of the first area, and determining the probability of the presence of foreign matter in the first area as a second predicted probability based on the infrared sensor data and the deep neural network;

[0118] Specifically, an infrared sensing detection branch is introduced at the same time to improve the accuracy of judgment, reduce false alarms, reduce losses, and improve user experience.

[0119] Step S204 (determination step), determining the final predicted probability of the existence of foreign matter in the first area according to the first predicted probability and the second predicted probability.

[0120] Specifically, the use of binary classification and infrared scanning weighted prediction of whether there is a paper foreign object in the pocket area is more accurate than judging based on only one condition, reducing false alarms, reducing losses, and improving user experience. At the same time, feedback is provided on false alarm data to continuously optimize the paper foreign object recognition performance.

[0121] The devices in this article can be servers, PCs, PADs, mobile phones, etc.

[0122] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program for initializing at least the following method steps:

[0123] Step S201 (positioning step), obtaining first image data of an object in the washing machine, and using a target detection algorithm to position a first area on the object in the washing machine according to the first image data;

[0124] Specifically, visual sensors are used to monitor the laundry environment, and clothing pockets are located and clothing pocket areas are captured based on the YOLO target detection algorithm, effectively reducing the detection area for identifying foreign objects.

[0125] Step S202 (first acquisition step), acquiring second image data of the first region, and determining the probability of the presence of foreign matter in the first region as a first predicted probability based on the second image data and a binary classification algorithm;

[0126] Specifically, the pocket area is classified into two categories: whether there are paper foreign objects or not. If paper foreign objects are found, an alarm is triggered and the washing program is stopped to prevent paper scraps from contaminating clothes and making them difficult to clean, thereby improving the user experience.

[0127] Step S203 (second acquisition step), acquiring infrared sensor data of the first area, and determining the probability of the presence of foreign matter in the first area as a second predicted probability based on the infrared sensor data and the deep neural network;

[0128] Specifically, an infrared sensing detection branch is introduced at the same time to improve the accuracy of judgment, reduce false alarms, reduce losses, and improve user experience.

[0129] Step S204 (determination step), determining the final predicted probability of the existence of foreign matter in the first area according to the first predicted probability and the second predicted probability.

[0130] Specifically, the use of binary classification and infrared scanning weighted prediction of whether there is a paper foreign object in the pocket area is more accurate than judging based on only one condition, reducing false alarms, reducing losses, and improving user experience. At the same time, feedback is provided on false alarm data to continuously optimize the paper foreign object recognition performance.

[0131] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0132] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0133] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0134] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0135] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0136] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0137] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0138] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0139] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0140] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0141] 1) The above-mentioned foreign body identification method in the washing machine of the present application first has a positioning step, which is to obtain the first image data of the object in the washing machine, and use the target detection algorithm to locate the first area on the object in the washing machine according to the first image data; then there is a first acquisition step, which is to obtain the second image data of the first area, and determine the probability of the existence of foreign bodies in the first area as the first predicted probability according to the second image data and the binary classification algorithm; then there is a second acquisition step, which is to obtain the infrared sensor data of the first area, and determine the probability of the existence of foreign bodies in the first area as the second predicted probability according to the infrared sensor data and the deep neural network; finally there is a determination step, which is to determine the final predicted probability of the existence of foreign bodies in the first area according to the first predicted probability and the second predicted probability. This method can detect paper foreign bodies in the first time, stop the washing program before they are diffused and damaged, and analyze the collected missed and false alarm information data, optimize the detection algorithm online, continuously optimize the detection performance, improve the user experience, and solve the problem that foreign bodies in the clothes in the existing washing machine cannot be distinguished.

[0142] 2) The foreign body identification device in the washing machine of the present application comprises: a positioning unit, a first acquisition unit, a second acquisition unit and a determination unit. The positioning unit is used to locate the first area on the object in the washing machine by using a target detection algorithm; the first acquisition unit is used to acquire real-time image data of the first area, and determine the probability of the existence of foreign bodies in the first area as a first predicted probability according to the real-time image data and the support vector machine model; the second acquisition unit is used to acquire infrared sensor data of the first area, and determine the probability of the existence of foreign bodies in the first area as a second predicted probability according to the infrared sensor data and the deep neural network; the determination unit is used to determine the final predicted probability of the existence of foreign bodies in the first area according to the first predicted probability and the second predicted probability. The device can detect paper foreign bodies at the first time, stop the washing program before they are diffused and damaged, and analyze the collected missed and false alarm information data, optimize the detection algorithm online, continuously optimize the detection performance, improve the user experience, and solve the problem that foreign bodies in clothes in the existing washing machine cannot be distinguished.

[0143] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for identifying foreign matter in a washing machine, characterized in that: include: a positioning step of acquiring first image data of the object in the washing machine, and using a target detection algorithm to locate a first area on the object in the washing machine according to the first image data; A first acquisition step, acquiring second image data of the first area, and determining a probability of a foreign object existing in the first area as a first predicted probability based on the second image data and a binary classification algorithm; A second acquisition step, acquiring infrared sensor data of the first area, and determining a probability that a foreign object exists in the first area as a second predicted probability based on the infrared sensor data and a deep neural network; A determination step is performed to determine a final predicted probability of the presence of foreign matter in the first area based on the first predicted probability and the second predicted probability.

2. The method according to claim 1, characterized in that Acquiring first image data of an object in a washing machine, and using a target detection algorithm to locate a first area on the object in the washing machine according to the first image data, comprises: controlling the washing machine to enter a waterless rotation mode, and acquiring first image data of the object in the washing machine; A YOLO target detection algorithm is constructed based on the backbone network, the feature enhancement network and the target detection head, wherein the YOLO target detection algorithm is trained using multiple sets of training data, and each set of training data in the multiple sets of training data includes: image data of historical objects in the washing machine and label data of the historical objects corresponding to the image data of the historical objects acquired within a historical time period; The first image data is input into the YOLO target detection algorithm to obtain a positioning result of the first area.

3. The method according to claim 1, characterized in that: Acquiring second image data of the first area, and determining a probability that a foreign object exists in the first area as a first predicted probability according to the second image data and a binary classification algorithm, comprising: intercepting the second image data from the first image data; Obtain a binary classification algorithm, wherein the binary classification algorithm is one of the following: support vector machine, random forest algorithm, decision tree algorithm; The first prediction probability is determined according to the second image data using the binary classification algorithm.

4. The method according to claim 1, characterized in that: Acquiring infrared sensor data of the first area, and determining a probability of a foreign object existing in the first area as a second predicted probability according to the infrared sensor data and a deep neural network, including: Active infrared technology is used to actively emit a first infrared light to the object in the washing machine, and a detector is used to receive a second infrared light reflected by the object. The active infrared technology is a technology in which an infrared light source actively emits infrared light to detect the object. Converting the second infrared light into an electrical signal to obtain the infrared sensing data, and amplifying and filtering the infrared sensing data to obtain a processed digital signal; generating an infrared image of the object using image processing technology according to the processed digital signal; The infrared image is input into the deep neural network to obtain the second prediction probability.

5. The method according to claim 1, characterized in that Determining a final predicted probability that a foreign object exists in the first area according to the first predicted probability and the second predicted probability includes: Get the weighted average formula A = α SVM ×0.5+α Infrared ×0.5, where A is the final predicted probability, α SVM is the first predicted probability, α Infrared is the second predicted probability; According to the weighted average formula, a final predicted probability of the presence of foreign matter in the first area is determined.

6. The method according to claim 1, characterized in that After determining a final predicted probability that a foreign object exists in the first area according to the first predicted probability and the second predicted probability, the method further includes: When the final predicted probability of the existence of foreign matter in the first area is greater than or equal to a preset probability, stopping the washing program of the washing machine and issuing a warning message; When the final predicted probability of the existence of foreign matter in the first area is less than the preset probability, the washing program of the washing machine is entered.

7. The method according to claim 6, characterized in that When the final predicted probability of the existence of foreign matter in the first area is less than the preset probability, after entering the washing program of the washing machine, the method further includes: The positioning step, the first acquisition step, the second acquisition step and the determination step are repeatedly performed until the final predicted probability is greater than or equal to the preset probability or the washing machine completes washing.

8. A device for identifying foreign matter in a washing machine, characterized in that: include: a positioning unit, configured to locate a first area on the object in the washing machine using a target detection algorithm; A first acquisition unit, configured to acquire real-time image data of the first area, and determine, based on the real-time image data and a support vector machine model, a probability that a foreign object exists in the first area as a first predicted probability; A second acquisition unit, configured to acquire infrared sensor data of the first area, and determine a probability that a foreign object exists in the first area as a second predicted probability according to the infrared sensor data and a deep neural network; A determination unit is used to determine a final predicted probability of the existence of a foreign object in the first area according to the first predicted probability and the second predicted probability.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method for identifying foreign matter in a washing machine according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing the method for identifying foreign matter in a washing machine as described in any one of claims 1 to 7.

Images