A fault warning remote management method for intelligent cleaning robots

Abstract

The present invention relates to the technical field of fault warning, and in particular to a fault warning remote management method applied to an intelligent cleaning robot, the method comprising: a remote management terminal controls a data acquisition device to take a photo of a product, obtains a dirty image of the product before cleaning, inputs the dirty image into a pre-trained machine learning model, and outputs a decontamination image; imports the dirty image and the decontamination image into an image comparison strategy to calculate the contamination index of the product before cleaning, and marks the degree of contamination of different areas of the product before cleaning; obtains a later image of the product after cleaning, imports the later image and the decontamination image into an image comparison strategy to calculate the contamination index of the product after cleaning, and calculates the change in the degree of contamination of different areas of the product after cleaning; according to the change in the degree of contamination of different areas of the product, it is judged whether the intelligent cleaning robot has a fault and the feedback is sent to the remote management terminal. The data processing volume is small, and the processing result is accurate, more intuitive and effective.

Description

Background Art

[0002] As an important part of modern home automation, smart cleaning robots mainly achieve autonomous cleaning through sensors, artificial intelligence algorithms and mechanical control systems. Smart robots usually have networking capabilities and can be connected to remote servers through the Internet of Things (IoT). Through the remote management system, users and service providers can remotely monitor the operating status of the equipment, perform fault diagnosis, and remotely repair or provide software updates when necessary. Through such a remote management system, maintenance costs can be reduced and user experience can be improved. However, as its complexity increases, the failure rate of equipment is also increasing, which leads to high equipment maintenance costs and poor user experience.

[0003] Problems in the existing technology: Most of the existing intelligent cleaning robot fault diagnosis solutions use big data and AI analysis technology to extract equipment operation rules from a large amount of data, identify potential faults, and predict the health status of the equipment. This fault judgment solution has too much data processing. During long-term use, the intelligent cleaning robot needs to record a large amount of operation data, but the data analysis capability of the existing system is limited, and it is impossible to effectively analyze and predict complex fault modes, which affects the maintenance efficiency of the equipment. Therefore, it is necessary to propose a fault warning remote management method for intelligent cleaning robots. Summary of the invention

[0004] The main purpose of the present invention is to provide a fault warning remote management method applied to an intelligent cleaning robot, which effectively solves the above-mentioned problems mentioned in the background technology.

[0005] The technical solution of the present invention is as follows:

[0006] In the first aspect, a fault warning remote management method for an intelligent cleaning robot is proposed, the method comprising the following steps:

[0007] S1. The remote management terminal controls the data acquisition device to take pictures of the product, obtain the dirty image of the product before cleaning, input the dirty image into the pre-trained machine learning model, and output the decontaminated image;

[0008] S2, importing the contaminated image and the decontaminated image into the image comparison strategy to calculate the contamination index of the product before cleaning, and marking the contamination degree of different areas of the product before cleaning;

[0009] S3, obtaining a later image of the product after cleaning, importing the later image and the decontamination image into the image comparison strategy to calculate the contamination index of the product after cleaning, and calculating the change of contamination degree of different areas of the product after cleaning;

[0010] S4. Based on the changes in the degree of contamination in different areas of the product, determine whether the intelligent cleaning robot has a fault and provide feedback to the remote management terminal.

[0011] A further improvement of the present invention is that S1 comprises the following specific steps:

[0012] S11, extracting the intensity of at least one color channel of each pixel of the contaminated image to form a color intensity set of the contaminated image, and simultaneously obtaining dust concentration data at the shooting location, light intensity data at the shooting location, and distance data from the shooting location to the product;

[0013] S12. Input the color intensity set of the contaminated image, the dust concentration data at the shooting location, the light intensity data at the shooting location, and the distance data from the shooting location to the product into a pre-trained machine learning model, and output a decontaminated image of the product.

[0014] A further improvement of the present invention is that the training method of the machine learning model in S1 is:

[0015] S121, obtaining images of the product in a contaminated state and a decontaminated state, and obtaining dust concentration data at the shooting position, light intensity data at the shooting position, and distance data from the shooting position to the product;

[0016] S122. Combine the contaminated image, the dust concentration data at the shooting location, the light intensity data at the shooting location, and the distance data from the shooting location to the product into the form of a feature vector, and use the set of all feature vectors as the input of a machine learning model. The machine learning model uses the decontaminated image predicted by each set of feature vectors as the output, and the actual decontaminated image corresponding to each set of feature vectors as the prediction target. Minimizing the sum of the prediction accuracies of all predicted decontamination images is used as the training target. The machine learning model is trained until the sum of the prediction accuracies converges and the training is stopped.

[0017] A further improvement of the present invention is that S2 comprises the following specific steps:

[0018] S21: Divide the contaminated image and the decontaminated image of the product before cleaning into pixels, extract the pixel value of each pixel, and form a pixel value sequence of the contaminated image , decontaminated image pixel value sequence ,in Represents the pixel value of the nth pixel of the dirty image, Represents the pixel value of the nth pixel of the decontaminated image;

[0019] S22: Calculate the contamination index of the product before cleaning. The calculation formula is: ,in It is the contamination index of the product before cleaning.

[0020] A further improvement of the present invention is that the step S2 further includes: presetting a pollution threshold a to divide the pollution degree of different areas of the product before cleaning. When , the product area before cleaning corresponding to the i-th pixel point of the dirty image is marked as the deep cleaning area. When , the product area before cleaning corresponding to the i-th pixel point of the dirty image is marked as a normal cleaning area.

[0021] A further improvement of the present invention is that S3 comprises the following specific steps:

[0022] S31, after the intelligent cleaning robot performs cleaning, obtain a later image of the product after cleaning, divide the later image of the product after cleaning into pixels, extract the pixel value of each pixel, and form a later image pixel value sequence , Represents the pixel value of the nth pixel in the later image;

[0023] S32. Calculate the contamination index of the product after cleaning. The calculation formula is: ;in It is the contamination index of the product after cleaning.

[0024] A further improvement of the present invention is that S3 further includes: calculating the change in contamination degree of the deep cleaning area and the ordinary cleaning area of ​​the product before cleaning, and the calculation formulas are: ;

[0025] ;in is the pollution degree change of m pixels marked as deep cleaning area, is the pollution degree change of nm pixels marked as normal clean area.

[0026] A further improvement of the present invention is that S4 comprises the following specific steps:

[0027] S41: Preset pollution index change threshold , the first change threshold , the second change threshold ,Compare and The size relationship is When the cleaning is up to standard and the robot has no faults, When the cleaning fails to meet the standard, the process proceeds to step S42;

[0028] S42: Comparison and The size relationship is When the robot has no fault, When the deep cleaning area is not cleaned up to standard, proceed to step S43;

[0029] S43: Comparison and The size relationship is When the robot has no fault, When the cleaning of the ordinary cleaning area does not meet the standards and the robot fails, the judgment result will be fed back to the remote management terminal.

[0030] In a second aspect, a computer-readable storage medium is proposed, on which a computer program is stored. When the computer program is executed, a remote management method for fault warning applied to an intelligent cleaning robot as described in any one of the above items is implemented.

[0031] In a third aspect, an electronic device is proposed, comprising a memory for storing instructions; and a processor for executing the instructions, so that the device implements a fault warning remote management method applied to an intelligent cleaning robot.

[0032] The technical effects of the present invention are as follows:

[0033] A remote management method for fault warning applied to intelligent cleaning robots is constructed. The remote management terminal controls the data acquisition device to take pictures of the product, obtain the dirty image of the product before cleaning, input the dirty image into the pre-trained machine learning model, and output the decontaminated image; import the dirty image and the decontaminated image into the image comparison strategy to calculate the contamination index of the product before cleaning, and mark the degree of contamination in different areas of the product before cleaning; obtain the later image of the product after cleaning, import the later image and the decontaminated image into the image comparison strategy to calculate the contamination index of the product after cleaning, and calculate the change of contamination degree in different areas of the product after cleaning; according to the change of contamination degree in different areas of the product, judge whether the intelligent cleaning robot has a fault and feedback to the remote management terminal. This solution judges the potential fault of the intelligent cleaning robot based on the actual cleaning results and combined with image analysis, without the need to analyze a large amount of operating data, with a small amount of data processing, accurate processing results, and more intuitive and effective. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0035] Figure 1 This is a flow chart of a fault warning remote management method applied to an intelligent cleaning robot according to Embodiment 1 of the present invention;

[0036] Figure 2 This is a schematic diagram of the detailed steps of a fault warning remote management method applied to an intelligent cleaning robot according to Example 1 of the present invention. DETAILED DESCRIPTION

[0037] Embodiment 1:

[0038] This embodiment aims to solve the problem that the fault judgment of the intelligent cleaning robot in the prior art is too cumbersome and the amount of data processing is large. A remote management method for fault warning applied to the intelligent cleaning robot is constructed. The remote management terminal controls the data acquisition device to take pictures of the product, obtain the dirty image of the product before cleaning, input the dirty image into the pre-trained machine learning model, and output the decontamination image; import the dirty image and the decontamination image into the image comparison strategy to calculate the pollution index of the product before cleaning, and mark the pollution degree of different areas of the product before cleaning; obtain the later image of the product after cleaning, import the later image and the decontamination image into the image comparison strategy to calculate the pollution index of the product after cleaning, and calculate the change of the pollution degree of different areas of the product after cleaning; according to the change of the pollution degree of different areas of the product, judge whether the intelligent cleaning robot has a fault and feedback to the remote management terminal. This solution judges the potential fault of the intelligent cleaning robot based on the actual cleaning results and combined with image analysis, without the need to analyze a large amount of operating data, with a small amount of data processing, and the processing results are accurate and more intuitive and effective.

[0039] A fault warning remote management method for an intelligent cleaning robot, such as Figure 1-Figure 2 As shown, the specific steps include:

[0040] S1. The remote management terminal controls the data acquisition device to take pictures of the product, obtain the dirty image of the product before cleaning, input the dirty image into the pre-trained machine learning model, and output the decontaminated image;

[0041] S2, importing the contaminated image and the decontaminated image into the image comparison strategy to calculate the contamination index of the product before cleaning, and marking the contamination degree of different areas of the product before cleaning;

[0042] S3, obtaining a later image of the product after cleaning, importing the later image and the decontamination image into the image comparison strategy to calculate the contamination index of the product after cleaning, and calculating the change of contamination degree of different areas of the product after cleaning;

[0043] S4. Based on the changes in the degree of contamination in different areas of the product, determine whether the intelligent cleaning robot has a fault and provide feedback to the remote management terminal.

[0044] In this embodiment, S1 includes the following specific steps:

[0045] S11, extracting the intensity of at least one color channel of each pixel of the contaminated image to form a color intensity set of the contaminated image, and simultaneously obtaining dust concentration data at the shooting location, light intensity data at the shooting location, and distance data from the shooting location to the product;

[0046] S12. Input the color intensity set of the contaminated image, the dust concentration data at the shooting location, the light intensity data at the shooting location, and the distance data from the shooting location to the product into a pre-trained machine learning model, and output a decontaminated image of the product.

[0047] In this embodiment, the training method of the machine learning model in S1 is:

[0048] S121, obtaining images of the product in a contaminated state and a decontaminated state, and obtaining dust concentration data at the shooting position, light intensity data at the shooting position, and distance data from the shooting position to the product;

[0049] S122. Combine the contaminated image, the dust concentration data at the shooting location, the light intensity data at the shooting location, and the distance data from the shooting location to the product into the form of a feature vector, and use the set of all feature vectors as the input of a machine learning model. The machine learning model uses the decontaminated image predicted by each set of feature vectors as the output, and the actual decontaminated image corresponding to each set of feature vectors as the prediction target. Minimizing the sum of the prediction accuracies of all predicted decontamination images is used as the training target. The machine learning model is trained until the sum of the prediction accuracies converges and the training is stopped.

[0050] In this embodiment, S2 includes the following specific steps:

[0051] S21: Divide the contaminated image and the decontaminated image of the product before cleaning into pixels, extract the pixel value of each pixel, and form a pixel value sequence of the contaminated image , decontaminated image pixel value sequence ,in Represents the pixel value of the nth pixel of the dirty image, Represents the pixel value of the nth pixel of the decontamination image;

[0052] S22: Calculate the contamination index of the product before cleaning. The calculation formula is: ,in It is the contamination index of the product before cleaning.

[0053] In this embodiment, S2 further includes: presetting a pollution threshold a to classify the pollution degree of different areas of the product before cleaning, presetting a pollution threshold a to classify the pollution degree of different areas of the product before cleaning, When , the product area before cleaning corresponding to the i-th pixel point of the dirty image is marked as the deep cleaning area. When , the product area before cleaning corresponding to the i-th pixel point of the dirty image is marked as a normal cleaning area.

[0054] In this embodiment, S3 includes the following specific steps:

[0055] S31, after the intelligent cleaning robot performs cleaning, obtain a later image of the product after cleaning, divide the later image of the product after cleaning into pixels, extract the pixel value of each pixel, and form a later image pixel value sequence , Represents the pixel value of the nth pixel in the later image;

[0056] S32. Calculate the contamination index of the product after cleaning. The calculation formula is: ;in It is the contamination index of the product after cleaning.

[0057] In this embodiment, S3 further includes: calculating the change in contamination degree of the deep cleaning area and the ordinary cleaning area of ​​the product before cleaning, and calculating the change in contamination degree of the deep cleaning area and the ordinary cleaning area of ​​the product before cleaning, and the calculation formulas are: ;

[0058] ;in is the pollution degree change of m pixels marked as deep cleaning area, is the pollution degree change of nm pixels marked as normal clean area.

[0059] In this embodiment, S4 includes the following specific steps:

[0060] S41: Preset pollution index change threshold , the first change threshold , the second change threshold ,Compare and The size relationship is When the cleaning is up to standard and the robot has no faults, When the cleaning fails to meet the standard, the process proceeds to step S42;

[0061] S42: Comparison and The size relationship is When the robot has no fault, When the deep cleaning area is not cleaned up to standard, proceed to step S43;

[0062] S43: Comparison and The size relationship is When the robot has no fault, When the cleaning of the ordinary cleaning area does not meet the standards and the robot fails, the judgment result will be fed back to the remote management terminal.

[0063] Embodiment 2:

[0064] This embodiment provides an electronic device, including: a processor and a memory, wherein the memory stores a computer program that can be called by the processor; the processor executes the above-mentioned fault warning remote management method applied to an intelligent cleaning robot by calling the computer program stored in the memory.

[0065] The electronic device may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) and one or more memories, wherein the memory stores at least one computer program, and the computer program is loaded and executed by the processor to implement a fault warning remote management method for an intelligent cleaning robot provided in the above method embodiment. The electronic device may also include other components for realizing the functions of the device, for example, the electronic device may also have components such as a wired or wireless network interface and an input and output interface for data input and output. This embodiment will not be described in detail here.

[0066] Those skilled in the art know that the present invention can be implemented as a system, method or computer program product. Therefore, the present disclosure can be specifically implemented in the following forms, namely: it can be complete hardware, it can be complete software (including firmware, resident software, microcode, etc.), or it can be a combination of hardware and software, which is generally referred to as a "circuit", "module" or "system" herein. In addition, in some embodiments, the present invention can also be implemented in the form of a computer program product in one or more computer-readable media, and the computer-readable medium contains computer-readable program code.

[0067] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.

[0068] The present invention is described with reference to flowcharts and block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process or block in the flowchart and block diagram, as well as the combination of processes and blocks in the flowchart or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts. Figure 1 Process or multiple processes and boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0069] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 Process or multiple processes and boxes Figure 1 The steps for the functions specified in one or more boxes.

[0070] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

Claims

1. A fault warning remote management method applied to an intelligent cleaning robot, characterized in that: The method comprises the following specific steps: S1. The remote management terminal controls the data acquisition device to take pictures of the product, obtain the dirty image of the product before cleaning, input the dirty image into the pre-trained machine learning model, and output the decontaminated image; S2, importing the contaminated image and the decontaminated image into the image comparison strategy to calculate the contamination index of the product before cleaning, and marking the contamination degree of different areas of the product before cleaning; S3, obtaining a later image of the product after cleaning, importing the later image and the decontamination image into the image comparison strategy to calculate the contamination index of the product after cleaning, and calculating the change of contamination degree of different areas of the product after cleaning; S4. According to the changes in the degree of contamination in different areas of the product, determine whether the intelligent cleaning robot has failed and feedback to the remote management terminal; The S3 includes the following specific steps: S31, after the intelligent cleaning robot performs cleaning, obtain a later image of the product after cleaning, divide the later image of the product after cleaning into pixels, extract the pixel value of each pixel, and form a later image pixel value sequence , Represents the pixel value of the nth pixel in the later image; S32. Calculate the contamination index of the product after cleaning. The calculation formula is: ;in It is the contamination index of the product after cleaning; The S3 also includes: calculating the change in the degree of contamination of the deep cleaning area and the ordinary cleaning area of ​​the product before cleaning, and the calculation formulas are: ; ;in is the pollution degree change of m pixels marked as deep cleaning area, is the pollution degree change of nm pixels marked as normal clean area.

2. A fault warning remote management method for an intelligent cleaning robot according to claim 1, characterized in that: The S1 comprises the following specific steps: S11, extracting the intensity of at least one color channel of each pixel of the contaminated image to form a color intensity set of the contaminated image, and simultaneously obtaining dust concentration data at the shooting location, light intensity data at the shooting location, and distance data from the shooting location to the product; S12. Input the color intensity set of the contaminated image, the dust concentration data at the shooting location, the light intensity data at the shooting location, and the distance data from the shooting location to the product into a pre-trained machine learning model, and output a decontaminated image of the product.

3. A fault warning remote management method for an intelligent cleaning robot according to claim 2, characterized in that: The training method of the machine learning model in S1 is: S121, obtaining images of the product in a contaminated state and a decontaminated state, and obtaining dust concentration data at the shooting position, light intensity data at the shooting position, and distance data from the shooting position to the product; S122. Combine the contaminated image, the dust concentration data at the shooting location, the light intensity data at the shooting location, and the distance data from the shooting location to the product into the form of a feature vector, and use the set of all feature vectors as the input of a machine learning model. The machine learning model uses the decontaminated image predicted by each set of feature vectors as the output, and the actual decontaminated image corresponding to each set of feature vectors as the prediction target. Minimizing the sum of the prediction accuracies of all predicted decontamination images is used as the training target. The machine learning model is trained until the sum of the prediction accuracies converges and the training is stopped.

4. A fault warning remote management method for an intelligent cleaning robot according to claim 3, characterized in that: The S2 comprises the following specific steps: S21: Divide the contaminated image and the decontaminated image of the product before cleaning into pixels, extract the pixel value of each pixel, and form a pixel value sequence of the contaminated image , decontaminated image pixel value sequence ,in Represents the pixel value of the nth pixel of the dirty image, Represents the pixel value of the nth pixel of the decontamination image; S22: Calculate the contamination index of the product before cleaning. The calculation formula is: ,in It is the contamination index of the product before cleaning.

5. A fault warning remote management method for an intelligent cleaning robot according to claim 4, characterized in that: The step S2 also includes: presetting a pollution threshold a to classify the pollution degree of different areas of the product before cleaning. When , the product area before cleaning corresponding to the i-th pixel point of the dirty image is marked as the deep cleaning area. When , the product area before cleaning corresponding to the i-th pixel point of the dirty image is marked as a normal cleaning area.

6. A fault warning remote management method for an intelligent cleaning robot according to claim 5, characterized in that: The S4 comprises the following specific steps: S41: Preset pollution index change threshold , the first change threshold , the second change threshold ,Compare and The size relationship is When the cleaning is up to standard and the robot has no faults, When the cleaning fails to meet the standard, the process proceeds to step S42; S42: Comparison and The size relationship is When the robot has no fault, When the deep cleaning area is not cleaned up to standard, proceed to step S43; S43: Comparison and The size relationship is When the robot has no fault, When the cleaning of the ordinary cleaning area does not meet the standards and the robot fails, the judgment result will be fed back to the remote management terminal.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, a fault warning remote management method for an intelligent cleaning robot as described in any one of claims 1 to 6 is implemented.

8. An electronic device, characterized in that: include: A memory for storing instructions; A processor is used to execute the instruction so that the device performs operations to implement a fault warning remote management method applied to an intelligent cleaning robot as described in any one of claims 1 to 6.

Images