A method, device, and cleaning robot for detecting surface stains

Abstract

The present invention relates to the field of autonomous driving technology, and provides a method and device for detecting surface stains and a cleaning robot. The method includes: acquiring an image, and preprocessing the image to obtain a grayscale image; performing edge detection to obtain an edge image; dividing the edge image into a plurality of sub-edge images, and selecting an edge image with a stain boundary therefrom; and obtaining a stain area in the image according to the edge image with the stain boundary. By the above method, the actual size and shape of the stain in the image can be estimated more accurately.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a method and device for detecting surface stains, and a cleaning robot. Background Art

[0002] The application of autonomous driving technology in ground vehicles has matured. It is efficient, safe, and can extend vehicle lifespan. Applying autonomous driving technology to cleaning robots can replace drivers, significantly increasing their operating time and ensuring they can consistently maintain a clean environment. Accurately and promptly identifying stains on surfaces like floor tiles and glass can significantly improve cleaning efficiency and effectiveness.

[0003] The prior art lacks a method for detecting stains on the surface of an object. Summary of the Invention

[0004] The embodiments of the present invention aim to provide a method, device and cleaning robot for detecting surface stains, which mainly solve the technical problems of related technologies in detecting stains on the surface of objects.

[0005] To achieve the above object, according to one aspect of the present invention, a method for detecting surface stains is provided, comprising:

[0006] Acquiring an image, and preprocessing the image to obtain a grayscale image;

[0007] Performing canny edge detection calculation on the grayscale image to obtain an edge image;

[0008] Dividing the edge image into a plurality of sub-edge images, and selecting an edge image with a stain boundary from the sub-edge images;

[0009] Merging any two of the stored sub-edge images that meet the overlapping condition;

[0010] further screening the stored merged sub-edge image and the single sub-edge image according to the next frame image to obtain a result image;

[0011] According to the result image, a stain area in the image is obtained.

[0012] Optionally, the step of dividing the edge image into a plurality of sub-edge images and selecting an edge image with a stain boundary therefrom further comprises:

[0013] dividing the edge image into a plurality of sub-edge images;

[0014] Traversing all the sub-edge images, and determining that the number of edge points in the sub-edge images is greater than a first threshold;

[0015] Statistically calculate the angles of the lines connecting any two of the edge points, and determine whether the proportion of the number of occurrences of the lines with the angle being the second threshold among all the occurrences of the lines is greater than the third threshold;

[0016] If so, discard the sub-edge image;

[0017] If not, store the sub-edge image;

[0018] If the number of edge points in the sub-edge image is less than or equal to the first threshold, discard the sub-edge image.

[0019] Optionally, the step of merging any two of the stored sub-edge images that meet the overlapping condition further includes:

[0020] Calculate the overlapping rate of any two of the stored sub-edge images, and determine whether the overlapping rate is greater than the fourth threshold;

[0021] If so, merge the two sub-edge images, take the coordinate mean of all the edge points in the merged sub-edge image as the center of gravity, and store it;

[0022] If not, store it as a single sub-edge image.

[0023] On the other hand, the present invention provides a surface stain detection device, including:

[0024] An acquisition module, configured to acquire an image and preprocess the image to obtain a grayscale image;

[0025] A calculation module, configured to perform canny edge detection calculation on the grayscale image to obtain an edge image;

[0026] A selection module, configured to divide the edge image into several sub-edge images and select the edge images with stain boundaries from them;

[0027] A merging module, configured to merge any two of the stored sub-edge images that meet the overlapping condition;

[0028] A screening module, configured to further screen the stored merged sub-edge images and single sub-edge images according to the next frame of image to obtain a result image;

[0029] A obtaining module, configured to obtain the stain area in the image according to the result image.

[0030] Optionally, the acquisition module specifically includes:

[0031] An acquisition unit, configured to acquire an image;

[0032] A reduction unit for reducing the size of the image;

[0033] A conversion unit for converting the reduced image into a grayscale image.

[0034] Optionally, the selection module specifically includes

[0035] A division unit for dividing the edge image into a plurality of sub-edge images;

[0036] A determination unit for traversing all the sub-edge images and determining that the number of edge points in the sub-edge image is greater than a first threshold;

[0037] A statistics unit for counting the angles of the lines connecting any two of the edge points, and determining whether the proportion of the number of occurrences of the lines with the angle being a second threshold in the total number of occurrences of all the lines is greater than a third threshold;

[0038] A discard unit for discarding the sub-edge image;

[0039] A first storage unit for storing the sub-edge image.

[0040] Optionally, the merging module specifically includes

[0041] A judgment unit for calculating the overlap rate between any two of the stored sub-edge images and determining whether the overlap rate is greater than a fourth threshold;

[0042] A merging unit for merging two of the sub-edge images, taking the coordinate mean of all the edge points in the merged sub-edge image as the centroid, and storing it;

[0043] A second storage unit for storing the sub-edge image as a single one.

[0044] According to another aspect of the present invention, a cleaning robot is provided, including:

[0045] At least one processor;

[0046] A memory communicatively connected to the at least one processor;

[0047] Wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in any one of the above methods for detecting surface stains.

[0048] Compared with the prior art, a method for detecting surface stains provided by an embodiment of the present invention includes obtaining an image, preprocessing the image to obtain a grayscale image, performing edge detection to obtain an edge image, dividing the edge image into several sub-edge images, selecting an edge image with a stain boundary from them, and obtaining the stain area in the image according to the edge image with the stain boundary. In this way, the actual size and shape of the stains in the image can be estimated more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] One or more embodiments are illustrated by way of example in the accompanying drawings, which do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are represented as similar elements, and unless otherwise stated, the drawings in the figures do not constitute a scale limitation.

[0050] Figure 1 It is a flowchart of a method for detecting surface stains provided by an embodiment of the present invention;

[0051] Figure 2 For Figure 1 It is a sub-flowchart of S110 in

[0052] Figure 3 For Figure 1 It is a sub-flowchart of S130 in

[0053] Figure 4 For Figure 1 It is a sub-flowchart of S140 in

[0054] Figure 5 It is a schematic structural diagram of a device for quickly cleaning floor liquid provided by an embodiment of the present invention;

[0055] Figure 6 For Figure 5 It is a schematic structural diagram of the selection module in

[0056] Figure 7 For Figure 5 It is a schematic structural diagram of the merging module in

[0057] Figure 8 It is a structural block diagram of a cleaning robot provided by another embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0058] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0059] It should be noted that if there is no conflict, the various features in the embodiments of the present invention can be combined with each other, and all are within the protection scope of the present invention. In addition, although the functional modules are divided in the device schematic diagram and the logical sequence is shown in the flowchart, in some cases, the steps shown or described can be executed in a different module division from that in the device schematic diagram or a different sequence from that in the flowchart.

[0060] In addition, the technical features involved in different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0061] An embodiment of the present invention provides a method for detecting surface stains, which can be applied to a cleaning robot. Among them, the cleaning robot can be an unmanned cleaning robot, which can automatically clean the garbage on the road, automatically bypass obstacles, drive automatically, turn around automatically, pass traffic lights automatically, park automatically, etc. Moreover, the areas that can be cleaned include underground parking lots, indoor playgrounds, or large indoor shopping malls, etc.

[0062] Please refer to Figure 1 , an embodiment of the present invention provides a method for detecting surface stains, which can be executed by a cleaning robot and includes the following steps:

[0063] S110. Obtain an image and preprocess the image to obtain a grayscale image.

[0064] Specifically, use an on-vehicle camera to collect the video during the vehicle driving. Extract images from the captured video. Images can be extracted for each frame, or images can be extracted every multiple frames. As Figure 2 shown, it shows that the sub-process of S110 specifically includes:

[0065] S111. Obtain an image;

[0066] S112. Reduce the size of the image;

[0067] In this embodiment, the size of the image is 640*480, and the image is reduced to 160*120.

[0068] S113. Convert the reduced image into a grayscale image.

[0069] S120. Perform canny edge detection calculation on the grayscale image to obtain an edge image.

[0070] The edge information of an image is mainly concentrated in the high-frequency band. Generally speaking, image sharpening or edge detection is essentially high-frequency filtering. We know that the differential operation is to find the change rate of a signal and has the effect of enhancing high-frequency components. In spatial domain operations, sharpening an image is to calculate the differential. Due to the discrete signal of a digital image, the differential operation becomes calculating the difference or gradient. There are various edge detection (gradient) operators in image processing, and the commonly used ones include ordinary first-order difference, Robert operator (cross difference), Sobel operator, etc., which are based on finding the gradient intensity. The Laplace operator (second-order difference) is based on zero-crossing detection. By calculating the gradient and setting a threshold, an edge image is obtained. In this embodiment, the canny initial threshold is `thresc = [150, 180]`.

[0071] S130. Divide the edge image into several sub-edge images, and select the edge image with the stain boundary from them.

[0072] As Figure 3 shown, it shows that the sub-process of S130 specifically includes:

[0073] S131. Divide the edge image into several sub-edge images.

[0074] Specifically, perform a block processing on the edge image. The size of the edge image is 160 * 120, and it is divided into 8 * 6 sub-edge images, and the size of the sub-edge image is 20 * 20.

[0075] S132. Traverse all the sub-edge images, and determine that the number of edge points in the sub-edge image is greater than the first threshold.

[0076] S133. Statistically calculate the angles of the connections between any two of the edge points, and determine whether the proportion of the number of times the connections with the second threshold appear in all the connection times is greater than the third threshold.

[0077] S134. If so, discard the sub-edge image.

[0078] In this embodiment, the second threshold is based on the included angle between the current camera and the floor tile boundary line. When the proportion of the number of times the connections with the second threshold appear in all the connection times is greater than the third threshold (set to 20% in this embodiment), it is considered that the sub-edge image contains a long straight line with the second threshold, and this long straight line is the floor tile boundary line, so it is discarded.

[0079] S135. If not, store the sub-edge image.

[0080] In addition, if the number of edge points in the sub-edge image is less than or equal to the first threshold, discard the sub-edge image.

[0081] S140. Merge any two of the stored sub-edge images that meet the overlap condition.

[0082] As Figure 4 shown, it shows that the sub-process of S140 specifically includes:

[0083] S141. Calculate the overlap rate of any two of the stored sub-edge images, and determine whether the coincidence rate is greater than the fourth threshold.

[0084] S142. If so, merge the two sub-edge images, and take the coordinate mean of all edge points in the merged sub-edge image as the centroid and store it.

[0085] S143. If not, store it as a single sub-edge image.

[0086] S150. Further screen the stored merged sub-edge images and single sub-edge images according to the next frame of image to obtain the result image.

[0087] Specifically, add a first counter and a second counter to each sub-edge image (including the merged sub-edge image and the single sub-edge image) corresponding to the current frame of image. The next frame of image is processed according to the aforementioned steps 110 to 140. If the merger of sub-edge images occurs, the first counter is incremented by 1, and the second counter is reset to 0; if no merger occurs, the first counter remains unchanged, and the second counter is incremented by 1. If the second counter of a certain sub-edge image is greater than 10, it means that within 10 consecutive frames of images, the neighboring sub-edge images where this sub-edge image is located have not been retained, and this sub-edge image is deleted. If the first counter of a certain sub-edge image is greater than 3, this sub-edge image has been merged more than three times, and the confidence level is relatively high, and the screened sub-edge image is stored as the result image.

[0088] S160. Obtain the stain area in the image according to the result image.

[0089] Specifically, create a black image with the same size as the edge image as the mask image. Convert the result image into a circular image (with the centroid of the result image as the center and the average of the length and width of the result image as the radius). In the above circular image, if the distance between the centers of two circles < 1.25 * the sum of the radii of the two circles, the two circles are adjacent. Between adjacent circles, draw a white rounded-corner straight line with a thickness equal to the sum of the radii of the two circles on the mask image with the two centers as endpoints. The white part on this mask image is the stain area.

[0090] Compared with the prior art, a method for detecting surface stains provided by an embodiment of the present invention includes obtaining an image and preprocessing the image to obtain a grayscale image; performing edge detection to obtain an edge image; dividing the edge image into several sub-edge images, and selecting an edge image with a stain boundary from them; obtaining the stain area in the image according to the edge image with the stain boundary. In the above manner, the actual size and shape of the stains in the image can be estimated more accurately.

[0091] It should be noted that in the above method embodiment, there is not necessarily a certain order among the above steps. Those of ordinary skill in the art can understand according to the description of the embodiments of the present application that in different embodiments, the above steps can have different execution orders, that is, they can be executed in parallel, or they can be exchanged and executed, etc.

[0092] Please refer to Figure 5 , an embodiment of the present application provides a surface stain detection device 10, including:

[0093] An acquisition module 11, configured to acquire an image and preprocess the image to obtain a grayscale image. Specifically, the acquisition module 11 specifically includes an acquisition unit for acquiring an image; a reduction unit for reducing the size of the image; and a conversion unit for converting the reduced image into a grayscale image.

[0094] A calculation module 12, configured to perform canny edge detection calculation on the grayscale image to obtain an edge image.

[0095] A selection module 13, configured to divide the edge image into several sub-edge images and select an edge image with a stain boundary from them.

[0096] Optionally, as Figure 6 shown, the selection module 13 specifically includes a division unit 131 for dividing the edge image into several sub-edge images; a determination unit 132 for traversing all the sub-edge images to determine that the number of edge points in the sub-edge image is greater than a first threshold; a statistics unit 133 for statistically calculating the angle of the connection line between any two edge points and determining whether the proportion of the number of times the connection line with an angle of a second threshold appears in all the connection lines is greater than a third threshold; a discard unit 134 for discarding the sub-edge image; and a first storage unit 135 for storing the sub-edge image.

[0097] A merging module 14, configured to merge any two of the stored sub-edge images that meet the overlapping condition.

[0098] Optionally, as Figure 7As shown, the merging module 14 specifically includes a judgment unit 141, which is used to calculate the overlap rate between any two of the stored sub-edge images and judge whether the coincidence rate is greater than a fourth threshold; a merging unit 142, which is used to merge the two sub-edge images, take the coordinate mean of all edge points in the merged sub-edge image as the center of gravity, and store it; a second storage unit 143, which is used to store it as a single sub-edge image.

[0099] A screening module 15, which is used to further screen the stored merged sub-edge images and single sub-edge images according to the next frame of image to obtain a result image.

[0100] An obtaining module 16, which is used to obtain the stain area in the image according to the result image.

[0101] Please refer to Figure 8 , an embodiment of the present invention provides a cleaning robot 30, including: at least one processor 31, a memory 32 communicatively connected to the at least one processor 31, and a communication module 33 for establishing a communication connection.

[0102] Wherein, the memory 32 stores instructions executable by the at least one processor 31, and the instructions are executed by the at least one processor 31 so that the at least one processor can execute the methods described in any one of the above method embodiments. For example, execute the method steps S110, S120, S130, S140, S150, etc. described above to implement Figure 5 the functions of modules 11, 12, 13, 14, and 15 in

[0103] The processor 31, the memory 32, and the communication module 33 establish a communication connection between any two of them in a bus manner.

[0104] The processor 31 can be of any type, a control chip with one or more processing cores. It can perform single-threaded or multi-threaded operations, and is used to parse instructions to perform operations such as obtaining data, performing logical operation functions, and issuing operation processing results.

[0105] The memory 32, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as program instructions / modules corresponding to the surface stain detection method in the embodiment of the present invention. The processor 31 realizes various functional applications and data processing of the surface stain detection device 10 by running the non-transitory software programs, instructions, and modules stored in the memory 32, that is, realizes the surface stain detection method in any one of the above method embodiments.

[0106] The memory 32 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the surface stain detection device 10, etc. In addition, the memory 32 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory 32 may optionally include a memory remotely set relative to the processor 31, and these remote memories can be connected to the cleaning robot 30 through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0107] Other components of the cleaning robot may refer to cleaning robots in the prior art in this field.

[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; under the idea of the present invention, the technical features in the above embodiments or different embodiments can also be combined, and the steps can be implemented in any order, and there are many other variations in different aspects of the present invention as described above. For the sake of brevity, they are not provided in detail; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for detecting surface stains, characterized in that, Comprising: Obtain an image, and preprocess the image to obtain a grayscale image; Perform canny edge detection calculation on the grayscale image to obtain an edge image; Divide the edge image into a plurality of sub-edge images, and select an edge image with a stain boundary therefrom, wherein, Traverse all the sub-edge images, determine that the number of edge points in the sub-edge image is greater than a first threshold; count the angles of the lines connecting any two of the edge points, and determine whether the proportion of the number of occurrences of the lines with an angle of a second threshold in all the line occurrences is greater than a third threshold; the second threshold is the angle between the current camera and the floor tile dividing line; if so, it is considered that the sub-edge image contains a long straight line with an angle of the second threshold, and the long straight line is the floor tile dividing line, and discard the sub-edge image; if not, store the sub-edge image; Merge any two of the stored sub-edge images that meet the overlap condition; Further screen the stored merged sub-edge images and single sub-edge images according to the next frame of image to obtain a result image; Create a black image of the same size as the edge image as a mask image, convert the result image into a circular image, obtain the center of the circular image according to the contour of the circular image, and obtain the stain area in the image on the mask image according to the line connecting the centers of the two circular images.

2. The method according to claim 1, characterized in that, The step of obtaining an image and preprocessing the image to obtain a grayscale image further includes: Obtain an image; Reduce the size of the image; Convert the reduced image into a grayscale image.

3. The method according to claim 1, wherein The step of merging any two of the stored sub-edge images that meet the overlap condition further includes: Calculate the overlap rate of any two of the stored sub-edge images, and determine whether the overlap rate is greater than a fourth threshold; If so, merge the two sub-edge images, and take the coordinate mean of all the edge points in the merged sub-edge image as the center of gravity and store it; If not, store it as a single sub-edge image.

4. A cleaning robot, characterized in that, Comprising: At least one processor; A memory communicatively connected to the at least one processor; Wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1 to 3.

Images