Stain detection method, device, electronic device and sweeping robot system

Abstract

The embodiments of the present application provide a stain detection method, device, electronic device, and robot vacuum system, relating to the field of image processing technology. The method comprises: obtaining an image to be detected containing the ground; grouping each pixel in the image to be detected based on the similarity between the target attributes of the pixels to obtain a segmented image; wherein pixels belonging to different groups in the segmented image are marked with different colors; the target attribute includes the color value of the pixel; based on the difference between the pixel values of each pixel in the segmented image, determining the edge of an object in the segmented image; and when the edge of an object determined meets a preset stain edge feature, determining that a stain exists on the ground corresponding to the object. In this way, the complexity of using the robot vacuum can be reduced, and the storage space occupied by the robot vacuum can be reduced.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a stain detection method, device, electronic device, and sweeping robot system. Background Art

[0002] As the pace of social life continues to accelerate, more and more users use sweeping robots to complete ground cleaning tasks.

[0003] In the related art, a sweeping robot can capture an image of the current floor and compare the image with a stored image of the floor when no stains are present, to determine whether there are currently stains on the floor.

[0004] It can be seen that in the related art, it is necessary to obtain images when there are no stains on the ground in advance and store them in the sweeping robot, which increases the complexity of using the sweeping robot and takes up the storage space of the sweeping robot. Summary of the Invention

[0005] The purpose of the embodiments of the present application is to provide a stain detection method, device, electronic device, and robot vacuum system to reduce the complexity of using the robot vacuum and reduce the storage space occupied by the robot vacuum. The specific technical solution is as follows:

[0006] In a first aspect, in order to achieve the above-mentioned objectives, the present application discloses a stain detection method, which includes:

[0007] Obtain the image to be detected containing the ground;

[0008] Grouping the pixels in the image to be detected based on similarities between target attributes of the pixels to obtain a segmented image; wherein pixels belonging to different groups in the segmented image are marked with different colors; the target attributes include color values of the pixels;

[0009] determining an edge of an object in the segmented image based on a difference between pixel values of pixels in the segmented image;

[0010] When the edge of an object determined to satisfy a preset stain edge feature, it is determined that stains exist on the ground corresponding to the object.

[0011] Optionally, determining the edge of the object in the segmented image based on the difference between pixel values of each pixel point in the segmented image includes:

[0012] Determining the edge of the object in the segmented image based on target contour pixels and / or target edge pixels in the segmented image;

[0013] The target contour pixel points represent the contour of the image area of each color in the segmented image; and the target edge pixel points are determined by performing edge detection on the segmented image based on a preset edge detection algorithm.

[0014] Optionally, determining the edge of the object in the segmented image based on target contour pixels and / or target edge pixels in the segmented image includes:

[0015] Determine pixel points among the target contour pixels that do not belong to the target edge pixels as pixel points to be filtered;

[0016] An edge formed by other pixel points in the target contour except the pixel points to be filtered is determined as an edge of the object in the segmented image.

[0017] Optionally, the process of determining the target contour pixel points includes:

[0018] Determine a pixel point at the outline of the image region of each color in the segmented image as an initial outline pixel point;

[0019] Based on the initial contour pixel points, target contour pixel points are determined.

[0020] Optionally, determining target contour pixel points based on the initial contour pixel points includes:

[0021] Determining the initial contour pixel point as the target contour pixel point;

[0022] or,

[0023] For each initial contour pixel point, a pixel point in a designated neighborhood of the initial contour pixel point in the segmented image is determined as a corresponding target contour pixel point.

[0024] Optionally, the process of determining the target edge pixel point includes:

[0025] Performing edge detection on the segmented image based on a preset edge detection algorithm to obtain initial edge pixel points;

[0026] Based on the initial edge pixel points, target edge pixel points are determined.

[0027] Optionally, determining target edge pixel points based on the initial edge pixel points includes:

[0028] Determining the initial edge pixel point as the target edge pixel point;

[0029] or,

[0030] For each initial edge pixel point, a pixel point in a designated neighborhood of the initial edge pixel point in the segmented image is determined as a corresponding target edge pixel point.

[0031] Optionally, when the determined edge of an object satisfies a preset stain edge feature, determining that a stain exists on the ground corresponding to the object includes:

[0032] If the edge of an object determined satisfies a preset stain edge feature, and the result of the laser scanning indicates that no object exists on the ground corresponding to the object, it is determined that stains exist on the ground corresponding to the object.

[0033] Optionally, the image to be detected includes multiple image frames collected during the movement of the image acquisition module;

[0034] The step of determining that a stain exists on the ground corresponding to the object when the edge of the object determines that the stain meets a preset stain edge feature includes:

[0035] If the edges of the same object determined based on each image frame all meet the preset stain edge features, it is determined that stains exist on the ground corresponding to the object.

[0036] Optionally, the target attribute further includes the position and / or brightness value of the pixel point.

[0037] In a second aspect, in order to achieve the above-mentioned purpose, the present application discloses a stain detection device, which includes:

[0038] The module for acquiring an image to be detected is used to acquire an image to be detected that includes the ground;

[0039] An image segmentation module, configured to group pixels in the image to be detected based on similarities between target attributes of the pixels to obtain a segmented image; wherein pixels belonging to different groups in the segmented image are marked with different colors; and the target attributes include the color values of the pixels;

[0040] an edge determination module, configured to determine an edge of an object in the segmented image based on a difference between pixel values of pixels in the segmented image;

[0041] The stain detection module is used to determine that there is a stain on the ground corresponding to the object when the edge of an object determined meets the preset stain edge feature.

[0042] Optionally, the edge determination module is specifically configured to determine the edge of the object in the segmented image based on target contour pixels and / or target edge pixels in the segmented image;

[0043] The target contour pixel points represent the contour of the image area of each color in the segmented image; and the target edge pixel points are determined by performing edge detection on the segmented image based on a preset edge detection algorithm.

[0044] Optionally, the edge determination module is specifically configured to determine pixel points in the target contour that do not belong to the target edge pixel points as pixel points to be filtered;

[0045] An edge formed by other pixel points in the target contour except the pixel points to be filtered is determined as an edge of the object in the segmented image.

[0046] Optionally, the device further includes:

[0047] an initial contour pixel point determination module, configured to determine pixel points at the contour of the image region of each color in the segmented image as initial contour pixel points;

[0048] The target contour pixel point determination module is used to determine the target contour pixel points based on the initial contour pixel points.

[0049] Optionally, the target contour pixel point determination module is specifically configured to determine the initial contour pixel point as the target contour pixel point;

[0050] or,

[0051] For each initial contour pixel point, a pixel point in a designated neighborhood of the initial contour pixel point in the segmented image is determined as a corresponding target contour pixel point.

[0052] Optionally, the device further includes:

[0053] An initial edge pixel point determination module is used to perform edge detection on the segmented image based on a preset edge detection algorithm to obtain initial edge pixel points;

[0054] The target edge pixel point determination module is used to determine the target contour pixel points based on the initial edge pixel points.

[0055] Optionally, the target edge pixel point determination module is specifically configured to determine the initial edge pixel point as the target edge pixel point;

[0056] or,

[0057] For each initial edge pixel point, a pixel point in a designated neighborhood of the initial edge pixel point in the segmented image is determined as a corresponding target edge pixel point.

[0058] Optionally, the stain detection module is specifically used to determine that there is a stain on the ground corresponding to the object if the edge of an object determined meets a preset stain edge feature and the result of laser scanning indicates that there is no object on the ground corresponding to the object.

[0059] Optionally, the image to be detected includes multiple image frames collected during the movement of the image acquisition module;

[0060] The stain detection module is specifically configured to determine that stains exist on the ground corresponding to the object if the edges of the same object determined based on each image frame all meet preset stain edge features.

[0061] Optionally, the target attribute further includes the position and / or brightness value of the pixel point.

[0062] In another aspect of the implementation of the present application, in order to achieve the above-mentioned purpose, the embodiment of the present application further discloses a sweeping robot system, including an image acquisition module and a processor;

[0063] The image acquisition module is used to acquire the image to be detected including the ground;

[0064] The processor is used to execute any of the above-mentioned stain detection methods.

[0065] Optionally, a laser sensor is also included to scan the ground to determine whether there is an object in the scanned ground.

[0066] In another aspect of the present application, in order to achieve the above-mentioned purpose, an embodiment of the present application further discloses an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0067] The memory is used to store computer programs;

[0068] The processor is configured to implement any of the above-mentioned stain detection methods when executing the program stored in the memory.

[0069] In another aspect of the present application, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium. When the computer program is executed by a processor, the stain detection method as described above is implemented.

[0070] An embodiment of the present application further provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any of the above-described stain detection methods.

[0071] Beneficial effects of the embodiments of the present application:

[0072] The stain detection method provided in an embodiment of the present application obtains an image to be detected containing a ground surface; groups each pixel point in the image to be detected based on the similarity between the target attributes of the pixel points to obtain a segmented image; wherein, the pixel points belonging to different groups in the segmented image are marked with different colors; the target attributes include the color values of the pixel points; based on the difference between the pixel values of each pixel point in the segmented image, the edge of the object in the segmented image is determined; when the edge of an object determined meets the preset stain edge feature, it is determined that a stain exists on the ground surface corresponding to the object.

[0073] Based on the above processing, the ground and objects on the ground can be distinguished in the segmented image according to the color value of the pixel point. Furthermore, the edges of the objects in the image can be determined based on the segmented image, and combined with the edge features of the stains, it can be determined whether there are stains. There is no need to pre-acquire an image when there are no stains on the ground. Therefore, the complexity of using the sweeping robot can be reduced, and the storage space occupied by the sweeping robot can be reduced.

[0074] Of course, it is not necessary to achieve all the advantages described above at the same time when implementing any product or method of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0076] Figure 1 A flowchart of a stain detection method provided in an embodiment of the present application;

[0077] Figure 2 A flowchart of another stain detection method provided in an embodiment of the present application;

[0078] Figure 3 A flowchart of another stain detection method provided in an embodiment of the present application;

[0079] Figure 4 A schematic diagram of the principle of stain detection provided in an embodiment of the present application;

[0080] Figure 5 A structural diagram of a stain detection device provided in an embodiment of the present application;

[0081] Figure 6 A structural diagram of a sweeping robot system provided in an embodiment of the present application;

[0082] Figure 7A structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0083] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field based on this application are within the scope of protection of this application.

[0084] In the related art, it is necessary to obtain images of the ground when there is no stain in advance and store them in the sweeping robot, which increases the complexity of using the sweeping robot and occupies the storage space of the sweeping robot.

[0085] In order to solve the above problems, an embodiment of the present application provides a stain detection method, which can be applied to an electronic device that can be integrated inside a sweeping robot to detect stains on the ground based on images collected by the sweeping robot (i.e., images to be detected).

[0086] See also Figure 1 , Figure 1 This is a flowchart of a stain detection method provided in an embodiment of the present application. The method may include the following steps:

[0087] S101: Acquire an image to be detected containing the ground.

[0088] S102: Grouping pixels in the image to be detected based on similarities between target attributes of the pixels to obtain a segmented image.

[0089] Pixels belonging to different groups in the segmented image are marked with different colors; the target attributes include the color values of the pixels.

[0090] S103: Determine the edge of the object in the segmented image based on the difference between the pixel values of the pixels in the segmented image.

[0091] S104: When the edge of an object is determined to meet a preset stain edge feature, it is determined that stains exist on the ground corresponding to the object.

[0092] The stain detection method provided in the embodiment of the present application can distinguish the ground from objects on the ground in the segmented image based on the color values of the pixel points. Furthermore, based on the segmented image, the edges of the objects in the image can be determined, and combined with the edge features of the stains, it can be determined whether there are stains. There is no need to pre-acquire an image when there are no stains on the ground. Therefore, it can reduce the complexity of using the sweeping robot and reduce the storage space occupied by the sweeping robot.

[0093] With respect to step S101, in one implementation, the image to be detected can be acquired by an image acquisition module (e.g., a camera in a sweeping robot). For example, the image to be detected is an RGB (Red, Green, and Blue) image. In addition, the ground is usually located at the bottom of the image acquired by the sweeping robot. Therefore, the image acquired by the sweeping robot can be cropped to obtain the portion occupied by the ground as the image to be detected. The image to be detected can be a single image frame or multiple image frames.

[0094] For step S102, the color value of the pixel point may include the values of the RGB three channels. The similarity between the target attributes of the pixels belonging to the same group is relatively high. For example, the similarity between the target attributes of the pixels belonging to the same group is greater than a preset similarity threshold. In a floor cleaning scenario, it can be determined that the pixels corresponding to each object on the floor belong to the same group, and the pixels corresponding to the floor belong to the same group. For example, since the colors of the pixels corresponding to the stains are relatively similar, it can be determined that the pixels corresponding to the stains on the floor belong to the same group, and the pixels corresponding to the floor belong to the same group.

[0095] Pixels are grouped based on the similarity between their target attributes to obtain multiple groups. Pixels belonging to different groups can then be marked with different colors to obtain a segmented image. That is, pixels belonging to the same group in the segmented image are marked with the same color.

[0096] In one implementation, the image to be detected can be segmented based on a flood fill algorithm, a watershed algorithm, or a GrabCuts algorithm in combination with target attributes of the pixels, thereby grouping the pixels.

[0097] In one implementation, each pixel can be clustered based on a clustering algorithm such as K-means or Mean-Shift, combined with the target attributes of the pixel points. Each cluster obtained by clustering is considered as a group.

[0098] In one implementation, the pixels in the image to be detected can also be grouped based on a deep learning network model, which is trained based on the target attributes of the pixels in the sample image and the labels of the pixels, where the labels of the pixels can indicate the group to which the pixel belongs.

[0099] In one embodiment, after the groups are determined based on the above algorithm, the grouping results can be optimized. For example, if the number of pixels belonging to the same group is less than a threshold, the pixels belonging to the group can be determined to be noise points. Noise points can be marked with the color of adjacent pixels.

[0100] In one embodiment, to further improve stain detection accuracy, the target attributes may also include pixel locations and / or brightness values. The pixel locations can be represented by their image coordinates within the image to be detected. In one implementation, the image to be detected can be converted to YUV format, and the Y channel value can be obtained as the pixel brightness value.

[0101] In step S103, because pixels belonging to different groups in the segmented image are marked with different colors, and pixels in different groups correspond to different objects, the contrast between different objects in the segmented image is more obvious. For example, the contrast between an object on the ground and the ground is more obvious. Furthermore, the accuracy of the object edges determined based on the segmented image is higher, which in turn improves the accuracy of stain detection.

[0102] In addition, the brightness changes of pixels in the image are greatly affected by light. In the embodiment of the present application, the edge is determined based on the segmented image instead of the grayscale image of the image to be detected. This can reduce the influence of light to improve the accuracy of the edge of the determined object, and accordingly, can improve the accuracy of stain detection.

[0103] Regarding step S104, the preset stain edge feature may represent the shape of the stain on the ground. For example, the stain edge feature represents that the edge of the object is closed and the shape of the edge is irregular.

[0104] In one embodiment, if the determined edges of each object do not meet the preset stain edge characteristics, it can be determined that no stain exists in the currently scanned ground.

[0105] Based on steps S101-S104 above, a corresponding detection result (referred to as an initial detection result) can be obtained for each image frame. The initial detection result indicates whether any of the object edges determined based on the image frame meet the preset stain edge characteristics. Based on this initial detection result, it is possible to directly determine whether there is a stain on the ground.

[0106] To improve the accuracy of stain detection, the image to be detected may also be multiple image frames. For example, the image to be detected may include multiple image frames captured by the image acquisition module during movement. The multiple image frames may be multiple consecutive image frames captured by the image acquisition module, or may be obtained by sampling multiple consecutive image frames. Accordingly, the above step S104 may include the following steps:

[0107] If the edges of the same object determined based on each image frame all meet the preset stain edge features, it is determined that stains exist on the ground corresponding to the object.

[0108] In this embodiment of the present application, a corresponding initial detection result can be obtained for each image frame based on steps S101-S104. Furthermore, if the initial detection results of multiple image frames indicate that the edges of the same object meet the preset stain edge characteristics, it can be determined that a stain exists on the ground corresponding to the object. Based on this, combining the initial detection results of multiple image frames can improve the accuracy of stain detection.

[0109] In one embodiment, the edges of the objects in the segmented image can be determined based on different methods. Figure 2 ,exist Figure 1 On the basis of, the above step S103 may include the following steps:

[0110] S1031: Determine the edge of the object in the segmented image based on target contour pixels and / or target edge pixels in the segmented image.

[0111] The target contour pixel points represent the contour of the image area of each color in the segmented image; the target edge pixel points are determined by edge detection of the segmented image based on a preset edge detection algorithm.

[0112] The preset edge detection algorithm can determine the edge of the object based on the gradient of the pixel values of the pixels in the segmented image. For example, the preset edge detection algorithm can be a Canny edge detection algorithm, a Sobel edge detection algorithm, a Scharr edge detection algorithm, or a Laplacian edge detection algorithm, but is not limited thereto.

[0113] In an embodiment of the present application, the electronic device can search for the contour of the image area of each color in the segmented image, that is, determine the pixel points corresponding to the contour (that is, the target contour pixel points), and / or perform edge detection on the segmented image based on a preset edge detection algorithm to obtain the target edge pixel points.

[0114] Based on the target contour pixels and / or target edge pixels, the electronic device may determine the edge of the object in the segmented image, that is, determine the edge pixels of each object in the segmented image.

[0115] Based on the above processing, the edges of objects in the segmented image can be determined by combining contour search and edge detection algorithms, which can improve the accuracy of the determined edges and, in turn, improve the accuracy of stain detection.

[0116] In one embodiment, see Figure 3 ,exist Figure 2 On the basis of, the above step S1031 may include the following steps:

[0117] S10311: Determine pixel points among the target contour pixels that do not belong to the target edge pixels as pixel points to be filtered.

[0118] S10312: Determine the edge formed by the pixel points of the target contour other than the pixel points to be filtered as the edge of the object in the segmented image.

[0119] In the embodiment of the present application, the contour pixels determined based on the contour search may include pixels other than the contour of the object, for example, pixels in the four edges of the segmented image.

[0120] The preset edge detection algorithm can determine all edge pixels in the segmented image, and thus the pixels at the object's outline. However, the pixels to be filtered belong to the target outline pixels and do not belong to the target edge pixels. In other words, the pixels to be filtered are pixels outside the object's outline determined based on the outline search. Therefore, when determining the edge of the object in the segmented image, the pixels to be filtered can be removed from the target outline pixels.

[0121] In one embodiment, after determining the contour pixels based on the contour search, the edges formed by the pixels in the contour other than the edge of the segmented image can be determined as the edges of the object in the segmented image. In other words, there is no need to perform edge detection on the segmented image based on a preset edge detection algorithm.

[0122] In one embodiment, edge detection can be performed on the segmented image based on a preset edge detection algorithm to obtain an edge detection result. The edge detection result can represent the pixels at the edge of each object in the segmented image. That is, based on the edge detection result, edge pixels belonging to the same object can be determined. Accordingly, the edge formed by the edge pixels can be directly determined as the edge of the object in the segmented image. This eliminates the need to determine contour pixels based on contour search.

[0123] Furthermore, if multiple objects are adjacent to each other in the image to be detected, the edge detection results obtained based on the preset edge detection algorithm may identify these multiple objects as a single object, meaning that the edges of each object cannot be distinguished. This situation does not occur with contour-based detection. Therefore, combining these two methods can improve the accuracy of edge determination and thus improve the accuracy of stain detection.

[0124] In one embodiment, the process of determining the target contour pixel points may include:

[0125] Step 1: Determine the pixel points at the outline of the image area of each color in the segmented image as the initial outline pixel points.

[0126] Step 2: Determine the target contour pixels based on the initial contour pixels.

[0127] In the embodiment of the present application, the electronic device may determine the pixel points at the outline of the image region of each color in the segmented image based on different methods.

[0128] In one implementation, for an image region of a certain color, the electronic device may scan the pixels in the segmented image column by column to determine the pixels of that color (referred to as candidate pixels). Furthermore, for the candidate pixels in each column, the electronic device may determine the candidate pixel with the largest height coordinate value and the candidate pixel with the smallest height coordinate value (referred to as the target pixel) based on the pixel coordinates (including height coordinate values and width coordinate values) of each candidate pixel. Furthermore, the target pixel in each column is determined as the initial contour pixel at the contour of the image region of that color.

[0129] In another implementation, for an image region of a certain color, the electronic device may scan the pixels in the segmented image row by row and column by column to determine the initial outline pixel points at the outline of the image region of that color. The principle of this method is similar to the column-by-column scanning method.

[0130] In one embodiment, the above step 2 may include the following steps:

[0131] The initial contour pixel points are determined as target contour pixel points; or, for each initial contour pixel point, the pixel points within a specified neighborhood of the initial contour pixel point in the segmented image are determined as corresponding target contour pixel points.

[0132] In one implementation, the electronic device may directly determine the initial contour pixel point as the target contour pixel point.

[0133] Alternatively, for the same object, the initial outline pixels of the image area occupied by the object, determined based on contour search, may not match the initial edge pixels of the object determined based on a preset edge detection algorithm. Therefore, after determining the initial outline pixels, dilation processing can be performed. That is, for each initial outline pixel, the pixels within a specified neighborhood of the initial outline pixel in the segmented image are determined as target outline pixels. In other words, the range of the outline pixels is expanded. This can improve the degree of match between the target outline pixels and the target edge pixels, further enhancing the effectiveness of the determined object edge.

[0134] The designated neighborhood can be set based on experience. The designated neighborhood of a pixel can represent an image region of a specified size centered on the pixel. For example, it represents a square image region of a specified size centered on the pixel.

[0135] In one embodiment, the process of determining the target edge pixel point may include:

[0136] Step 1: Perform edge detection on the segmented image based on a preset edge detection algorithm to obtain initial edge pixels.

[0137] Step 2: Based on the initial edge pixels, determine the target contour pixels.

[0138] In one embodiment, the above step 2 may include the following steps:

[0139] The initial edge pixel points are determined as target edge pixel points; or, for each initial edge pixel point, the pixel points within a specified neighborhood of the initial edge pixel point in the segmented image are determined as the corresponding target edge pixel point.

[0140] In one implementation, the electronic device may directly determine the initial edge pixel point as the target edge pixel point.

[0141] Alternatively, for the same object, the initial outline pixels of the image area occupied by the object, determined based on contour search, may not match the initial edge pixels of the object determined based on a preset edge detection algorithm. Therefore, after determining the initial edge pixels, dilation processing can be performed. That is, for each initial edge pixel, the pixels within a designated neighborhood of the initial edge pixel in the segmented image are determined as target edge pixels. In other words, the range of edge pixels is expanded. This can improve the matching degree between the target outline pixels and the target edge pixels, further enhancing the effectiveness of the determined object edge.

[0142] In one embodiment, when determining the edge of an object in a segmented image based on contour pixel points determined by contour search and edge pixel points determined based on a preset edge detection algorithm, only the initial contour pixel points can be expanded to obtain target contour pixel points, and the initial edge pixel points can be determined as target edge pixel points and processed.

[0143] Alternatively, only the initial edge pixels may be expanded to obtain target edge pixels, and the initial contour pixels may be determined as target contour pixels for further processing.

[0144] Alternatively, the initial edge pixel points and the initial contour pixel points may be expanded at the same time to obtain target edge pixel points and target contour pixel points, and then processed.

[0145] In one embodiment, the above step S104 may include the following steps: if the edge of an object determined meets the preset stain edge feature, and the result based on the laser scanning indicates that there is an object on the ground corresponding to the object, then it is determined that there is a stain on the ground corresponding to the object.

[0146] In an embodiment of the present application, for an image frame, if the edge of an object determined to meet the preset stain edge characteristics, the laser sensor in the sweeping robot can also scan the ground corresponding to the object. Since the height of the stain on the ground is relatively low, when the laser sensor scans the ground corresponding to the object, it determines based on the obtained point cloud image that its height is relatively low, and further determines that there is no object at that location, indicating that the object is a stain. Conversely, if the laser sensor scans the ground corresponding to the object and determines based on the obtained point cloud image that its height is relatively high, it indicates that the object is another object on the ground, not a stain.

[0147] In one embodiment, if the image to be detected includes multiple image frames captured by an image acquisition module in a robot vacuum during movement, the electronic device can determine, for each image frame, whether a stain exists on the ground corresponding to the object, based on the laser scanning results, thereby obtaining an initial detection result for each image frame. Furthermore, if the initial detection results for multiple image frames indicate the presence of a stain on the ground corresponding to the same object, the electronic device can determine the presence of a stain on the ground corresponding to the object.

[0148] See also Figure 4 , Figure 4 A schematic diagram of the principle of stain detection provided in an embodiment of the present application.

[0149] The electronic device can obtain an RGB image, that is, an image to be detected that includes the ground and is collected by the sweeping robot. Then, the RGB image is segmented (that is, each pixel is grouped based on the target attribute of the pixel in the RGB image) to obtain a segmented image. Furthermore, contour search can be performed based on the segmented image to obtain target contour pixels, and edge detection can be performed based on a preset edge detection algorithm to obtain target edge pixels. Furthermore, the electronic device can perform stain determination, that is, determine whether the edge of the object determined based on the image to be detected meets the preset stain edge feature. In addition, the electronic device can also perform stain filtering, that is, if the edge of an object determined meets the preset stain edge feature, it is determined that there is a stain on the ground corresponding to the object, and then combined with the results of the laser scanning, it is determined whether there is an object on the ground corresponding to the object.

[0150] Based on the same inventive concept, the present application also provides a stain detection device, see Figure 5 , Figure 5 A structural diagram of a stain detection device provided in an embodiment of the present application includes:

[0151] The image acquisition module 501 is used to acquire the image to be detected containing the ground;

[0152] An image segmentation module 502 is configured to group pixels in the image to be detected based on similarities between target attributes of the pixels to obtain a segmented image; wherein pixels belonging to different groups in the segmented image are marked with different colors; the target attributes include the color values of the pixels;

[0153] an edge determination module 503 for determining the edge of an object in the segmented image based on the difference between pixel values of pixels in the segmented image;

[0154] The stain detection module 504 is configured to determine that a stain exists on the ground corresponding to an object when the edge of the object determines that the edge of the object satisfies a preset stain edge feature.

[0155] Optionally, the edge determination module 503 is specifically configured to determine the edge of the object in the segmented image based on target contour pixels and / or target edge pixels in the segmented image;

[0156] The target contour pixel points represent the contour of the image area of each color in the segmented image; and the target edge pixel points are determined by performing edge detection on the segmented image based on a preset edge detection algorithm.

[0157] Optionally, the edge determination module 503 is specifically configured to determine pixel points in the target contour that do not belong to the target edge pixel points as pixel points to be filtered;

[0158] An edge formed by other pixel points in the target contour except the pixel points to be filtered is determined as an edge of the object in the segmented image.

[0159] Optionally, the device further includes:

[0160] an initial contour pixel point determination module, configured to determine pixel points at the contour of the image region of each color in the segmented image as initial contour pixel points;

[0161] The target contour pixel point determination module is used to determine the target contour pixel points based on the initial contour pixel points.

[0162] Optionally, the target contour pixel point determination module is specifically configured to determine the initial contour pixel point as the target contour pixel point;

[0163] or,

[0164] For each initial contour pixel point, a pixel point in a designated neighborhood of the initial contour pixel point in the segmented image is determined as a corresponding target contour pixel point.

[0165] Optionally, the device further includes:

[0166] An initial edge pixel point determination module is used to perform edge detection on the segmented image based on a preset edge detection algorithm to obtain initial edge pixel points;

[0167] The target edge pixel point determination module is used to determine the target contour pixel points based on the initial edge pixel points.

[0168] Optionally, the target edge pixel point determination module is specifically configured to determine the initial edge pixel point as the target edge pixel point;

[0169] or,

[0170] For each initial edge pixel point, a pixel point in a designated neighborhood of the initial edge pixel point in the segmented image is determined as a corresponding target edge pixel point.

[0171] Optionally, the stain detection module 504 is specifically configured to determine that there is stain on the ground corresponding to the object if the edge of an object determined meets preset stain edge features and the result of laser scanning indicates that there is no object on the ground corresponding to the object.

[0172] Optionally, the image to be detected includes multiple image frames collected during the movement of the image acquisition module;

[0173] The stain detection module 504 is specifically configured to determine that stains exist on the ground corresponding to the object if the edges of the same object determined based on each image frame all meet preset stain edge features.

[0174] Optionally, the target attribute further includes the position and / or brightness value of the pixel point.

[0175] See also Figure 6 , Figure 6 This is a structural diagram of a sweeping robot system provided in an embodiment of the present application, wherein the sweeping robot system includes an image acquisition module 601 and a processor 602;

[0176] The image acquisition module 601 is used to acquire an image to be detected that includes the ground;

[0177] The processor 602 is configured to execute any of the above-mentioned stain detection methods.

[0178] Optionally, the sweeping robot system further includes a laser sensor for scanning the ground to determine whether there is an object in the scanned ground.

[0179] The present application also provides an electronic device, such as Figure 7 As shown, it includes a processor 701, a communication interface 702, a memory 703 and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704.

[0180] Memory 703, for storing computer programs;

[0181] The processor 701 is configured to execute the program stored in the memory 703, and implement the following steps:

[0182] Obtain the image to be detected containing the ground;

[0183] Grouping the pixels in the image to be detected based on similarities between target attributes of the pixels to obtain a segmented image; wherein pixels belonging to different groups in the segmented image are marked with different colors; the target attributes include color values of the pixels;

[0184] determining an edge of an object in the segmented image based on a difference between pixel values of pixels in the segmented image;

[0185] When the edge of an object determined to satisfy a preset stain edge feature, it is determined that stains exist on the ground corresponding to the object.

[0186] The communication bus mentioned in the electronic device mentioned above may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0187] The communication interface is used for communication between the above electronic device and other devices.

[0188] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0189] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0190] In another embodiment provided in the present application, a computer-readable storage medium is further provided, wherein a computer program is stored in the computer-readable storage medium. When the computer program is executed by a processor, the steps of any of the above-mentioned stain detection methods are implemented.

[0191] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any one of the stain detection methods in the above embodiments.

[0192] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0193] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0194] Each embodiment in this specification is described in a related manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences from other embodiments. In particular, the device, robot vacuum system, electronic device, computer-readable storage medium, and computer program product embodiments are generally similar to the method embodiments, so their descriptions are relatively simple. For related parts, refer to the description of the method embodiments.

[0195] The above description is only a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application are included in the scope of protection of the present application.

Claims

1. A stain detection method, characterized in that: The method comprises: Obtain the image to be detected containing the ground; Grouping the pixels in the image to be detected based on the similarity between the target attributes of the pixels to obtain a segmented image; wherein the pixels belonging to different groups in the segmented image are marked with different colors; the target attributes include the color values of the pixels; determining an edge of an object in the segmented image based on a difference between pixel values of pixels in the segmented image; When the edge of an object determined satisfies a preset stain edge feature, determining that a stain exists on the ground corresponding to the object; The determining the edge of the object in the segmented image based on the difference between the pixel values of the pixels in the segmented image includes: Determine the pixel points among the target contour pixels that do not belong to the target edge pixels as the pixel points to be filtered; Determine an edge formed by other pixel points in the target contour except the pixel points to be filtered as an edge of the object in the segmented image; The target contour pixel points represent the contour of the image area of each color in the segmented image; and the target edge pixel points are determined by performing edge detection on the segmented image based on a preset edge detection algorithm.

2. The method according to claim 1, characterized in that The process of determining the target contour pixel points includes: Determine a pixel point at the outline of the image region of each color in the segmented image as an initial outline pixel point; Based on the initial contour pixel points, target contour pixel points are determined.

3. The method according to claim 2, characterized in that The step of determining target contour pixel points based on the initial contour pixel points includes: Determining the initial contour pixel point as the target contour pixel point; or, For each initial contour pixel point, a pixel point in a designated neighborhood of the initial contour pixel point in the segmented image is determined as a corresponding target contour pixel point.

4. The method according to claim 1, wherein The process of determining the target edge pixel point includes: Performing edge detection on the segmented image based on a preset edge detection algorithm to obtain initial edge pixel points; Based on the initial edge pixel points, target edge pixel points are determined.

5. The method according to claim 4, characterized in that The step of determining target edge pixel points based on the initial edge pixel points includes: Determining the initial edge pixel point as the target edge pixel point; or, For each initial edge pixel point, a pixel point in a designated neighborhood of the initial edge pixel point in the segmented image is determined as a corresponding target edge pixel point.

6. The method according to claim 1, characterized in that The step of determining that a stain exists on the ground corresponding to the object when the edge of the object determines that the stain meets a preset stain edge feature includes: If the edge of an object determined satisfies a preset stain edge feature, and the result of the laser scanning indicates that no object exists on the ground corresponding to the object, it is determined that stains exist on the ground corresponding to the object.

7. The method according to claim 1, characterized in that The image to be detected includes a plurality of image frames collected during the movement of the image acquisition module; The step of determining that a stain exists on the ground corresponding to the object when the edge of the object determines that the stain meets a preset stain edge feature includes: If the edges of the same object determined based on each image frame all meet the preset stain edge features, it is determined that stains exist on the ground corresponding to the object.

8. The method according to claim 1, characterized in that The target attributes also include the position and / or brightness value of the pixel point.

9. A stain detection device, characterized in that: The device comprises: The module for acquiring an image to be detected is used to acquire an image to be detected that includes the ground; An image segmentation module, configured to group pixels in the image to be detected based on similarities between target attributes of the pixels to obtain a segmented image; wherein pixels belonging to different groups in the segmented image are marked with different colors; and the target attributes include the color values of the pixels; an edge determination module, configured to determine an edge of an object in the segmented image based on a difference between pixel values of pixels in the segmented image; A stain detection module is used to determine that a stain exists on the ground corresponding to an object when the edge of the object meets the preset stain edge characteristics; The edge determination module is specifically used to determine pixel points that are not target edge pixels among the target contour pixels as pixels to be filtered; Determine an edge formed by other pixel points in the target contour except the pixel points to be filtered as an edge of the object in the segmented image; The target contour pixel points represent the contour of the image area of each color in the segmented image; and the target edge pixel points are determined by performing edge detection on the segmented image based on a preset edge detection algorithm.

10. The device according to claim 9, characterized in that The device further comprises: an initial contour pixel point determination module, configured to determine pixel points at the contour of the image region of each color in the segmented image as initial contour pixel points; A target contour pixel point determination module is used to determine target contour pixel points based on the initial contour pixel points; The target contour pixel point determination module is specifically configured to determine the initial contour pixel point as the target contour pixel point; or, For each initial contour pixel point, determining a pixel point in a specified neighborhood of the initial contour pixel point in the segmented image as a corresponding target contour pixel point; The device further comprises: An initial edge pixel point determination module is used to perform edge detection on the segmented image based on a preset edge detection algorithm to obtain initial edge pixel points; A target edge pixel point determination module is used to determine target contour pixel points based on the initial edge pixel points; The target edge pixel point determination module is specifically configured to determine the initial edge pixel point as the target edge pixel point; or, For each initial edge pixel point, a pixel point in a designated neighborhood of the initial edge pixel point in the segmented image is determined as a corresponding target edge pixel point.

11. The device according to claim 10, characterized in that The stain detection module is specifically configured to determine that there is a stain on the ground corresponding to the object if the edge of an object determined meets a preset stain edge feature and the result of the laser scanning indicates that there is no object on the ground corresponding to the object; or The image to be detected includes multiple image frames collected by the image acquisition module during the movement; The stain detection module is specifically configured to determine that there is a stain on the ground corresponding to the object if the edges of the same object determined based on each image frame all meet the preset stain edge characteristics; The target attributes also include the position and / or brightness value of the pixel point.

12. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; The memory is used to store computer programs; The processor is configured to implement the method steps described in any one of claims 1 to 8 when executing the program stored in the memory.

13. A sweeping robot system, characterized in that: Including image acquisition module and processor; The image acquisition module is used to acquire the image to be detected including the ground; The processor is configured to execute the method steps described in any one of claims 1-8.

14. The robot vacuum cleaner system according to claim 13, wherein: A laser sensor is also included for scanning the ground to determine whether an object is present in the scanned ground.

15. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps of any one of claims 1 to 8 are implemented.

Images