Cleaning equipment control method and device, storage medium, controller and equipment
By acquiring images through a monocular camera and performing segmentation and edge processing, a target map for the cleaning equipment is constructed, solving the problem that existing LiDAR cannot accurately identify ground bumps and obstacles, and enabling the cleaning equipment to clean efficiently in complex environments.
Patent Information
- Application Number
- CN202410322675.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-20
- Publication Date
- 2024-11-12
AI Technical Summary
Existing cleaning equipment relies on lidar to obtain environmental information, which cannot accurately distinguish between uneven ground and low obstacles, nor can it distinguish between the type and location of stains, thus limiting the complexity of cleaning strategies.
Images of the area to be cleaned are acquired using a monocular camera. A target map, including obstacle information, is constructed through image segmentation and edge processing. Image segmentation is performed using a pre-trained segmentation model, and the control strategy for the cleaning equipment is constructed by combining the edge processing results.
It improves the efficiency and accuracy of cleaning equipment in complex environments, enabling it to more accurately identify low obstacles and adopt optimized cleaning solutions.
Smart Images

Figure CN118924195A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of cleaning equipment, and in particular to a control method, device and storage medium, controller and equipment for cleaning equipment. Background Art
[0002] Currently, cleaning equipment on the market, such as sweepers, mainly rely on laser instruments such as lidar to collect information about the surrounding environment, and determine the location of obstacles based on the surrounding environment information, and then perform cleaning control based on the location of obstacles. However, the information obtained by laser instruments is relatively limited. For example, based on this information, it is impossible to accurately distinguish between uneven surfaces and low obstacles, nor can it distinguish between the type and location of stains, etc., resulting in the inability of cleaning equipment to apply more complex cleaning strategies. Summary of the invention
[0003] The present invention aims to solve one of the technical problems in the related art at least to a certain extent. To this end, the purpose of the present invention is to provide a control method, device and storage medium, controller, and device for cleaning equipment to improve the working efficiency and accuracy of the cleaning equipment in complex environments.
[0004] To achieve the above-mentioned objectives, the first aspect of the embodiment of the present invention proposes a control method for a cleaning device, comprising: acquiring an image of an area to be cleaned; performing image segmentation and edge processing on the image of the area to be cleaned; constructing a target map of the area to be cleaned based on the segmentation results and the edge processing results, wherein the target map includes obstacle information; and controlling the cleaning device based on the target map.
[0005] In addition, the control method of the cleaning device according to the embodiment of the present invention may also have the following additional technical features:
[0006] According to one embodiment of the present invention, a pre-trained segmentation model is used to perform image segmentation on the image of the area to be cleaned, and the training process of the segmentation model includes: acquiring a plurality of work scene images; dividing the plurality of work scene images into a training set and a test set, and respectively annotating the targets to be inspected in the work scene images in the training set and the test set, wherein the targets to be inspected include the background, the ground, and traversable obstacles on the ground; constructing a segmentation model, and pre-training the segmentation model using the training set and its corresponding annotation information, and testing the pre-trained segmentation model using the test set and its corresponding annotation information to obtain a final trained segmentation model.
[0007] According to one embodiment of the present invention, constructing a target map of the area to be cleaned based on the segmentation results and the edge processing results includes: performing corrosion processing on the ground area in the segmentation results to obtain a first image; obtaining an edge line where the background and the ground are in contact in the image of the area to be cleaned based on the edge processing results and the first image; and constructing the target map based on the edge line and traversable obstacles on the ground in the segmentation results.
[0008] According to one embodiment of the present invention, the image of the area to be cleaned is acquired by using an image acquisition element installed on the cleaning device, and the edge line where the background and the ground are in contact in the image of the area to be cleaned is obtained based on the edge processing result and the first image, including: removing the edge in the ground area in the edge processing result based on the first image; and determining the edge line in the edge processing result after removing the edge in the ground area according to the line of sight of the image acquisition element from near to far.
[0009] According to one embodiment of the present invention, constructing the target map based on the edge line and the traversable obstacles on the ground in the segmentation result includes: determining a traversable area based on the edge line, and obtaining the category and position of the traversable obstacles in the traversable area based on the traversable area and the traversable obstacles on the ground in the segmentation result; and constructing the target map based on the category and position of the traversable obstacles in the traversable area.
[0010] According to one embodiment of the present invention, controlling the cleaning device according to the target map includes: updating the global map according to the target map; determining the target cleaning area and the target cleaning strategy according to the updated global map; and controlling the cleaning device to clean the target cleaning area according to the target cleaning strategy.
[0011] To achieve the above-mentioned purpose, the second aspect of the embodiment of the present invention proposes a control device for cleaning equipment, including: an acquisition module for acquiring an image of the area to be cleaned; a processing module for performing image segmentation and edge processing on the image of the area to be cleaned; a construction module for constructing a target map of the area to be cleaned according to the edge processing results and the segmentation results; and a control module for controlling the cleaning equipment according to the target map.
[0012] To achieve the above objectives, a third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the control method of the cleaning device described in the first aspect of the present invention is implemented.
[0013] To achieve the above objectives, the fourth aspect of the present invention proposes a controller, including a memory, a processor and a computer program stored in the memory. When the computer program is executed by the processor, the control method of the cleaning equipment described in the first aspect of the present invention is implemented.
[0014] To achieve the above-mentioned purpose, a fifth aspect of the present invention provides a cleaning device, including the above-mentioned controller.
[0015] According to the control method, device, storage medium, controller, and device of the cleaning equipment of the embodiment of the present invention, firstly, an image of the area to be cleaned is acquired; then, the image of the area to be cleaned is segmented and edge processed; then, a target map containing obstacle information of the area to be cleaned is constructed based on the segmentation result and the edge processing result; finally, the cleaning equipment is controlled based on the target map. Thus, the target map is constructed by fusing the edge processing result with the image segmentation result to control the cleaning equipment, thereby improving the working efficiency and accuracy of the cleaning equipment in complex environments.
[0016] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 is a flow chart of a method for controlling a cleaning device according to an embodiment of the present invention;
[0018] Figure 2 is a training flow chart of a segmentation model according to an embodiment of the present invention;
[0019] Figure 3 is a flowchart of constructing a target map according to a specific embodiment of the present invention;
[0020] FIG4( a ) is an image of an area to be cleaned according to an example of the present invention;
[0021] FIG4( b ) is a segmentation result diagram of the image in FIG4( a );
[0022] FIG4( c ) is a schematic diagram of edge lines obtained based on FIG4( a );
[0023] Figure 4(d) is the target map constructed based on Figure 4(a);
[0024] Figure 5 is a schematic structural diagram of a control device for a cleaning device according to an embodiment of the present invention;
[0025] Figure 6 is a structural block diagram of a controller according to an embodiment of the present invention;
[0026] Figure 7It is a structural block diagram of a cleaning device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0027] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.
[0028] The following describes the control method, device and storage medium, controller and device of the cleaning device according to the embodiments of the present invention with reference to the accompanying drawings.
[0029] Figure 1 4 is a flow chart of a method for controlling a cleaning device according to an embodiment of the present invention.
[0030] like Figure 1 As shown, the control method of the cleaning equipment includes:
[0031] S11, acquiring an image of the area to be cleaned.
[0032] The image of the area to be cleaned may be acquired by using an image acquisition element installed on the cleaning device (such as a monocular camera installed in front of the cleaning device).
[0033] Specifically, taking the acquisition of images by a monocular camera as an example, the image of the area to be cleaned can be collected by the monocular camera, and the collected image of the area to be cleaned can be stored in the memory equipped with the cleaning equipment for easy retrieval, wherein the image of the area to be cleaned can be an RGB image. The monocular camera is installed in front of the cleaning equipment and can effectively observe the ground area within a certain range. Compared with the scene information obtained by laser instruments such as lidar, the monocular camera can obtain richer scene information, which is convenient for more accurate judgment of low obstacles in the future, and allows the cleaning equipment to adopt a more optimized cleaning plan based on the obstacle category information. The monocular camera has a simple structure and low cost, and the obtained image is easy to calibrate and identify.
[0034] S12, performing image segmentation and edge processing on the image of the area to be cleaned.
[0035] Specifically, the canny edge processing algorithm, the sobel edge processing algorithm, etc. may be used to perform edge processing on the image of the area to be cleaned.
[0036] Among them, the Canny algorithm is a classic algorithm for edge processing. The processing process includes: first, perform a Gaussian blur on the image to suppress the noise belonging to the high-frequency signal; then use the Sobel operator to calculate the gradient size and direction; then perform non-maximum pixel gradient suppression on the image, the purpose is to eliminate the stray response caused by edge processing. The basic method is to compare the gradient intensity of the current pixel with the gradient intensity of the adjacent pixels along the positive and negative gradient directions. If it is an extreme value, the edge point of the pixel is retained. If not, it is suppressed and not regarded as an edge point; then the image is thresholded and a high threshold and a low threshold are defined. Pixels with gradient intensity lower than the low threshold are suppressed and not regarded as edge points. Pixels with gradient intensity higher than the high threshold are defined as strong edges and retained as edge points. Pixels between the high and low thresholds are defined as weak edges and are left for further processing; finally, the image is isolated and weak edge suppressed. For each weak edge in the result of the previous step, if one of the eight adjacent pixels is a strong edge, the weak edge is considered to be a strong edge, otherwise it is considered to be an isolated point and discarded.
[0037] In some embodiments of the present invention, a pre-trained segmentation model is used to perform image segmentation on the image of the area to be cleaned, and the training process of the segmentation model includes: acquiring multiple work scene images; dividing the multiple work scene images into a training set and a test set, and marking the targets to be inspected in the work scene images in the training set and the test set, respectively, wherein the targets to be inspected include the background, the ground, and traversable obstacles on the ground (such as carpets, stains, etc.); constructing a segmentation model, and pre-training the segmentation model using the training set and its corresponding annotation information, and testing the pre-trained segmentation model using the test set and its corresponding annotation information to obtain a final trained segmentation model.
[0038] Specifically, considering accuracy and lightness, the DeepLab v3+ segmentation model can be used for image segmentation. The segmentation model training process is as follows: Figure 2 As shown. First, image acquisition of work scenes (such as home scenes) is performed, and the acquired images are screened to ensure the quantitative balance between categories and the correct division of training sets and test sets to a certain extent, and then the screened images are labeled by category (i.e., the objects to be inspected are labeled). Among them, in order to ensure the diversity of the objects to be inspected and work scenes, the acquired images can cover a variety of house types, multiple target types, multiple ambient lighting, multiple shooting distances and multiple shooting angles. In addition, in order to improve the detection efficiency, the screened images can also be pre-processed, such as image enhancement processing, cropping processing, denoising processing, etc.
[0039] Then, select or build an image segmentation model, such as the DeepLab v3+ segmentation model mentioned above. After that, build a model training environment, and use the training set to train the image segmentation model, and then use the test set to evaluate and optimize the image segmentation model. Repeat the above process to complete the image segmentation model training and obtain the segmentation model required for the cleaning equipment, that is, the final trained segmentation model.
[0040] Among them, DeepLab v3+ is a semantic segmentation algorithm that can assign a category to each pixel in an image, but objects in the same category will not be distinguished. Compared with traditional semantic segmentation algorithms, the biggest feature of DeepLab v3+ is the introduction of dilated convolution, which extracts more effective image features without losing information, so that each convolution output contains a wider range of information.
[0041] It should be noted that the above-mentioned image segmentation and edge processing can be implemented through one integrated processing module or through two separate processing modules; they can be performed simultaneously or sequentially.
[0042] S13, constructing a target map of the area to be cleaned according to the segmentation results and the edge processing results.
[0043] Among them, the target map may include obstacle information.
[0044] In some embodiments of the present invention, a target map of the area to be cleaned is constructed based on the segmentation results and the edge processing results, including: performing corrosion processing on the ground area in the segmentation results to obtain a first image; obtaining an edge line where the background and the ground are in contact in the image of the area to be cleaned based on the edge processing results and the first image; and constructing a target map based on the edge line and traversable obstacles on the ground in the segmentation results.
[0045] Among them, erosion processing is a basic morphological operation, which is aimed at the highlight part in the image, that is, the highlight part in the original image is eroded to obtain an area smaller than the original image. The processing process includes: covering each pixel of the original image with the center of a structural element, and taking the minimum value of the covered part of the pixels in the original image to replace the original image pixel value covered by the center of the structural element.
[0046] Specifically, the segmentation model can preliminarily segment the field of view area in front of the cleaning equipment, but due to labeling errors and inference errors, it is still impossible to obtain a relatively accurate edge of the area. To this end, the present invention performs corrosion processing on the ground area in the image segmentation result to obtain a first image to expose more edge areas and avoid the ground segmentation result covering the edge. Then, the edge processing result of removing the edge in the ground area can be obtained in combination with the edge processing result, and then a more accurate contact edge between the insurmountable obstacles in the background and the ground can be obtained to obtain the corresponding edge line. Afterwards, based on the edge lines and the surmountable obstacles on the ground in the segmentation results, a target map can be constructed, which can show the cleanable area of the cleaning equipment, the category of obstacles that can be surmounted in the cleanable area, etc.
[0047] In some embodiments of the present invention, an edge line where the background and the ground meet in the image of the area to be cleaned is obtained based on the edge processing result and the first image, including: removing the edge in the ground area in the edge processing result based on the first image; and determining the edge line in the edge processing result after removing the edge in the ground area according to the line of sight of the image acquisition element from near to far.
[0048] Specifically, the edge processing result can be a binary edge map, and the edge in the ground area in the binary edge map can be removed according to the first image. After that, the binary edge map after removing the edge in the ground area can be queried from near to far along the line of sight of the image acquisition element arranged at the front end of the cleaning device until the edge pixel position is found. The process includes: in each column of the binary edge map after removing the edge in the ground area, the search starts from the point closest to the cleaning device to the point far away, and the first non-zero pixel found is the edge pixel. The line connecting the edge pixels found is the above-mentioned edge line.
[0049] In some embodiments of the present invention, a target map is constructed based on edge lines and traversable obstacles on the ground in segmentation results, including: determining a traversable area based on the edge lines, and obtaining the category and position of the traversable obstacles in the traversable area based on the traversable area and the traversable obstacles on the ground in the segmentation results; and constructing a target map based on the category and position of the traversable obstacles in the traversable area.
[0050] Specifically, the area where the edge line in the image of the area to be cleaned is close to the image acquisition element is a passable area, and then the category and position of the passable obstacles in the passable area are obtained according to the passable area and the traversable obstacles on the ground in the segmentation result (such as carpets, stains, etc.). After that, the coordinates of the traversable obstacles in the passable area are converted from the image coordinate system to the world coordinate system to construct a target map. When performing coordinate conversion, the first conversion matrix between the image coordinate system and the camera coordinate system can be used to convert the coordinates of the traversable obstacles from the image coordinate system to the camera coordinate system, and then the second conversion matrix between the camera coordinate system and the world coordinate system can be used to convert the coordinates of the traversable obstacles from the camera coordinate system to the world coordinate system. Among them, the first conversion matrix can be obtained according to the intrinsic parameters, extrinsic parameters, center, distortion, etc. of the image acquisition element, and the second conversion matrix can be obtained according to the camera posture calculated by the laser radar.
[0051] S14, controlling the cleaning equipment according to the target map.
[0052] In some embodiments of the present invention, controlling the cleaning equipment according to the target map includes: updating the global map according to the target map; determining the target cleaning area and the target cleaning strategy according to the updated global map; and controlling the cleaning equipment to clean the target cleaning area according to the target cleaning strategy.
[0053] Specifically, the first global map can be custom-built or fused according to multiple target maps corresponding to the entire work scene. The cleaning equipment can navigate according to the global map, such as from the bedroom to the kitchen. At the same time, the above steps S11-S13 can be used to build a target map during navigation, and the global map can be updated according to the target map. Afterwards, the target cleaning area (such as the currently uncleaned and passable area) and the target cleaning strategy (such as avoiding carpets and stains during navigation; lifting the mop on the carpet to increase suction; focusing on cleaning stains; avoiding obstacles that cannot be identified by sensors such as lidar and line lasers, etc.) can be determined according to the updated global map, and the cleaning equipment is controlled to clean the target cleaning area according to the target cleaning strategy. In this way, the working efficiency and accuracy of the cleaning equipment can be improved.
[0054] For ease of understanding, the following Figure 3 The specific embodiment process shown, and Figure 4(a)-Figure 4(d) The specific scenario example shown describes the control method of the cleaning device according to the embodiment of the present invention:
[0055] like Figure 3As shown, firstly, image acquisition is performed, and the image of the area to be cleaned is shown in FIG4(a). Then, on the one hand, the image of the area to be cleaned is segmented by the image segmentation model, and the segmentation result is shown in FIG4(b), where area B represents the ground, area A represents the carpet, and the unlabeled part represents the background (including obstacles that cannot be crossed). After that, the ground area in FIG4(b) is eroded to expose more edge areas and avoid the ground segmentation result covering the edge. On the other hand, the Canny edge processing algorithm is used to process the edge of the image of the area to be cleaned, and the edge processing result is obtained.
[0056] Afterwards, the edge of the ground area in the edge processing result is removed in combination with the corrosion processing result, and the edge pixel positions of the remaining edges in the edge processing result are found from near to far along the line of sight of the image acquisition element to obtain the edge line of the removed ground area, as shown by the thick white line in Figure 4(c).
[0057] Next, determine the pixel coordinates of the edge where the obstacle and the ground meet, and determine the obstacle category. The ground area before reaching the edge line is regarded as a passable area (i.e., the ground without insurmountable obstacles). The segmentation result of the segmentation model already contains the category of surmountable obstacles and the coordinates in the image coordinate system. By querying the calibration file (including the conversion matrix), the coordinates of the surmountable obstacles in the image coordinate system are converted to the world coordinate system, as shown in Figure 4(d), which shows the carpet position (i.e., the black block part in Figure 4(d)) and the position of the insurmountable obstacle (i.e., the line part in Figure 4(d)). Thus, a BEV (Bird's Eye View), i.e., the target map, is constructed. After that, the target cleaning area and target cleaning strategy can be further determined in combination with the global map, and the cleaning equipment can be controlled to perform cleaning work according to the target cleaning area and target cleaning strategy.
[0058] Among them, the bird's-eye view is a perspective of viewing an object or scene from above. In the field of obstacle avoidance, data obtained by sensors (such as image acquisition components) are usually converted into BEV representations to better perform tasks such as object detection and path planning. The advantages of BEV include: the ability to simplify complex three-dimensional environments into two-dimensional images, which can save a lot of resources in computing and storage; it provides a unique visual effect that makes the objects and spatial relationships in the scene more clearly visible; processing tasks such as object detection, tracking and classification in BEV is much simpler than processing directly in the original 3D data; image acquisition component detection will result in a situation where objects are larger when they are near and smaller when they are far away, while BEV has almost no scale differences in the same type of targets, making it easier to learn feature scale consistency.
[0059] Corresponding to the target detection method of the above embodiment, the present invention also proposes a control device for cleaning equipment.
[0060] Figure 5 Schematic diagram of the structure of the control device of the cleaning equipment according to the embodiment of the present invention.
[0061] like Figure 5 As shown, the control device 500 of the cleaning equipment includes: an acquisition module 501, a detection module 502, a segmentation module 503, a construction module 504 and a control module 505.
[0062] Among them, the acquisition module 501 is used to acquire the image of the area to be cleaned; the processing module 502 is used to perform image segmentation and edge processing on the image of the area to be cleaned; the construction module 504 is used to construct a target map of the area to be cleaned according to the edge processing results and the segmentation results, and the target map includes obstacle information; the control module 505 is used to control the cleaning equipment according to the target map.
[0063] In some embodiments of the present invention, a pre-trained segmentation model is used to perform image segmentation on the image of the area to be cleaned, and the training process of the segmentation model includes: acquiring a plurality of work scene images; dividing the plurality of work scene images into a training set and a test set, and respectively marking the targets to be inspected in the work scene images in the training set and the test set, wherein the targets to be inspected include the background, the ground, and traversable obstacles on the ground; constructing a segmentation model, and pre-training the segmentation model using the training set and its corresponding annotation information, and testing the pre-trained segmentation model using the test set and its corresponding annotation information to obtain a final trained segmentation model.
[0064] In some embodiments of the present invention, a target map of the area to be cleaned is constructed based on edge processing results and segmentation results, including: performing corrosion processing on the ground area in the segmentation results to obtain a first image; obtaining an edge line where the background and the ground are in contact in the image of the area to be cleaned based on the edge processing results and the first image; and constructing a target map based on the edge line and traversable obstacles on the ground in the segmentation results.
[0065] In some embodiments of the present invention, the image of the area to be cleaned is acquired by using an image acquisition element installed on the cleaning equipment, and the edge line where the background and the ground meet in the image of the area to be cleaned is obtained based on the edge processing result and the first image, including: based on the edge of the background and the ground in the edge processing result, the edge line is determined in the first image from near to far according to the line of sight of the image acquisition element.
[0066] In some embodiments of the present invention, a target map is constructed based on edge lines and traversable obstacles on the ground in segmentation results, including: determining a traversable area based on the edge lines, and obtaining the category and position of the traversable obstacles in the traversable area based on the traversable area and the traversable obstacles on the ground in the segmentation results; and constructing a target map based on the category and position of the traversable obstacles in the traversable area.
[0067] In some embodiments of the present invention, controlling the cleaning equipment according to the target map includes: updating the global map according to the target map; determining the target cleaning area and the target cleaning strategy according to the updated global map; and controlling the cleaning equipment to clean the target cleaning area according to the target cleaning strategy.
[0068] It should be noted that, for other specific implementations of the control device 500 for a cleaning device according to an embodiment of the present invention, reference may be made to the specific implementations of the control method for a cleaning device according to the above-mentioned embodiment of the present invention.
[0069] Based on the control method of the cleaning device in the above embodiment, the present invention also proposes a computer-readable storage medium.
[0070] In this embodiment, a computer program is stored thereon, and when the computer program is executed by the processor, the above-mentioned control method of the cleaning device is implemented.
[0071] Based on the control method of the cleaning device in the above embodiment, the present invention further proposes a controller.
[0072] Figure 6 4 is a structural block diagram of a controller according to an embodiment of the present invention.
[0073] As shown in the figure, the controller 600 includes a processor 601, a memory 603 and a computer program stored in the memory. When the computer program is executed by the processor, the control method of the cleaning device described above is implemented.
[0074] Processor 601 may be a CPU (Central Processing Unit), a general purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of the present invention. Processor 601 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.
[0075] The bus 602 may include a path to transmit information between the above components. The bus 602 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 602 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0076] The memory 603 is used to store a computer program corresponding to the control method of the cleaning device of the above embodiment of the present invention, and the computer program is controlled and executed by the processor 601. The processor 601 is used to execute the computer program stored in the memory 603 to implement the content shown in the above method embodiment.
[0077] The controller 600 includes but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), etc., and fixed terminals such as desktop computers, etc. Figure 6 The controller 600 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.
[0078] Figure 7 It is a structural block diagram of a cleaning device according to an embodiment of the present invention.
[0079] like Figure 7 As shown, the cleaning device 700 includes the controller 600 of the above embodiment.
[0080] In summary, the control method, device, storage medium, controller, and device of the cleaning equipment of the embodiment of the present invention perform image segmentation and edge processing on the image of the area to be cleaned acquired by the monocular image acquisition element, and detect the open areas on the ground and key obstacles (such as stains, carpets, etc.) in the working scene in combination with the segmentation results and edge processing results, and construct a target map, thereby realizing the control of the cleaning equipment according to the target map. As a result, the working efficiency and accuracy of the automatic cleaning equipment in complex environments are improved.
[0081] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.
[0082] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0083] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.
[0084] In the description of the present invention, it is to be understood that the terms “center”, “longitudinal”, “lateral”, “length”, “width”, “thickness”, “up”, “down”, “front”, “back”, “left”, “right”, “vertical”, “horizontal”, “top”, “bottom”, “inside”, “outside”, “clockwise”, “counterclockwise”, “axial”, “radial”, “circumferential”, etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.
[0085] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0086] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0087] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being "above", "above" or "above" a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" or "below" a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.
[0088] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.
Claims
1. A method for controlling a cleaning device, characterized in that: include: Acquire an image of the area to be cleaned; Performing image segmentation and edge processing on the image of the area to be cleaned; Constructing a target map of the area to be cleaned according to the segmentation result and the edge processing result, wherein the target map includes obstacle information; The cleaning device is controlled according to the target map.
2. The control method of the cleaning equipment according to claim 1, characterized in that: The image of the area to be cleaned is segmented using a pre-trained segmentation model, and the training process of the segmentation model includes: A plurality of working scene images are acquired; Dividing the plurality of work scene images into a training set and a test set, and respectively marking the targets to be inspected in the work scene images in the training set and the test set, wherein the targets to be inspected include a background, a ground, and traversable obstacles on the ground; A segmentation model is constructed, and the segmentation model is pre-trained using the training set and its corresponding annotation information, and the pre-trained segmentation model is tested using the test set and its corresponding annotation information to obtain a final trained segmentation model.
3. The control method of the cleaning equipment according to claim 2, characterized in that: The step of constructing a target map of the area to be cleaned according to the segmentation results and the edge processing results includes: Performing corrosion processing on the ground area in the segmentation result to obtain a first image; Obtaining an edge line where the background and the ground are in contact in the image of the area to be cleaned according to the edge processing result and the first image; The target map is constructed according to the edge lines and the traversable obstacles on the ground in the segmentation results.
4. The control method of the cleaning equipment according to claim 3, characterized in that: The image of the area to be cleaned is acquired by using an image acquisition element installed on the cleaning device, and the edge line where the background and the ground are in contact in the image of the area to be cleaned is obtained according to the edge processing result and the first image, including: According to the first image, removing the edge in the ground area in the edge processing result; The edge line is determined from the edge processing result after removing the edge in the ground area according to the line of sight of the image acquisition element from near to far.
5. The control method of the cleaning equipment according to claim 3, characterized in that: The step of constructing the target map according to the edge line and the traversable obstacles on the ground in the segmentation result includes: Determine a traversable area according to the edge line, and obtain the type and position of the traversable obstacles in the traversable area according to the traversable area and the traversable obstacles on the ground in the segmentation result; The target map is constructed according to the categories and positions of the traversable obstacles in the traversable area.
6. The control method of the cleaning equipment according to claim 1, characterized in that: The controlling the cleaning device according to the target map includes: Updating the global map according to the target map; Determine a target cleaning area and a target cleaning strategy based on the updated global map; The cleaning device is controlled to clean the target cleaning area according to the target cleaning strategy.
7. A control device for a cleaning device, characterized in that: include: An acquisition module, used for acquiring an image of the area to be cleaned; A processing module, used for performing image segmentation and edge processing on the image of the area to be cleaned; A construction module, used to construct a target map of the area to be cleaned according to the edge processing result and the segmentation result, wherein the target map includes obstacle information; A control module is used to control the cleaning device according to the target map.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the control method of the cleaning device according to any one of claims 1 to 6 is implemented.
9. A controller comprising a memory, a processor and a computer program stored in the memory, characterized in that: When the computer program is executed by the processor, the control method of the cleaning device according to any one of claims 1 to 6 is implemented.
10. A cleaning device, characterized in that: Comprising a controller according to claim 9.
Citation Information
Cited By
Intelligent recognition system for recognizing dirty area
CN120070874A
An intelligent recognition system for stain area recognition
CN120070874B
