Intelligent cleaning method and related apparatus

Abstract

This application relates to the fields of image processing and data processing, and provides an intelligent cleaning method and related apparatus. The method is applied to an intelligent cleaning system, which includes an intelligent cleaning device. The method includes: acquiring point cloud data and a first image set of an area to be cleaned using the intelligent cleaning device; determining information about objects to be cleaned in the area to be cleaned based on the first image; determining cleaning mode information based on the information about objects to be cleaned; constructing a three-dimensional image of the area to be cleaned based on the point cloud data to obtain a target three-dimensional image; determining a movement route information for the intelligent cleaning device to move to the area to be cleaned based on the target three-dimensional image; and controlling the intelligent cleaning device to clean the area to be cleaned using the cleaning mode information and the movement route information, thereby improving the efficiency and accuracy of cleaning the area to be cleaned.

Description

Technical Field

[0001] This application relates to the fields of image processing and data processing technology, specifically to an intelligent cleaning method and related apparatus. Background Technology

[0002] When community property management staff clean the community, especially outdoor areas, the current plan typically uses a combination of manual labor and physical cleaning tools, such as brooms and mops. Manual cleaning, especially when dealing with large areas of debris (e.g., large areas of oil stains), requires a significant amount of manpower and is difficult to complete quickly in one go, resulting in low accuracy and efficiency. Summary of the Invention

[0003] This application provides an intelligent cleaning method and related apparatus, which can acquire point cloud data and images of the area to be cleaned using intelligent cleaning equipment, generate corresponding cleaning modes and movement route information, and finally control the intelligent cleaning equipment to perform cleaning, thereby improving the efficiency and accuracy of cleaning the area to be cleaned.

[0004] A first aspect of this application provides an intelligent cleaning method, the method being applied to an intelligent cleaning system, the intelligent cleaning system including intelligent cleaning equipment, the method comprising:

[0005] The point cloud data and first image set of the area to be cleaned are collected using intelligent cleaning equipment;

[0006] The information of the object to be cleaned in the area to be cleaned is determined based on the first image;

[0007] The cleaning mode information is determined based on the information of the object to be cleaned;

[0008] A three-dimensional image of the area to be cleaned is constructed based on the point cloud data to obtain a target three-dimensional image.

[0009] Based on the target 3D image, determine the movement route information of the intelligent cleaning device to the area to be cleaned;

[0010] The intelligent cleaning device is controlled to clean the area to be cleaned based on the cleaning mode information and the movement route information.

[0011] In one possible implementation, determining the information of the object to be cleaned in the area to be cleaned based on the first image includes:

[0012] Feature extraction is performed on the first image to obtain first feature data;

[0013] Based on the first feature data, contour extraction is performed to obtain k contour information;

[0014] Based on the k contour information and the corresponding pixel values, the object to be cleaned is identified, and the object information of the area to be cleaned is obtained.

[0015] In one possible implementation, the step of identifying the object to be cleaned based on the k contour information and corresponding pixel values ​​to obtain the object information of the area to be cleaned includes:

[0016] Perform contour type identification on k contour information to obtain the contour type corresponding to each contour information;

[0017] Based on the contour types corresponding to k contour information, the cleaning object information is determined, and a first reference cleaning object information set is obtained.

[0018] Based on the pixel values ​​corresponding to the k contour information, the cleaning material information corresponding to the k contour information is determined, and a second reference cleaning material information set is obtained.

[0019] The information of the object to be cleaned is determined based on the first set of reference cleaning object information and the second set of reference cleaning object information.

[0020] In one possible implementation, determining the cleaning mode information based on the information of the object to be cleaned includes:

[0021] Extract the outline and type information of the object to be cleaned from the object information;

[0022] The type of cleaning materials is determined based on the aforementioned type information;

[0023] The amount of cleaning materials to be used is determined based on the contour information, and the cleaning instructions are determined based on the contour information.

[0024] The cleaning mode information is determined based on the cleaning instruction, the type of cleaning material, and the dosage information.

[0025] In one possible implementation, determining the cleaning instruction based on the contour information includes:

[0026] Obtain the surrounding area information corresponding to the contour information;

[0027] Based on the surrounding area information, determine n cleaning movement directions;

[0028] Based on the n cleaning movement direction information, the contour indicated by the contour information is split to obtain a set of n sub-contours;

[0029] Based on the n sub-contour sets and the corresponding cleaning direction information, n cleaning paths are determined;

[0030] The cleaning cost is scored for n cleaning paths to obtain n cleaning cost scores.

[0031] The cleaning path corresponding to the minimum cleaning cost score is determined as the target cleaning path;

[0032] A cleaning instruction is generated based on the target cleaning path.

[0033] A second aspect of this application provides an intelligent cleaning device, which is applied to an intelligent cleaning system. The intelligent cleaning system includes intelligent cleaning equipment, and the device includes:

[0034] The acquisition unit is used to acquire point cloud data and a first image set of the area to be cleaned through intelligent cleaning equipment;

[0035] The first determining unit is configured to determine the information of the object to be cleaned in the area to be cleaned based on the first image.

[0036] The second determining unit is used to determine cleaning mode information based on the information of the object to be cleaned;

[0037] A construction unit is used to construct a three-dimensional image of the area to be cleaned based on the point cloud data, thereby obtaining a target three-dimensional image.

[0038] The third determining unit is used to determine the movement route information of the intelligent cleaning device to the area to be cleaned based on the target three-dimensional image;

[0039] The control unit is used to control the intelligent cleaning device to clean the area to be cleaned according to the cleaning mode information and the movement route information.

[0040] In one possible implementation, the first determining unit is specifically used for:

[0041] Feature extraction is performed on the first image to obtain first feature data;

[0042] Based on the first feature data, contour extraction is performed to obtain k contour information;

[0043] Based on the k contour information and the corresponding pixel values, the object to be cleaned is identified, and the object information of the area to be cleaned is obtained.

[0044] In one possible implementation, regarding the step of identifying the object to be cleaned based on the k contour information and corresponding pixel values ​​to obtain the object information of the area to be cleaned, the first determining unit is specifically used for:

[0045] Perform contour type identification on k contour information to obtain the contour type corresponding to each contour information;

[0046] Based on the contour types corresponding to k contour information, the cleaning object information is determined, and a first reference cleaning object information set is obtained.

[0047] Based on the pixel values ​​corresponding to the k contour information, the cleaning material information corresponding to the k contour information is determined, and a second reference cleaning material information set is obtained.

[0048] The information of the object to be cleaned is determined based on the first set of reference cleaning object information and the second set of reference cleaning object information.

[0049] In one possible implementation, the second determining unit is specifically used for:

[0050] Extract the outline and type information of the object to be cleaned from the object information;

[0051] The type of cleaning materials is determined based on the aforementioned type information;

[0052] The amount of cleaning materials to be used is determined based on the contour information, and the cleaning instructions are determined based on the contour information.

[0053] The cleaning mode information is determined based on the cleaning instruction, the type of cleaning material, and the dosage information.

[0054] In one possible implementation, regarding the determination of the cleaning instruction based on the contour information, the second determining unit is specifically configured to:

[0055] Obtain the surrounding area information corresponding to the contour information;

[0056] Based on the surrounding area information, determine n cleaning movement directions;

[0057] Based on the n cleaning movement direction information, the contour indicated by the contour information is split to obtain a set of n sub-contours;

[0058] Based on the n sub-contour sets and the corresponding cleaning direction information, n cleaning paths are determined;

[0059] The cleaning cost is scored for n cleaning paths to obtain n cleaning cost scores.

[0060] The cleaning path corresponding to the minimum cleaning cost score is determined as the target cleaning path;

[0061] A cleaning instruction is generated based on the target cleaning path.

[0062] A third aspect of this application provides a terminal including a processor, an input device, an output device, and a memory, wherein the processor, input device, output device, and memory are interconnected, wherein the memory is used to store a computer program, the computer program including program instructions, and the processor is configured to invoke the program instructions to execute the step instructions as described in the first aspect of this application.

[0063] A fourth aspect of this application provides a computer-readable storage medium storing a computer program for electronic data interchange, wherein the computer program causes a computer to perform some or all of the steps described in the first aspect of this application.

[0064] A fifth aspect of this application provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps described in the first aspect of this application. The computer program product may be a software installation package.

[0065] Implementing the embodiments of this application has the following beneficial effects:

[0066] By collecting point cloud data and a first image set of the area to be cleaned using intelligent cleaning equipment, determining the information of the object to be cleaned in the area based on the first image, determining cleaning mode information based on the information of the object to be cleaned, constructing a three-dimensional image of the area to be cleaned based on the point cloud data to obtain a target three-dimensional image, determining the movement route information of the intelligent cleaning equipment to the area to be cleaned based on the target three-dimensional image, and controlling the intelligent cleaning equipment to clean the area to be cleaned according to the cleaning mode information and the movement route information, the efficiency and accuracy of cleaning the area to be cleaned can be improved. Attached Figure Description

[0067] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0068] Figure 1 This application provides a flowchart illustrating an intelligent cleaning method.

[0069] Figure 2 This is a schematic diagram of the structure of a terminal provided in an embodiment of this application;

[0070] Figure 3 This application provides a schematic diagram of the structure of an intelligent cleaning device. Detailed Implementation

[0071] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0072] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0073] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0074] To better understand the intelligent cleaning method provided in this application, a brief introduction to existing cleaning methods is given below. In existing methods, cleaning (washing) large areas of garbage in outdoor areas is typically done manually, requiring a significant workforce and making it difficult to complete the cleaning quickly and efficiently in one go. While other cleaning methods exist, such as manually operating large cleaning machines, these typically use brushes, requiring manual selection of cleaning paths and methods, resulting in lower efficiency and accuracy.

[0075] To address the aforementioned issues, this application provides an intelligent cleaning method that uses intelligent cleaning equipment to acquire point cloud data and images of the area to be cleaned, generates corresponding cleaning modes and movement route information, and finally controls the intelligent cleaning equipment to perform cleaning, thereby improving the efficiency and accuracy of cleaning the area to be cleaned.

[0076] Please see Figure 1 , Figure 1 This is a flowchart illustrating an intelligent cleaning method provided in an embodiment of this application. Figure 1 As shown, this method is applied to an intelligent cleaning system, which includes intelligent cleaning equipment. The method includes:

[0077] 101. Collect point cloud data and a first image set of the area to be cleaned using intelligent cleaning equipment.

[0078] The intelligent cleaning system can include multiple intelligent cleaning devices and a server. The system controls these devices to perform cleaning operations. The intelligent cleaning devices include cameras, motion modules, radar modules, and cleaning modules. The cameras capture images of the area to be cleaned and upload them to the server. The server then performs subsequent steps in the intelligent cleaning process, such as confirming the cleaning mode and motion route information. The radar module scans the area to be cleaned, obtaining point cloud data, which is then sent to the server for further processing. The intelligent cleaning devices can also receive cleaning mode and motion route information from the server and use their motion modules to travel along the corresponding routes to the area to be cleaned and perform cleaning based on the cleaning mode information.

[0079] Specifically, this can be achieved by using a camera in a smart cleaning device to collect a first set of images of the area to be cleaned, and by using a radar module to collect point cloud data of the area to be cleaned. The first images in the first set of images include images of the area to be cleaned, allowing for the extraction of relevant information about the objects to be cleaned within the area and subsequent analysis and processing.

[0080] 102. Determine the information of the object to be cleaned in the area to be cleaned based on the first image.

[0081] The process involves extracting features from the first image to obtain corresponding feature data, then extracting contours based on the feature data to obtain multiple contour information, and finally identifying the object to be cleaned based on the contour information and pixel values ​​to obtain the object information. Specifically, the feature data can be, for example, grayscale values.

[0082] 103. Determine the cleaning mode information based on the information of the object to be cleaned.

[0083] Specifically, contour and type information can be extracted from the information of the object to be cleaned, and then cleaning instructions can be determined based on the contour and type information. For example, the type of material can be determined based on the type information, and the amount and cleaning instructions can be determined based on the contour information. Finally, cleaning mode information can be obtained based on the material type, cleaning instructions, and amount information.

[0084] After obtaining the cleaning mode information, the movement route information can be determined subsequently, and the area to be cleaned can be moved along the movement route to perform the cleaning process using the cleaning mode information.

[0085] 104. Construct a three-dimensional image of the area to be cleaned based on the point cloud data to obtain a target three-dimensional image.

[0086] The target 3D image can be constructed using a common method of building 3D images from point cloud data. This target 3D image can include both the 3D image of the area to be cleaned and the 3D image of the surrounding area. This allows for path planning based on the 3D image of the surrounding area, thus obtaining the movement route information.

[0087] 105. Determine the movement route information of the intelligent cleaning device to the area to be cleaned based on the target three-dimensional image.

[0088] After determining the target 3D image, a pre-trained path planning model can be used. The target 3D image and the current position of the smart cleaning device are input into the path planning model to calculate the movement route information. During model training, the model is adapted to the scenario in which the smart cleaning device is used. The model can be specifically adjusted for the smart cleaning device to obtain a suitable path planning model. Specifically, multiple images of the scenario in which the smart cleaning device is used need to be collected. These multiple images should generally cover the usage scenario, allowing path planning to be performed based on the current position of the smart cleaning device and the target 3D image to obtain the movement route information.

[0089] 106. Control the intelligent cleaning device to clean the area to be cleaned according to the cleaning mode information and the movement route information.

[0090] Specifically, the mobile module in the intelligent cleaning device can be controlled to move to the area to be cleaned according to the mobile route information, and the cleaning module can be controlled to execute the cleaning mode to clean the area to be cleaned.

[0091] In this example, a smart cleaning device collects point cloud data and a first image set of the area to be cleaned. Based on the first image, it determines the information of the object to be cleaned in the area. Based on the information of the object to be cleaned, it determines the cleaning mode information. Based on the point cloud data, it constructs a 3D image of the area to be cleaned to obtain a target 3D image. Based on the target 3D image, it determines the movement route information of the smart cleaning device to the area to be cleaned. It then controls the smart cleaning device to clean the area to be cleaned according to the cleaning mode information and the movement route information. Therefore, the smart cleaning device can acquire point cloud data and images of the area to be cleaned, generate corresponding cleaning modes and movement route information, and finally control the smart cleaning device to perform cleaning, thereby improving the efficiency and accuracy of cleaning the area to be cleaned.

[0092] In one possible implementation, a method for determining information about the object to be cleaned in the area to be cleaned based on the first image includes:

[0093] A1. Extract features from the first image to obtain first feature data;

[0094] A2. Based on the first feature data, extract the contours to obtain k contour information;

[0095] A3. Based on the k contour information and the corresponding pixel values, identify the object to be cleaned to obtain the object information of the area to be cleaned.

[0096] In this process, a general feature extraction method can be used to extract features from the first image to obtain the first feature data. The first feature data can be, for example, grayscale values.

[0097] After extracting the first feature data, a general contour extraction method can be used to extract the contours, resulting in k contour information pieces. These k contour information pieces include the contour information of the object to be cleaned and the contour information of the object not being cleaned.

[0098] Since there are certain differences between the outlines of clean and unclean objects, a preliminary screening can be performed based on the outline information to obtain a set of reference clean object information. Then, another set of reference clean object information can be obtained by type recognition based on pixel values. Finally, the information of the object to be cleaned in the intersection of the two sets of clean objects can be determined as the actual information of the object to be cleaned.

[0099] Therefore, the object to be cleaned can be comprehensively identified by combining contour and pixel values, thus improving the accuracy of the obtained information about the object to be cleaned.

[0100] In one possible implementation, a method for identifying objects to be cleaned based on the k contour information and corresponding pixel values ​​to obtain information about the objects to be cleaned in the area to be cleaned includes:

[0101] B1. Perform contour type identification on k contour information to obtain the contour type corresponding to each contour information;

[0102] B2. Determine the cleaning object information based on the contour type corresponding to the k contour information to obtain the first reference cleaning object information set;

[0103] B3. Determine the cleaning material information corresponding to each of the k contour information based on the pixel values ​​corresponding to the k contour information, and obtain the second reference cleaning material information set;

[0104] B4. Determine the information of the object to be cleaned based on the first reference cleaning object information set and the second reference cleaning object information set.

[0105] Specifically, a pre-set contour recognition model can be used to identify the type of k contour information to obtain the contour type. The contour recognition model is a pre-trained model used for contour type identification, such as a CNN model.

[0106] Different contour types have corresponding cleaning object information, thus allowing the determination of a first reference cleaning object information set based on the contour type. After determining the first reference cleaning object information set, to further improve the accuracy of cleaning object determination, pixel values ​​can be used to determine the cleaning object information, resulting in a second reference cleaning object information set.

[0107] Specifically, the color information corresponding to the contour information can be determined based on the pixel value, and finally the second reference cleaning object information set can be determined based on the color information. Since the second reference cleaning object information set is determined by the color information, the number of elements in the set will be higher than the number of elements in the first reference cleaning object information set. However, since it is used for verification, it will not affect the accuracy of subsequent determinations.

[0108] Pixel values ​​can be represented using RGB values, allowing color information to be determined based on these values. Finally, the second set of reference cleaning material information is determined based on this color information. Different colors correspond to different reference cleaning material information, thus enabling the determination of the second set of reference cleaning material information.

[0109] The elements in the intersection of the first and second reference cleaning object information sets can be identified as the object information to be cleaned. This object information includes the object's outline and type.

[0110] In this example, the object to be cleaned can be identified by combining the contour and pixel values. When using contour information for identification, if the color of the reference object to be cleaned is different even if the contour matches, there may be cases where the object is not the one that needs to be cleaned. Therefore, combining color information for further correction can improve the accuracy of the information on the object to be cleaned.

[0111] In one possible implementation, a method for determining cleaning mode information based on the information of the object to be cleaned includes:

[0112] C1. Extract the outline information and type information of the object to be cleaned from the information of the object to be cleaned;

[0113] C2. Determine the cleaning material type information based on the aforementioned type information;

[0114] C3. Determine the amount of cleaning materials to be used based on the contour information, and determine the cleaning instructions based on the contour information;

[0115] C4. Determine the cleaning mode information based on the cleaning instruction, the cleaning material type information, and the dosage information.

[0116] Since the information about the object to be cleaned includes its outline and type information, the outline and type information of the object to be cleaned can be extracted directly.

[0117] Different cleaning materials are available for different types of items. For example, for oily items, the cleaning material used could be an oil-removing cleaner; for household scraps, the cleaning material could be an oil-removing and odor-removing cleaner. These are just examples and not specific limitations.

[0118] The size of the outline and the predicted volume corresponding to the outline can be extracted based on the outline information. For example, volume prediction can be performed based on the outline size and the type of the object to be cleaned, resulting in the predicted volume (the predicted volume corresponding to the outline). The dosage information can then be determined based on either the outline size or the predicted volume. Specifically, for example, if the object to be cleaned is oily, the dosage information can be determined based on the outline size; a larger outline size requires a larger dosage, and a smaller outline size requires a smaller dosage. Similarly, if the object to be cleaned is household food scraps, the dosage information can be determined based on the predicted volume; a larger volume requires a larger dosage, and a smaller volume requires a smaller dosage.

[0119] When determining cleaning instructions, contour instructions can be determined based on information about the surrounding area. Contour instructions are used to specify the specific cleaning actions to be performed during cleaning.

[0120] For example, when dealing with solid items to be cleaned, the items can be divided into sections to improve cleaning efficiency; when dealing with non-solid items, they can be cleaned directly to improve efficiency.

[0121] The cleaning instruction, the cleaning material type information, and the dosage information can be determined as the corresponding instruction, material, and dosage in the cleaning mode to obtain the cleaning mode information.

[0122] In one possible implementation, a method for determining a cleaning instruction based on the contour information includes:

[0123] D1. Obtain the surrounding area information corresponding to the contour information;

[0124] D2. Determine n cleaning movement directions based on the surrounding area information;

[0125] D3. Based on the n cleaning movement direction information, the contour indicated by the contour information is split to obtain a set of n sub-contours.

[0126] D4. Based on the n sub-contour sets and the corresponding cleaning direction information, determine n cleaning paths;

[0127] D5. Perform cleaning cost scoring on n cleaning paths to obtain n cleaning cost score values;

[0128] D6. Determine the cleaning path corresponding to the minimum cleaning cost score as the target cleaning path;

[0129] D7. Generate cleaning instructions based on the target cleaning path.

[0130] Among them, the surrounding area information can be understood as the environmental information of the object to be cleaned. The environmental information may include the distribution of surrounding objects, the distribution of nearby facilities, etc.

[0131] When carrying out specific cleaning, it is necessary to take into account the distribution of objects and facilities around the object to be cleaned (i.e., obstacles). Such obstacles will affect the cleaning effect. For example, if there is an object to be cleaned next to a trash can, the trash can can be understood as an obstacle, which will hinder the cleaning of the object to be cleaned. Therefore, it is necessary to plan special cleaning paths to improve the cleaning effect and efficiency.

[0132] Obstacle information can be extracted from the surrounding area information. A cleaning movement direction can be set for each obstacle. For example, a direction with the obstacle as the starting point and the center point of the outline of the object to be cleaned as the ending point can be used as the cleaning movement direction.

[0133] This allows the object to be cleaned to be segmented into n sub-contour sets based on its center. Due to occlusion by obstacles, multiple cleaning paths are determined using the sub-contours and their corresponding cleaning directions. Specifically, a pre-set cleaning path planning model can be used to determine the n cleaning paths. In actual execution, the n sub-contour sets and their corresponding cleaning direction information can be input into the cleaning path planning model to plan the cleaning paths, resulting in n cleaning paths. The cleaning path planning model is a pre-trained model used for cleaning path planning, and its type can be CNN, etc.

[0134] A general cleaning cost scoring method can be used to obtain a clear cost score. Alternatively, the following method can be used: In practical applications, the score can be calculated by combining the movement distance of the cleaning path and the average energy consumption during movement. Average energy consumption can be determined based on the cleaning difficulty of the cleaning path; the greater the cleaning difficulty, the higher the average energy consumption, and vice versa. Cleaning difficulty can be determined based on the contours of moving obstacles along each cleaning path; the larger the obstacle contour, the greater the cleaning difficulty, and vice versa. The obstacle contour can be represented by the height of the object to be cleaned; the higher the height, the larger the contour, and vice versa. Here, the contour size is used to represent the size of the contour in the vertical direction.

[0135] The target cleaning path can be determined as the cleaning path in the cleaning command, and corresponding control commands can be generated to obtain the cleaning command.

[0136] For examples consistent with the above embodiments, please refer to... Figure 2 , Figure 2 This is a schematic diagram of the structure of a terminal provided in an embodiment of this application, such as... Figure 2 As shown, it includes a processor, an input device, an output device, and a memory, which are interconnected. The memory is used to store a computer program, which includes program instructions. The processor is configured to call the program instructions. The program includes instructions for performing the following steps.

[0137] The point cloud data and first image set of the area to be cleaned are collected using intelligent cleaning equipment;

[0138] The information of the object to be cleaned in the area to be cleaned is determined based on the first image;

[0139] The cleaning mode information is determined based on the information of the object to be cleaned;

[0140] A three-dimensional image of the area to be cleaned is constructed based on the point cloud data to obtain a target three-dimensional image.

[0141] Based on the target 3D image, determine the movement route information of the intelligent cleaning device to the area to be cleaned;

[0142] The intelligent cleaning device is controlled to clean the area to be cleaned based on the cleaning mode information and the movement route information.

[0143] The above mainly describes the solutions of the embodiments of this application from the perspective of the method execution process. It is understood that, in order to achieve the above functions, the terminal includes the corresponding hardware structure and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments provided herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0144] This application embodiment can divide the terminal into functional units according to the above method example. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0145] For those consistent with the above, please refer to Figure 3 , Figure 3 This application provides a schematic diagram of the structure of an intelligent cleaning device. For example... Figure 3 As shown, the device is applied to an intelligent cleaning system, which includes intelligent cleaning equipment, and the device includes:

[0146] The acquisition unit 301 is used to acquire point cloud data and a first image set of the area to be cleaned through the intelligent cleaning equipment.

[0147] The first determining unit 302 is used to determine the information of the object to be cleaned in the area to be cleaned based on the first image;

[0148] The second determining unit 303 is used to determine cleaning mode information based on the information of the object to be cleaned;

[0149] Construction unit 304 is used to construct a three-dimensional image of the area to be cleaned based on the point cloud data to obtain a target three-dimensional image;

[0150] The third determining unit 305 is used to determine the movement route information of the intelligent cleaning device to the area to be cleaned based on the target three-dimensional image;

[0151] The control unit 306 is used to control the intelligent cleaning device to clean the area to be cleaned according to the cleaning mode information and the movement route information.

[0152] In one possible implementation, the first determining unit 302 is specifically used for:

[0153] Feature extraction is performed on the first image to obtain first feature data;

[0154] Based on the first feature data, contour extraction is performed to obtain k contour information;

[0155] Based on the k contour information and the corresponding pixel values, the object to be cleaned is identified, and the object information of the area to be cleaned is obtained.

[0156] In one possible implementation, regarding the step of identifying the object to be cleaned based on the k contour information and corresponding pixel values ​​to obtain the object information of the area to be cleaned, the first determining unit 302 is specifically used for:

[0157] Perform contour type identification on k contour information to obtain the contour type corresponding to each contour information;

[0158] Based on the contour types corresponding to k contour information, the cleaning object information is determined, and a first reference cleaning object information set is obtained.

[0159] Based on the pixel values ​​corresponding to the k contour information, the cleaning material information corresponding to the k contour information is determined, and a second reference cleaning material information set is obtained.

[0160] The information of the object to be cleaned is determined based on the first set of reference cleaning object information and the second set of reference cleaning object information.

[0161] In one possible implementation, the second determining unit 303 is specifically used for:

[0162] Extract the outline and type information of the object to be cleaned from the object information;

[0163] The type of cleaning materials is determined based on the aforementioned type information;

[0164] The amount of cleaning materials to be used is determined based on the contour information, and the cleaning instructions are determined based on the contour information.

[0165] The cleaning mode information is determined based on the cleaning instruction, the type of cleaning material, and the dosage information.

[0166] In one possible implementation, regarding the determination of the cleaning instruction based on the contour information, the second determining unit 303 is specifically configured to:

[0167] Obtain the surrounding area information corresponding to the contour information;

[0168] Based on the surrounding area information, determine n cleaning movement directions;

[0169] Based on the n cleaning movement direction information, the contour indicated by the contour information is split to obtain a set of n sub-contours;

[0170] Based on the n sub-contour sets and the corresponding cleaning direction information, n cleaning paths are determined;

[0171] The cleaning cost is scored for n cleaning paths to obtain n cleaning cost scores.

[0172] The cleaning path corresponding to the minimum cleaning cost score is determined as the target cleaning path;

[0173] A cleaning instruction is generated based on the target cleaning path.

[0174] This application also provides a computer storage medium storing a computer program for electronic data interchange, which causes a computer to perform some or all of the steps of any of the intelligent cleaning methods described in the above method embodiments.

[0175] This application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program that causes a computer to perform some or all of the steps of any of the intelligent cleaning methods described in the above method embodiments.

[0176] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0177] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0178] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0179] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0180] Furthermore, the functional units in the various embodiments of the application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software program module.

[0181] If the integrated unit is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0182] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage device, which may include: a flash drive, a read-only memory, a random access memory, a magnetic disk, or an optical disk, etc.

[0183] The embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An intelligent cleaning method, characterized by, The method is applied to an intelligent cleaning system, the intelligent cleaning system including intelligent cleaning equipment, and the method includes: The point cloud data and first image set of the area to be cleaned are collected using intelligent cleaning equipment; The information of the object to be cleaned in the area to be cleaned is determined based on the first image; The cleaning mode information is determined based on the information of the object to be cleaned; A three-dimensional image of the area to be cleaned is constructed based on the point cloud data to obtain a target three-dimensional image. Based on the target 3D image, determine the movement route information of the intelligent cleaning device to the area to be cleaned; The intelligent cleaning device is controlled to clean the area to be cleaned based on the cleaning mode information and the movement route information. The step of determining the cleaning mode information based on the information of the object to be cleaned includes: Extract the outline and type information of the object to be cleaned from the object information; The type of cleaning materials is determined based on the aforementioned type information; The amount of cleaning materials to be used is determined based on the contour information, and the cleaning instructions are determined based on the contour information. The cleaning mode information is determined based on the cleaning instruction, the cleaning material type information, and the dosage information. The step of determining the cleaning instruction based on the contour information includes: Obtain the surrounding area information corresponding to the outline information; wherein, the surrounding area information is the environmental information of the object to be cleaned, and the environmental information includes the distribution of surrounding objects and the distribution of nearby facilities; Based on the surrounding area information, determine n cleaning movement directions; Based on the n cleaning movement direction information, the contour indicated by the contour information is split to obtain a set of n sub-contours; Based on the n sub-contour sets and the corresponding cleaning direction information, n cleaning paths are determined; The cleaning cost is scored for n cleaning paths to obtain n cleaning cost scores. The cleaning path corresponding to the minimum cleaning cost score is determined as the target cleaning path; A cleaning instruction is generated based on the target cleaning path.

2. The intelligent cleaning method of claim 1, wherein, The step of determining the information of the object to be cleaned in the area to be cleaned based on the first image includes: Feature extraction is performed on the first image to obtain first feature data; Based on the first feature data, contour extraction is performed to obtain k contour information; Based on the k contour information and the corresponding pixel values, the object to be cleaned is identified, and the object information of the area to be cleaned is obtained.

3. The intelligent cleaning method of claim 2, wherein, The step of identifying the object to be cleaned based on the k contour information and corresponding pixel values ​​to obtain the object information of the area to be cleaned includes: Perform contour type identification on k contour information to obtain the contour type corresponding to each contour information; Based on the contour types corresponding to k contour information, the cleaning object information is determined, and a first reference cleaning object information set is obtained. Based on the pixel values ​​corresponding to the k contour information, the cleaning material information corresponding to the k contour information is determined, and a second reference cleaning material information set is obtained. The information of the object to be cleaned is determined based on the first set of reference cleaning object information and the second set of reference cleaning object information.

4. An intelligent cleaning device, characterized by The device is applied to an intelligent cleaning system, the intelligent cleaning system including intelligent cleaning equipment, and the device includes: The acquisition unit is used to acquire point cloud data and a first image set of the area to be cleaned through intelligent cleaning equipment; The first determining unit is configured to determine the information of the object to be cleaned in the area to be cleaned based on the first image. The second determining unit is used to determine cleaning mode information based on the information of the object to be cleaned; A construction unit is used to construct a three-dimensional image of the area to be cleaned based on the point cloud data, thereby obtaining a target three-dimensional image. The third determining unit is used to determine the movement route information of the intelligent cleaning device to the area to be cleaned based on the target three-dimensional image; A control unit is used to control the intelligent cleaning device to clean the area to be cleaned according to the cleaning mode information and the movement route information. The second determining unit is used to determine cleaning mode information based on the information of the object to be cleaned, specifically for: Extract the outline and type information of the object to be cleaned from the object information; The type of cleaning materials is determined based on the aforementioned type information; The amount of cleaning materials to be used is determined based on the contour information, and the cleaning instructions are determined based on the contour information. The cleaning mode information is determined based on the cleaning instruction, the cleaning material type information, and the dosage information. The second determining unit is used to determine a cleaning instruction based on the contour information, specifically for: Obtain the surrounding area information corresponding to the outline information; wherein, the surrounding area information is the environmental information of the object to be cleaned, and the environmental information includes the distribution of surrounding objects and the distribution of nearby facilities; Based on the surrounding area information, determine n cleaning movement directions; Based on the n cleaning movement direction information, the contour indicated by the contour information is split to obtain a set of n sub-contours; Based on the n sub-contour sets and the corresponding cleaning direction information, n cleaning paths are determined; The cleaning cost is scored for n cleaning paths to obtain n cleaning cost scores. The cleaning path corresponding to the minimum cleaning cost score is determined as the target cleaning path; A cleaning instruction is generated based on the target cleaning path.

5. The intelligent cleaning device of claim 4, wherein, The first determining unit is specifically used for: Feature extraction is performed on the first image to obtain first feature data; Based on the first feature data, contour extraction is performed to obtain k contour information; Based on the k contour information and the corresponding pixel values, the object to be cleaned is identified, and the object information of the area to be cleaned is obtained.

6. The intelligent cleaning device of claim 5, wherein, In the process of identifying the object to be cleaned based on the k contour information and corresponding pixel values ​​to obtain the object information of the area to be cleaned, the first determining unit is specifically used for: Perform contour type identification on k contour information to obtain the contour type corresponding to each contour information; Based on the contour types corresponding to k contour information, the cleaning object information is determined, and a first reference cleaning object information set is obtained. Based on the pixel values ​​corresponding to the k contour information, the cleaning material information corresponding to the k contour information is determined, and a second reference cleaning material information set is obtained. The information of the object to be cleaned is determined based on the first set of reference cleaning object information and the second set of reference cleaning object information.

7. A terminal, characterized by comprising: The device includes a processor, an input device, an output device, and a memory, which are interconnected. The memory is used to store a computer program, which includes program instructions. The processor is configured to invoke the program instructions to execute the intelligent cleaning method as described in any one of claims 1-3.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the intelligent cleaning method as described in any one of claims 1-3.

Images