Intelligent cleaning method and device based on historical image, equipment and medium

Abstract

The invention relates to an intelligent cleaning method and device based on historical images, equipment, a storage medium and a computer program.The method comprises the steps that a target cleaning area is divided into a plurality of grid units, and corresponding target cleanliness is generated according to historical cleaning data of each grid unit, acquiring a current ground sampling image in a current moving unit in a cleaning process, positioning a plurality of target grid units according to a coverage range of the current ground sampling image, and splicing or cutting the ground images with the target cleanliness of the plurality of target grid units, the method comprises the steps that a current ground sampling image is obtained, a current target cleaning image with the same coverage range as the current ground sampling image is obtained, the current ground sampling image is compared with the current target cleaning image, the current cleanliness corresponding to the current ground sampling image is obtained, and current cleaning operation is determined according to the current cleanliness. And updating the historical cleaning data of each grid unit by using the ground image after the current cleaning operation is completed.

Description

Technical Field

[0001] The present disclosure relates to the field of image recognition technology, and in particular to an intelligent cleaning method, device, equipment and storage medium based on historical images. Background Art

[0002] Sweeping robots developed and produced by many companies have been widely used in thousands of households. They can automatically clean the floor of the target cleaning area (including vacuuming, sweeping and mopping, etc.). Among them, cleaning ability (i.e. the ability to clean the floor sufficiently) is one of the main indicators for measuring sweeping robots and is also the issue that consumers are most concerned about.

[0003] In order to improve the cleaning ability and automation ability of the sweeping robot, the existing technology records the historical cleaning data of each grid unit on the target cleaning area map to determine the target cleanliness of each grid unit, and compares the target cleanliness of each grid unit with the current cleanliness to more accurately determine whether the current grid unit is clean or not. However, it is difficult to ensure that the starting point and travel path of the sweeping robot are exactly the same between multiple rounds of cleaning. This makes it difficult to completely match the trajectory range of the sweeping robot during the cleaning process (such as the trajectory range formed by mopping cleaning) with the grid division, that is, the range of a section of the trajectory is likely to completely cover several grid units and partially cover several grid units at the same time. Therefore, it is necessary to solve the real-time mapping problem between the dynamically changing trajectory range image and the gridded image. Summary of the invention

[0004] The present disclosure provides an intelligent cleaning method, device, equipment and storage medium based on historical images to solve the problem that the cleaning trajectory of the existing sweeping robot does not match the grid division after multiple rounds of cleaning.

[0005] In a first aspect, the present disclosure provides an intelligent cleaning method based on historical images, comprising:

[0006] A plurality of historical cleaning data of a domain, each piece of historical cleaning data including a ground image of a corresponding sub-region;

[0007] Generating a target cleanliness of each grid unit according to historical cleaning data of each grid unit, wherein the target cleanliness is represented as a ground image;

[0008] During the cleaning process, the current ground sampling image within the current moving unit is obtained;

[0009] Locating a number of target grid cells according to the coverage of the current ground sampling image, and splicing or cropping the ground images of the target cleanliness of the target grid cells to obtain a current target clean image with the same coverage as the current ground sampling image;

[0010] By comparing the current ground sampling image with the current target cleaning image, a current cleanliness corresponding to the current ground sampling image is obtained;

[0011] determining a current cleaning operation according to the current cleanliness;

[0012] The historical cleaning data of each grid unit is updated using the ground image after the current cleaning operation is completed.

[0013] In some embodiments, after dividing the target cleaning area into a grid unit consisting of a plurality of units, the method further includes:

[0014] Use cleaning equipment to clean the coverage area of ​​each grid unit;

[0015] Obtain the cleaned historical image of the area covered by each grid cell;

[0016] Obtain cleaning intensity data and cleaning duration data for the area covered by each grid unit;

[0017] The post-cleaning historical image, the cleaning intensity data, and the cleaning duration data are aggregated to obtain a piece of historical cleaning data for each grid unit.

[0018] In some embodiments, the step of acquiring a current ground sampling image within a current mobile unit during the cleaning process includes:

[0019] Using a collection device disposed in the cleaning device to continuously collect ground images at a preset collection frequency to obtain a ground image set;

[0020] Sorting each ground image in the ground image set according to the order of shooting time to obtain a ground image sequence;

[0021] Identifying overlapping areas of adjacent images in the ground image sequence;

[0022] Overlapping areas of adjacent images in the ground image sequence are removed and then spliced ​​to obtain a ground sampling image.

[0023] In some embodiments, the method of locating a plurality of target grid units according to the coverage of the current ground sampling image, splicing or cropping the ground images of the target cleanliness of the plurality of target grid units, and obtaining a current target clean image with the same coverage as the current ground sampling image includes:

[0024] Acquire the position information of the area covered by the current ground sampling image in the target cleaning area by positioning;

[0025] Locate the grid cells covered by the current ground sampling image in the target cleaning area according to the position information to obtain a grid cell set;

[0026] Acquire a target cleanliness ground image of each grid cell in the grid cell set;

[0027] The target cleanliness ground images of all grid cells are spliced ​​according to the positional relationship between the grid cells to obtain a spliced ​​target clean image;

[0028] If the stitching target clean image is larger than the range of the current ground sampling image, the stitching target clean image is cropped to the same range as the current ground sampling image to obtain the current target clean image.

[0029] In some embodiments, obtaining the current cleanliness corresponding to the current ground sampling image by comparing the current ground sampling image with the current target cleaning image includes:

[0030] Obtaining the current cleanliness by comparing the pixel difference between the current ground sampling image and the current target cleaning image;

[0031] Alternatively, characteristic information of the current ground sampling image and the current target cleaning image are calculated respectively, and the current cleanliness is obtained by comparing the characteristic information of the two images, wherein the characteristic information includes any one or a combination of characteristic values, characteristic vectors or characteristic sub-regions.

[0032] In some embodiments, determining a current cleaning operation according to the current cleanliness comprises:

[0033] Determine whether the current cleanliness is less than a preset cleanliness threshold and whether the cleaning time of the floor area corresponding to the current cleanliness is less than a preset time threshold;

[0034] If the floor cleanliness is less than a preset cleanliness threshold and the cleaning time of the floor area corresponding to the current cleanliness is less than a preset time threshold, the floor area corresponding to the current cleanliness is cleaned using a cleaning device;

[0035] If the floor cleanliness is greater than or equal to a preset cleanliness threshold or the cleaning time of the floor within the moving unit range is greater than or equal to a preset time threshold, it is confirmed that the cleaning of the floor area corresponding to the current cleanliness is completed.

[0036] In some embodiments, generating a target cleanliness level for each grid unit according to the historical cleaning data of each grid unit includes:

[0037] Using the ground images of several pieces of historical cleaning data of each grid unit, a target cleaning image of each grid unit is generated based on an analysis and prediction method;

[0038] For each grid unit, the analysis and prediction method is used to analyze and predict the area of ​​the grid unit as a whole, and a target cleaning image of each grid unit is generated to obtain a target cleanliness of each grid unit;

[0039] Alternatively, the area of ​​the grid unit is divided into several small areas, each small area is analyzed and predicted respectively, and then the analysis and prediction results of all small areas are combined to obtain the analysis and prediction results of the grid unit, and the target cleanliness of each grid unit is obtained;

[0040] Alternatively, each pixel point in the area of ​​the grid unit is analyzed and predicted respectively, and then the analysis and prediction results of the grid unit are summarized to obtain the target cleanliness of each grid unit.

[0041] In a second aspect, the present disclosure provides an intelligent cleaning device based on historical images, comprising:

[0042] A unit division module is used to divide the target cleaning area into a grid unit composed of a plurality of units, wherein each grid unit stores a plurality of historical cleaning data of a corresponding sub-area, and each piece of historical cleaning data includes a ground image of the corresponding sub-area;

[0043] An image acquisition module is used to generate a target cleanliness of each grid unit according to the historical cleaning data of each grid unit, wherein the target cleanliness is expressed as a ground image, and during the cleaning process, a current ground sampling image in the current moving unit is acquired, and a number of target grid units are located according to the coverage of the current ground sampling image, and the ground images of the target cleanliness of the several target grid units are spliced ​​or cropped to obtain a current target cleaning image with the same coverage as the current ground sampling image;

[0044] A data calculation module, used for obtaining a current cleanliness level corresponding to the current ground sampling image by comparing the current ground sampling image with the current target clean image;

[0045] The cleaning operation module is used to determine the current cleaning operation according to the current cleanliness, and update the historical cleaning data of each grid unit using the ground image after completing the current cleaning operation.

[0046] In a third aspect, the present disclosure provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in the above aspects.

[0047] In a fourth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the method described in the above aspects when executed by a processor.

[0048] The present disclosure provides an intelligent cleaning method, device, equipment and storage medium based on historical images. The method divides a target cleaning area into several grid units, wherein each grid unit stores several historical cleaning data of the corresponding sub-area, and each historical cleaning data includes a ground image of the corresponding sub-area. According to the historical cleaning data of each grid unit, a target cleanliness of each grid unit is generated, wherein the target cleanliness is represented by a ground image. During the cleaning process, a current ground sampling image within a current moving unit is obtained, and several target grid units are located according to the coverage of the current ground sampling image. The ground images of the target cleanliness of the several target grid units are spliced ​​or cropped to obtain a current target cleaning image with the same coverage as the current ground sampling image. By comparing the current ground sampling image with the current target cleaning image, the current cleanliness corresponding to the current ground sampling image is obtained, the current cleaning operation is determined according to the current cleanliness, and the historical cleaning data of each grid unit is updated using the ground image after the current cleaning operation is completed. Thus, the problem that the cleaning trajectory of the existing sweeping robot does not match the grid division after multiple rounds of cleaning is solved, and the cleaning efficiency and cleaning effect of the sweeping robot are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The present disclosure will be described in more detail below based on embodiments and with reference to the accompanying drawings:

[0050] Figure 1 A schematic diagram of a process flow of an intelligent cleaning method based on historical images provided in an embodiment of the present disclosure;

[0051] Figure 2 A functional module diagram of an intelligent cleaning device based on historical images provided in an embodiment of the present disclosure.

[0052] In the drawings, the same reference numerals are used for the same components, and the drawings are not drawn to scale. DETAILED DESCRIPTION

[0053] In order to enable those skilled in the art to better understand the technical solution of the present disclosure, and to fully understand and implement how the present disclosure applies technical means to solve technical problems and achieve the corresponding technical effects, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only embodiments of a part of the present disclosure, not all of the embodiments. The embodiments of the present disclosure and the various features in the embodiments can be combined with each other without conflict, and the technical solutions formed are all within the scope of protection of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of the present disclosure.

[0054] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0055] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0056] Embodiment 1

[0057] Figure 1 The following is a flow chart of an intelligent cleaning method based on historical images provided in an embodiment of the present disclosure. Figure 1 As shown, a smart device control method includes:

[0058] S1. Divide the target cleaning area into several grid units.

[0059] In the embodiment of the present invention, the target cleaning area refers to an area that needs to be cleaned and is preset by a user. This area may be a specific room in the room, a specific space, or a series of continuous spaces.

[0060] In detail, the target cleaning area may be identified and memorized using a navigation system and a mapping system built into the cleaning device.

[0061] In the embodiment of the present invention, dividing the target cleaning area into a plurality of grid units means dividing the target cleaning area into a plurality of cleaning units for subsequent cleaning planning.

[0062] In detail, the target cleaning area is divided into a number of grid units, which may be evenly divided into the target cleaning area into a number of grid units, or non-evenly divided into the target cleaning area into a number of grid units, which needs to be determined according to the shape and size of the target cleaning area.

[0063] In detail, the shape of each grid unit can be regular (such as square and rectangular) or irregular, and the area of ​​each grid unit needs to be larger than the coverage area of ​​the cleaning device.

[0064] In detail, each grid unit stores several pieces of historical cleaning data of the corresponding sub-region, and each piece of historical cleaning data includes a ground image of the corresponding sub-region.

[0065] In the embodiment of the present invention, after dividing the target cleaning area into a grid unit consisting of a plurality of units, the method further includes:

[0066] Use cleaning equipment to clean the coverage area of ​​each grid unit;

[0067] Obtain the cleaned historical image of the area covered by each grid cell;

[0068] Obtain cleaning intensity data and cleaning duration data for the area covered by each grid unit;

[0069] The post-cleaning historical image, the cleaning intensity data, and the cleaning duration data are aggregated to obtain a piece of historical cleaning data for each grid unit.

[0070] In the embodiment of the present invention, by dividing the target cleaning area into a plurality of grid units, the efficiency of subsequent floor cleaning is improved, and the accuracy of subsequent recognition of floor cleanliness is improved.

[0071] S2. Generate a target cleanliness level for each grid unit based on the historical cleaning data of each grid unit.

[0072] In the embodiment of the present invention, the historical cleaning data of each grid unit includes a ground image of the area where the grid unit is located after cleaning, as well as information such as the number of cleanings, cleaning intensity, and cleaning duration of the area where the grid unit is located.

[0073] In detail, the historical cleaning data is recorded as initial historical cleaning data after the target cleaning area is cleaned using a cleaning device, and the historical cleaning data is continuously updated after subsequent cleaning operations are completed.

[0074] In detail, the target cleanliness is represented as a ground image.

[0075] In the embodiment of the present invention, generating the target cleanliness of each grid unit according to the historical cleaning data of each grid unit includes:

[0076] Using the ground images of several pieces of historical cleaning data of each grid unit, a target cleaning image of each grid unit is generated based on an analysis and prediction method;

[0077] For each grid unit, the analysis and prediction method is used to analyze and predict the area of ​​the grid unit as a whole, and a target cleaning image according to each grid unit is generated to obtain a target cleanliness of each grid unit;

[0078] Alternatively, the area of ​​the grid unit is divided into several small areas, each small area is analyzed and predicted respectively, and then the analysis and prediction results of all small areas are combined to obtain the analysis and prediction results of the grid unit, and the target cleanliness of each grid unit is obtained;

[0079] Alternatively, each pixel point in the area of ​​the grid unit is analyzed and predicted respectively, and then the analysis and prediction results of the grid unit are summarized to obtain the target cleanliness of each grid unit.

[0080] In the embodiment of the present invention, the target cleanliness of each grid unit is generated according to the historical cleaning data of each grid unit, thereby improving the subsequent acquisition of the current target cleaning image.

[0081] S3. Acquire the current ground sampling image within the current moving unit during the cleaning process.

[0082] In the embodiment of the present invention, the current ground sampling image is obtained by continuously collecting ground images while the cleaning device is moving to obtain an image sequence, cutting out the overlapping areas of all images in the image sequence, and finally stitching them together.

[0083] In the embodiment of the present invention,

[0084] The step of obtaining the current ground sampling image in the current mobile unit during the cleaning process includes:

[0085] Using a collection device disposed in the cleaning device to continuously collect ground images at a preset collection frequency to obtain a ground image set;

[0086] Sorting each ground image in the ground image set according to the order of shooting time to obtain a ground image sequence;

[0087] Identifying overlapping areas of adjacent images in the ground image sequence;

[0088] Overlapping areas of adjacent images in the ground image sequence are removed and then spliced ​​to obtain a ground sampling image.

[0089] In detail, the preset acquisition frequency needs to be set according to actual requirements. The higher the preset acquisition frequency is, the higher the accuracy of the final ground sampling image is, but the higher the requirement on the hardware is.

[0090] In detail, removing the overlapping areas of adjacent images in the ground image sequence and then stitching them together refers to using a later-shot image in the ground image sequence to cover the overlapping areas with the earlier-shot images.

[0091] In the embodiment of the present invention, by acquiring the current ground sampling image in the current moving unit during the cleaning process, the efficiency of subsequently acquiring the current cleanliness through comparison is improved.

[0092] S4. Locate a number of target grid units according to the coverage of the current ground sampling image, and splice or crop the ground images of the target cleanliness of the target grid units to obtain a current target clean image with the same coverage as the current ground sampling image.

[0093] In the embodiment of the present invention, the current target clean image refers to an image having the same size as the current ground sampling image that is obtained by cropping and splicing historical clean images included in the historical cleaning data.

[0094] In an embodiment of the present invention, the method of locating a plurality of target grid units according to the coverage of the current ground sampling image, splicing or cropping the ground images of the target cleanliness of the plurality of target grid units, and obtaining a current target clean image having the same coverage as the current ground sampling image includes:

[0095] Acquire position information of the area covered by the current ground sampling image in the target cleaning area by positioning;

[0096] Locate the grid cells covered by the current ground sampling image in the target cleaning area according to the position information to obtain a grid cell set;

[0097] Acquire a target cleanliness ground image of each grid cell in the grid cell set;

[0098] The target cleanliness ground images of all grid cells are spliced ​​according to the positional relationship between the grid cells to obtain a spliced ​​target clean image;

[0099] If the stitching target clean image is larger than the range of the current ground sampling image, the stitching target clean image is cropped to the same range as the current ground sampling image to obtain the current target clean image.

[0100] In detail, since the current ground sampling image may be larger than a single grid unit, the covered grid unit includes multiple grid units.

[0101] In detail, since some areas of the covered grid cells are covered, the stitched image needs to be cropped to obtain a target clean image of the same size as the current ground sampling image.

[0102] In an embodiment of the present invention, a number of target grid units are located based on the coverage of the current ground sampling image, and the ground images of the target cleanliness of the several target grid units are spliced ​​or cropped to obtain a current target cleaning image with the same coverage as the current ground sampling image, thereby improving the efficiency of obtaining the current cleanliness.

[0103] S5. Obtain a current cleanliness level corresponding to the current ground sampling image by comparing the current ground sampling image with the current target cleaning image.

[0104] In the embodiment of the present invention, the current cleanliness corresponding to the current ground sampling image is obtained by comparing the current ground sampling image with the current target clean image, including:

[0105] Obtaining the current cleanliness by comparing the pixel difference between the current ground sampling image and the current target cleaning image;

[0106] Alternatively, characteristic information of the current ground sampling image and the current target cleaning image are calculated respectively, and the current cleanliness is obtained by comparing the characteristic information of the two images, wherein the characteristic information includes any one or a combination of characteristic values, characteristic vectors or characteristic sub-regions.

[0107] Furthermore, the step of obtaining the current cleanliness corresponding to the current ground sampling image by comparing the current ground sampling image with the current target clean image further includes:

[0108] gray-scale the current ground sampling image to obtain a gray-scale ground image;

[0109] Performing grayscale processing on the target cleaning image to obtain a grayscale target image;

[0110] Scaling the grayscale ground image and the grayscale target image to the same size, to scale the ground image and the scaled target image;

[0111] The pixel difference value between the scaled ground image and the scaled target image is calculated to obtain the current cleanliness.

[0112] In detail, the pixel difference value between the scaled ground image and the scaled target image is calculated to obtain the current cleanliness, which is to calculate the gray value difference between each corresponding pixel point of the scaled ground image and the scaled target image, and finally calculate the average value as the difference value between the scaled ground image and the scaled target image to reflect the cleanliness level.

[0113] In the embodiment of the present invention, the step of calculating the pixel difference value between the grayscale ground image and the grayscale target image to obtain the current cleanliness level includes:

[0114] The current cleanliness is calculated using the following formula:

[0115]

[0116] Among them, C is the current cleanliness, M is the number of pixel rows of the scaled ground image and the scaled target image, N is the number of pixel columns of the scaled ground image and the scaled target image, I(x,y) represents the grayscale value of the pixel point in the xth row and yth column of the target ground image, and J(x,y) represents the grayscale value of the pixel point in the xth row and yth column of the scaled ground image.

[0117] In detail, the grayscale process is a basic operation in image processing, and its main purpose is to convert a color image into a grayscale image. A grayscale image is an image that only contains brightness information, not color information. In the grayscale process, each pixel of the image is assigned a grayscale value according to a certain algorithm. This value represents the brightness of the pixel, usually ranging from 0 (black) to 255 (white).

[0118] In detail, scaling the grayscale ground image and the grayscale target image to the same size, to scale the ground image and the scaled target image, means scaling the grayscale ground image and the grayscale target image to the same resolution size, among the grayscale ground image and the grayscale target image, the resolution of the image with the smallest resolution is used as the target resolution, for example, if the resolution of the grayscale ground image is 1920*1080 and the resolution of the grayscale target image is 2160*1440, then the resolution of the grayscale target image is scaled to 1920*1080, and the grayscale ground image is not processed.

[0119] Furthermore, obtaining the current cleanliness by comparing the current ground sampling image with the target cleaning image further includes:

[0120] The characteristic information of the current ground sampling image and the current target clean image are calculated respectively, and the current cleanliness is obtained by comparing the characteristic information of the two images in terms of color or roughness, wherein the characteristic information includes any one or a combination of characteristic values, characteristic vectors or characteristic sub-regions.

[0121] In some cases, the ground image in the historical cleaning data and the current target cleaning image are obtained under different lighting conditions, which makes it difficult to directly use the pixel difference value between the two images to obtain the current cleanliness. By calculating the characteristic information such as the eigenvalues, eigenvectors or characteristic sub-regions of the two images to compare the two images, a current cleanliness that is more in line with the actual situation can be obtained when there is a difference in lighting. The dirty sub-regions on the ground often have some obvious features on the image, such as the characteristic sub-regions with colors or roughness that are obviously different from the surrounding images, or there are obvious color or roughness gradient changes at the junction with the surrounding images.

[0122] In addition, the pixel comparison and feature information methods can be combined to obtain the current cleanliness. For example, two images are first compared based on pixels to obtain a comparison image; then the feature information of the comparison image is calculated, and the current cleanliness is determined based on the feature information.

[0123] In the embodiment of the present invention, by comparing the current ground sampling image with the current target cleaning image, the current cleanliness corresponding to the current ground sampling image is obtained, thereby improving the efficiency of subsequent judgment on whether cleaning is required.

[0124] S6. Determine a current cleaning operation according to the current cleanliness.

[0125] In an embodiment of the present invention, determining the current cleaning operation according to the current cleanliness level includes:

[0126] Determine whether the current cleanliness is less than a preset cleanliness threshold and whether the cleaning time of the floor area corresponding to the current cleanliness is less than a preset time threshold;

[0127] If the floor cleanliness is less than a preset cleanliness threshold and the cleaning time of the floor area corresponding to the current cleanliness is less than a preset time threshold, the floor area corresponding to the current cleanliness is cleaned using a cleaning device;

[0128] If the floor cleanliness is greater than or equal to a preset cleanliness threshold or the cleaning time of the floor within the moving unit range is greater than or equal to a preset time threshold, it is confirmed that the cleaning of the floor area corresponding to the current cleanliness is completed.

[0129] In an embodiment of the present invention, the cleanliness threshold can be set according to the specific needs of the user. A cleanliness threshold that is too high may cause the current cleanliness obtained by each cleanliness identification of the same area to be less than the cleanliness threshold, thereby causing the number of cleaning times for the same area to be too low; correspondingly, a cleanliness threshold that is too low will cause the same area to be cleaned every time the cleanliness is identified, thereby reducing the overall cleaning efficiency of the target cleaning area.

[0130] In the embodiment of the present invention, when the current cleanliness is less than a preset cleanliness threshold, it indicates that the current floor needs to be cleaned, and thus a cleaning device is used to clean the area covered by the current floor sampling image.

[0131] In detail, after using the cleaning equipment to clean the area covered by the current ground sampling image, the acquisition device set in the cleaning equipment is reused to obtain the current ground sampling image, and the step of calculating the cleanliness based on the historical image is re-executed to re-judge whether the current ground is clean.

[0132] In the embodiment of the present invention, when the current cleanliness is greater than or equal to a preset cleanliness threshold, it indicates that the current floor cleanliness meets the requirements and does not need to be cleaned.

[0133] In the embodiment of the present invention, when the current cleanliness is greater than or equal to a preset cleanliness threshold, the efficiency of intelligent cleaning is improved by updating the current floor cleaning image to the historical cleaning data of the grid unit covered by the current floor cleaning image.

[0134] S7. Update the historical cleaning data of each grid unit using the ground image after the current cleaning operation is completed.

[0135] In the embodiment of the present invention, the historical cleaning data contained in each grid unit needs to be continuously updated. After each cleaning operation is completed, the latest ground image of each grid unit is obtained and updated to the historical cleaning data for analysis and comparison during the next cleaning operation.

[0136] Embodiment 2

[0137] Based on the above embodiments, this embodiment provides an application example.

[0138] This embodiment provides an intelligent cleaning technology based on historical image data, characterized in that the method includes:

[0139] The target cleaning area is divided into a grid composed of a number of units, each grid unit stores a number of historical cleaning data of the corresponding sub-area, and each historical cleaning data includes a ground image of the corresponding sub-area;

[0140] Generate a target cleanliness level for each grid unit based on historical cleaning data of each grid unit, wherein the target cleanliness level is represented as a ground image;

[0141] During the cleaning process, the current ground sampling image within the current moving unit is obtained;

[0142] According to the range of the current ground sampling image, the corresponding target grid units are located, and the ground images of the target cleanliness of the target grid units are spliced ​​or cropped to form a current target clean image with the same range as the current ground sampling image; and the current target clean image is compared to determine the current cleanliness corresponding to the current ground sampling image;

[0143] determining a current cleaning operation according to the current cleanliness;

[0144] The historical cleaning data of each grid cell is updated using the ground image after cleaning.

[0145] In the embodiment of the present invention, the target cleaning area is divided into a grid composed of a plurality of units, each grid unit stores a plurality of historical cleaning data of the corresponding sub-area, and each historical cleaning data includes a ground image of the corresponding sub-area, which means that the grid division for the target cleaning area can be uniform or non-uniform, regular (such as a square or rectangle) or irregular. There is usually no overlap between different sub-areas corresponding to different grid units, and the sub-areas of all grid units are spliced ​​together to cover the entire target cleaning area.

[0146] Furthermore, during multiple rounds of cleaning for the same target cleaning area, the grid division remains unchanged. Each grid unit stores several pieces of historical cleaning data for the corresponding sub-area, and each piece of historical cleaning data includes a ground image after the corresponding sub-area is cleaned, that is, the ground image is a representation method of cleanliness. In addition, each piece of historical cleaning data also includes information such as the number of cleaning times, cleaning intensity or cleaning time of the corresponding sub-area.

[0147] In an embodiment of the present invention, the target cleanliness of each grid unit is generated based on the historical cleaning data of each grid unit, and the target cleanliness is expressed as a ground image, which means that the target cleanliness image of a grid unit is generated by analyzing and predicting the ground images of several historical cleaning data of the grid unit. The analysis and prediction method can be a prediction method based on mathematical functions, or a prediction method based on artificial intelligence, etc. The analysis and prediction method can consider the area of ​​the grid unit as a whole (that is, consider each historical ground image as a whole); or divide the area of ​​the grid unit into several small areas, and determine the target cleanliness of each small area, and then determine the target cleanliness of the entire grid unit (at this time, each historical ground image will be divided into several sub-images in the same way); or even consider it in units of pixels.

[0148] In an embodiment of the present invention, the movement unit is used to quantify the movement behavior of the sweeping robot during the cleaning process, such as movement per unit time, movement per unit displacement, etc., including displacement modes such as straight-line movement, turning, turning around, and reciprocating within a certain range (repeated cleaning).

[0149] In an embodiment of the present invention, the current ground sampling image in the current mobile unit is derived from the image captured by the sensor (such as a camera) during the movement of the sweeping robot, which means that the sensor usually needs to take pictures of the ground at a certain frequency (video is essentially high-frequency photography) to form an image sequence. On the one hand, a collection of image sequences needs to completely cover the trajectory range of the current mobile unit, and on the other hand, there are likely to be overlapping sub-areas between different images. Therefore, the current ground sampling image needs to be obtained by synthesizing the image sequence. The synthesis method is mainly to splice according to the positional relationship and sequence between the images in the sequence and remove the overlapping sub-areas (generally speaking, in the final current ground sampling image, the later-shot image can cover the overlapping area with the earlier-shot image).

[0150] In detail, the range of the current ground sampling image may completely cover several grid cells and partially cover several grid cells at the same time. These grid cells are called target grid cells of the current ground sampling image. It is necessary to synthesize the ground images of the target cleanliness of all target grid cells to obtain the current target clean image with the same range as the current ground sampling image, so as to compare and analyze the two images. The specific synthesis method may include two aspects: 1) splicing all ground images of target cleanliness according to the position relationship; 2) cropping the spliced ​​ground image to remove the sub-area ranges not covered by the current ground sampling image.

[0151] There are relatively mature technologies for comparing the current ground sampling image with the current target clean image, such as pixel-based comparison, AI-based image recognition technology, etc. The greater the difference between the current ground sampling image and the current target clean image, the lower the current cleanliness corresponding to the current ground sampling image.

[0152] In an embodiment of the present invention, determining the current cleaning operation according to the current cleanliness level includes:

[0153] When the current cleanliness level is high, there is no need to clean the ground sub-area corresponding to the current ground sampling image; when the current cleanliness level is low, the ground sub-area corresponding to the current ground sampling image can be cleaned again or repeatedly until the continuously updated current cleanliness level meets the cleanliness requirements or the cleaning time exceeds the threshold limit.

[0154] In the embodiment of the present invention, the use of the ground image after cleaning to update the historical cleaning data of each grid unit refers to

[0155] The ground image after cleaning will first be collected and synthesized according to the moving unit to form several ground sampling images after cleaning. The historical cleaning data of each grid unit that overlaps with the image area can be updated immediately after obtaining a ground sampling image after cleaning, or the historical cleaning data of all grid units can be updated after the entire round of cleaning of the entire target cleaning area is completed and all ground sampling images after cleaning are obtained. There is a generation order between different ground sampling images after cleaning (which also corresponds to the order in which the cleaning of the corresponding image area is completed), and there may also be overlapping areas; for any overlapping area, usually only the last generated ground sampling image after cleaning needs to be paid attention to, that is, the historical cleaning data in the grid unit corresponding to the overlapping area is determined by the last generated ground sampling image after cleaning.

[0156] In an embodiment of the present invention, the historical cleaning data of a grid unit may include not only the image of the ground after cleaning, but also the number of cleanings, cleaning intensity, cleaning duration, etc.; this information may be updated to the historical cleaning data of the corresponding grid unit. During the cleaning process, the sweeping robot may record information such as the number of cleanings, cleaning intensity, cleaning duration, etc. at different locations on the ground. Therefore, when a sampling image of the ground after cleaning is generated, the number of cleanings, cleaning intensity, cleaning duration, etc. corresponding to the image may be obtained for updating the historical cleaning data.

[0157] In an embodiment of the present invention, when a post-cleaning ground image of the historical cleaning data of a grid unit is synthesized from partial areas of multiple post-cleaning ground sampling images, it can be recorded as an entire ground image, and the number of cleanings, cleaning intensity and cleaning time corresponding to the image are comprehensively determined by the corresponding information of the multiple post-cleaning ground sampling images (such as the average value); it can also be recorded in the form of several sub-images, wherein each sub-image comes from a post-cleaning ground sampling image, and each sub-image has corresponding information such as the number of cleanings, cleaning intensity and cleaning time.

[0158] Embodiment 3

[0159] like Figure 2 , which is a functional module diagram of an intelligent cleaning device based on historical images provided in this embodiment.

[0160] The intelligent cleaning device 100 based on historical images of the present invention can be installed in an electronic device. According to the functions to be implemented, the intelligent cleaning device 100 based on historical images can include a unit division module 101, an image acquisition module 102, a data calculation module 103 and a cleaning operation module 104. The module of the present invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0161] In this embodiment, the functions of each module / unit are as follows:

[0162] The unit division module 101 is used to divide the target cleaning area into a grid unit composed of a plurality of units, wherein each grid unit stores a plurality of historical cleaning data of a corresponding sub-area, and each piece of historical cleaning data includes a ground image of the corresponding sub-area;

[0163] The image acquisition module 102 is used to generate a target cleanliness of each grid unit according to the historical cleaning data of each grid unit, wherein the target cleanliness is expressed as a ground image, and during the cleaning process, a current ground sampling image in the current moving unit is acquired, and a number of target grid units are located according to the coverage of the current ground sampling image, and the ground images of the target cleanliness of the several target grid units are spliced ​​or cropped to obtain a current target cleaning image with the same coverage as the current ground sampling image;

[0164] The data calculation module 103 is used to obtain the current cleanliness corresponding to the current ground sampling image by comparing the current ground sampling image with the current target clean image;

[0165] The cleaning operation module 104 is used to determine a current cleaning operation according to the current cleanliness, and to update the historical cleaning data of each grid unit using the floor image after the current cleaning operation is completed.

[0166] Embodiment 4

[0167] On the basis of the above embodiments, this embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in the above embodiments.

[0168] In some implementations of this embodiment, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that when the computer program is executed by a processor, the steps of the method described in the above embodiment are implemented.

[0169] In some implementations of this embodiment, a computer program product is provided, including a computer program, characterized in that when the computer program is executed by a processor, the steps of the method described in the above embodiment are implemented.

[0170] The processor may include, but is not limited to, one or more processors or microprocessors, etc. Each processor may be an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor or other electronic components to execute the methods in the above embodiments.

[0171] Computer-readable storage media can be implemented by any type of volatile or non-volatile storage device or a combination thereof. Computer-readable storage media may include, but are not limited to, for example, random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, computer storage media (such as hard disks, floppy disks, solid-state drives, removable disks, Blu-ray disks, etc.).

[0172] The computer-readable storage medium may also store at least one computer executable program, such as a computer-readable instruction. The computer-readable storage medium includes, but is not limited to, for example, a volatile memory and / or a non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The computer-readable storage medium may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. For example, a non-transitory computer-readable storage medium may be connected to a computing device such as a computer, and then, when the computing device runs the computer-readable instructions stored on the computer-readable storage medium, the various methods described above may be performed.

[0173] In addition, the computer device may also include (but not limited to) a data bus, an input / output (I / O) bus, a display, and input / output devices (eg, keyboard, mouse, speaker, etc.), etc.

[0174] The processor may communicate with external devices via an I / O bus via a wired or wireless network.

[0175] In one embodiment, the at least one computer executable instruction may also be compiled into or constitute a software product / computer program product, wherein one or more computer executable instructions are executed by a processor to perform the various functions and / or method steps in the embodiments described in the present technology.

[0176] In the embodiments provided in the present disclosure, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0177] It should be noted that in the present disclosure, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element limited by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0178] Although the embodiments disclosed in the present disclosure are as above, the above contents are only embodiments adopted for facilitating the understanding of the present disclosure and are not intended to limit the present disclosure. Any technician in the technical field to which the present disclosure belongs can make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in the present disclosure, but the scope of patent protection of the present disclosure shall still be subject to the scope defined in the attached claims.

Claims

1. An intelligent cleaning method based on historical images, characterized in that: include: Divide the target cleaning area into a number of grid units, wherein each grid unit stores a number of historical cleaning data of the corresponding sub-area, and each piece of historical cleaning data includes a ground image of the corresponding sub-area; Generating a target cleanliness of each grid unit according to historical cleaning data of each grid unit, wherein the target cleanliness is represented as a ground image; During the cleaning process, the current ground sampling image within the current moving unit is obtained; Locating a number of target grid units according to the coverage of the current ground sampling image, splicing or cropping the ground images of the target cleanliness of the target grid units, and obtaining a current target clean image with the same coverage as the current ground sampling image; By comparing the current ground sampling image with the current target cleaning image, a current cleanliness corresponding to the current ground sampling image is obtained; determining a current cleaning operation according to the current cleanliness; The historical cleaning data of each grid unit is updated using the ground image after the current cleaning operation is completed.

2. The method according to claim 1, characterized in that After dividing the target cleaning area into a grid unit consisting of a plurality of units, the method further includes: Use cleaning equipment to clean the coverage area of ​​each grid unit; Obtain the cleaned historical image of the area covered by each grid cell; Obtain cleaning intensity data and cleaning duration data for the area covered by each grid unit; The post-cleaning historical image, the cleaning intensity data, and the cleaning duration data are aggregated to obtain a piece of historical cleaning data for each grid unit.

3. The method according to claim 1, characterized in that The step of obtaining the current ground sampling image in the current mobile unit during the cleaning process includes: Using a collection device disposed in the cleaning device to continuously collect ground images at a preset collection frequency to obtain a ground image set; Sorting each ground image in the ground image set according to the order of shooting time to obtain a ground image sequence; Identifying overlapping areas of adjacent images in the ground image sequence; Overlapping areas of adjacent images in the ground image sequence are removed and then spliced ​​to obtain a ground sampling image.

4. The method according to claim 1, characterized in that: The method of locating a plurality of target grid units according to the coverage of the current ground sampling image, splicing or cropping the ground images of the target cleanliness of the plurality of target grid units, and obtaining a current target clean image having the same coverage as the current ground sampling image, includes: Acquire the position information of the area covered by the current ground sampling image in the target cleaning area by positioning; Locate the grid cells covered by the current ground sampling image in the target cleaning area according to the position information to obtain a grid cell set; Acquire a target cleanliness ground image of each grid cell in the grid cell set; The target cleanliness ground images of all grid cells are spliced ​​according to the positional relationship between the grid cells to obtain a spliced ​​target clean image; If the stitching target clean image is larger than the range of the current ground sampling image, the stitching target clean image is cropped to the same range as the current ground sampling image to obtain the current target clean image.

5. The method according to claim 1, characterized in that The step of comparing the current ground sampling image with the current target clean image to obtain the current cleanliness corresponding to the current ground sampling image includes: Obtaining the current cleanliness by comparing the pixel difference between the current ground sampling image and the current target cleaning image; Alternatively, characteristic information of the current ground sampling image and the current target cleaning image are calculated respectively, and the current cleanliness is obtained by comparing the characteristic information of the two images, wherein the characteristic information includes any one or a combination of characteristic values, characteristic vectors or characteristic sub-regions.

6. The method according to claim 1, characterized in that Determining the current cleaning operation according to the current cleanliness comprises: Determine whether the current cleanliness is less than a preset cleanliness threshold and whether the cleaning time of the floor area corresponding to the current cleanliness is less than a preset time threshold; If the floor cleanliness is less than a preset cleanliness threshold and the cleaning time of the floor area corresponding to the current cleanliness is less than a preset time threshold, the floor area corresponding to the current cleanliness is cleaned using a cleaning device; If the floor cleanliness is greater than or equal to a preset cleanliness threshold or the cleaning time of the floor within the moving unit range is greater than or equal to a preset time threshold, it is confirmed that the cleaning of the floor area corresponding to the current cleanliness is completed.

7. The method according to claim 1, characterized in that The method of generating a target cleanliness level of each grid unit according to the historical cleaning data of each grid unit includes: Using the ground images of several pieces of historical cleaning data of each grid unit, a target cleaning image of each grid unit is generated based on an analysis and prediction method; For each grid unit, the analysis and prediction method is used to analyze and predict the area of ​​the grid unit as a whole, and a target cleaning image of each grid unit is generated to obtain a target cleanliness of each grid unit; Alternatively, the area of ​​the grid unit is divided into several small areas, each small area is analyzed and predicted respectively, and then the analysis and prediction results of all small areas are combined to obtain the analysis and prediction results of the grid unit, and the target cleanliness of each grid unit is obtained; Alternatively, each pixel point in the area of ​​the grid unit is analyzed and predicted respectively, and then the analysis and prediction results of the grid unit are summarized to obtain the target cleanliness of each grid unit.

8. An intelligent cleaning device based on historical images, characterized in that: include: A unit division module is used to divide the target cleaning area into a grid unit composed of a plurality of units, wherein each grid unit stores a plurality of historical cleaning data of a corresponding sub-area, and each piece of historical cleaning data includes a ground image of the corresponding sub-area; An image acquisition module is used to generate a target cleanliness of each grid unit according to the historical cleaning data of each grid unit, wherein the target cleanliness is expressed as a ground image, and during the cleaning process, a current ground sampling image in the current moving unit is acquired, and a number of target grid units are located according to the coverage of the current ground sampling image, and the ground images of the target cleanliness of the several target grid units are spliced ​​or cropped to obtain a current target cleaning image with the same coverage as the current ground sampling image; A data calculation module, used for obtaining a current cleanliness level corresponding to the current ground sampling image by comparing the current ground sampling image with the current target clean image; The cleaning operation module is used to determine the current cleaning operation according to the current cleanliness, and update the historical cleaning data of each grid unit using the ground image after completing the current cleaning operation.

9. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images