Method for estimating position of cleaning machine

Abstract

A method of estimating a location of a cleaning machine includes performing mapping of an ambient environment with a smart module of the cleaning machine. While the cleaning machine is in use, a path of the cleaning machine is recorded. The cleaning machine is connected to the cloud computer. The cleaning machine shares data with the cloud computer. At least one of a position, a speed, an acceleration, an angular heading, a rotation rate, an acceleration rate, or a state of a combination of the foregoing of the cleaning machine is then estimated.

Description

Technical Field

[0001] The present disclosure relates generally to cleaning machines. In particular, the present disclosure relates to a method of identifying a location of a cleaning machine. Background Art

[0002] Industrial and commercial floors are cleaned regularly for aesthetic and hygienic purposes. There are many types of industrial and commercial floors, ranging from hard surfaces such as concrete, terrazzo, wood, etc. found in factories, schools, hospitals, etc., to soft surfaces such as carpet floors found in restaurants and offices. In order to properly clean and maintain these different floor surfaces, different types of floor cleaning equipment have been developed, such as vacuum cleaners, scrubbers, sweepers, and extractors.

[0003] Existing manual cleaning machines often lack features found in autonomous cleaning machines and their platforms. The lack of these features may increase the amount of time used to clean a surface, use more water and energy, and be less efficient overall with respect to cleaning material usage. The lack of these features also increases the difficulty of confirming whether cleaning activity has occurred.

[0004] The inventors have recognized that there is a need for an improved system that overcomes the above-mentioned disadvantages. Summary of the invention

[0005] One aspect relates to a method for estimating the location of a cleaning machine, the method comprising performing a mapping of the surrounding environment using an intelligent module of the cleaning machine. When the cleaning machine is in use, the path of the cleaning machine is recorded. The cleaning machine is connected to a cloud computer. The cleaning machine shares data with the cloud computer. Then, at least one of the state of the position, velocity, acceleration, angular heading, rotation rate, acceleration rate, or a combination of the above of the cleaning machine is estimated.

[0006] A second aspect relates to a method for cleaning using a cleaning machine, the method comprising using at least one of a sensor, an intelligent module, or a combination of the cleaning machine to detect at least one of a floor type of a floor, a degree of dirtiness of the floor, or a combination of the two. A cleaning setting is adjusted in response to at least one of the detected floor type, the detected degree of dirtiness, or a combination of the two. The cleaning machine is connected to a cloud computer. The cleaning machine shares data with the cloud computer. The data shared with the cloud computer includes data representing the detected floor type, the detected degree of dirtiness, or a combination of the two.

[0007] A third aspect relates to a method for estimating the position of a cleaning machine, the method comprising processing the information history of the cleaning machine using an intelligence module of the cleaning machine. In response to the processed information history, an estimate of the position of the cleaning machine is increased. A map of the cleaning area is created using the intelligence module.

[0008] A fourth aspect relates to a method of estimating the positioning of a cleaning machine adapted for manual operation, the method comprising:

[0009] - Installing the smart module on the cleaning machine, or providing the cleaning machine with a pre-installed smart module;

[0010] - when the cleaning machine is in use, recording the path of the cleaning machine in the surrounding environment where the cleaning machine is located;

[0011] - performing mapping of the surrounding environment using the installed intelligent modules;

[0012] - Using the smart module to store the collected data;

[0013] - Connecting the cleaning machine and / or the smart module to a cloud computer;

[0014] - sharing the collected data from the cleaning machine and / or the smart module with a cloud computer; and

[0015] - Using the smart module or using the cloud computer, estimate at least the current position of the cleaning machine, wherein the current position includes at least one of the status of the cleaning machine's position, speed, acceleration, angular heading, rotation rate, acceleration rate, or a combination thereof.

[0016] It should be noted that embodiments and features described in the context of one aspect of the invention may also be applicable to other aspects of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A flow chart showing the steps of a method of estimating the positioning of a cleaning machine.

[0018] Figure 2 Another flow chart showing additional steps of the method of estimating the positioning of a cleaning machine.

[0019] Figure 3 Another flow chart showing additional steps of the method of estimating the positioning of a cleaning machine. DETAILED DESCRIPTION

[0020] The present disclosure provides an opportunity to utilize the concept of autonomy in manual cleaning machines, such as manual cleaning machines operating at a larger scale than autonomous cleaning machines can operate. The present disclosure also helps to solve the problem at scale, thereby providing additional benefits in autonomy for higher technology platforms, reducing costs and improving the robustness of the solution.

[0021] The present disclosure provides a smart module for use with a cleaning machine. In embodiments, the smart module can be an add-on module that can be added to an existing cleaning machine (e.g., a manual cleaning machine) to bring, for example, artificial intelligence ("AI") functionality and other features to any type of cleaning machine.

[0022] The smart module can be installed on the external surface or internal surface of the cleaning machine. The smart module can communicate with the control unit of the cleaning machine. The smart module can communicate with the cloud server.

[0023] In an embodiment, the smart module can be connected to the control module of the cleaning machine via a wired connection, a wireless connection, or a combination of the two. In addition, the smart module can include, be combined with, or be used in conjunction with an inertial measurement unit ("IMU"), a two-dimensional and / or three-dimensional camera, a light detection and ranging ("LIDAR") device, a device configured to provide a range input (e.g., a range meter), or a combination of the above.

[0024] In an embodiment, the first configuration of the smart module includes the computing power of the first group of sensors and the first amount. For example, the first configuration of the smart module may include one or more IMUs, monocular cameras, wheel distance measuring devices, WiFi and / or Bluetooth. In addition, one or more sensor readings from the first group of sensors can be combined or fused together using filters, such as extended Kalman filters, marginalized particle filters, or a combination of the two, to estimate the position, speed, acceleration and corresponding angular heading, rotation rate, acceleration rate, or a combination of the above of the cleaning machine.

[0025] With such a first set of sensors, one or more estimated states may contain a high degree of error.In embodiments, a neural network may be used to map a set of inputs (from cleaning equipment, users, or a combination of both) for multiple cleaning cycles to a corrected output.

[0026] In another embodiment, a second cleaning machine (eg, different in size from the first set of cleaning machines associated with the first set of sensors) can include a second configuration of a second intelligence module. The second intelligence module enables increased computing capabilities when combined with one or more sensors.

[0027] In another embodiment, the second cleaning machine can transmit data to the first cleaning machine. The first cleaning machine uses a neural network to map the data from the first cleaning machine and the second cleaning machine into a more accurate position estimate.

[0028] In another embodiment, the first cleaning machine, the second cleaning machine, or a combination of the two transmits data to a cloud computer. The cloud computer uses a neural network to map the data from the first cleaning machine, the second cleaning machine, or a combination of the two into a more accurate position estimate. The cloud computer sends the more accurate position back to the first cleaning machine, the second cleaning machine, or a combination of the two.

[0029] In an embodiment, the location is a historical location showing a path traveled by the cleaning machine.

[0030] In an embodiment, the second set of sensors used by the second intelligent module may include an image sensor, such as a two-dimensional camera, a three-dimensional camera, or a combination of the two. For example, a two-dimensional camera or a three-dimensional camera can create a more detailed map of the environment including walls, floors, objects, etc.

[0031] In embodiments, one or more cameras operatively connected to a second cleaning machine, a second smart module, or a combination of both can extract landmarks from an area (e.g., a cleaning area) to determine in which room the cleaning machine is located and apply a set of preselected cleaning settings, automatically calculated settings, or a combination of both.

[0032] Additionally, a second intelligence module can use artificial intelligence to analyze images and annotate room types to add context to the resulting map, such as terms like “corridor” or “hallway.” This classification provides context, allowing the cleaning scheduling algorithm to determine how often to clean a space and allows the system operator to indicate “clean the hallway” without programming or changing settings.

[0033] In an embodiment, a group of sensors used by the smart module may include an image sensor, such as a two-dimensional camera, a three-dimensional camera, or a combination of the two. A neural network can be used to create segmentation and annotation of an image, separating the floor from a wall, a person, a mooring pile, a trash can, etc., thereby allowing the cleaning machine to generate a safety warning or a safety control signal to prevent a collision. In addition, annotating the image can prevent accidental damage, such as cleaning a carpet area with water or vacuuming a can of soda. Creating segmentation, annotation, or a combination of the two can also be done in conjunction with the depth information provided by a three-dimensional camera or a laser radar. For example, a two-dimensional image can be superimposed with depth information so that the annotated object can be perceived three-dimensionally. Creating segmentation, annotation, or a combination of the two can also be used so that the cleaning machine automatically limits (e.g., prevents exceeding a maximum threshold) the speed (and / or acceleration) of the cleaning machine when the cleaning machine approaches a specific object (e.g., a person, a pet, or another annotated object).

[0034] In an embodiment, the second set of sensors may include a higher level of quality and accuracy than the other sensors. In such an embodiment, the second set of sensors with higher quality and higher accuracy can achieve sufficiently accurate position tracking of the second cleaning machine to indicate whether the second cleaning machine has cleaned the area or sub-area more than once, thereby saving time, water, energy, materials, costs, or a combination of the above. In an embodiment, the second smart module can provide an operator or user with an indication in the form of an alarm to the operator, feedback to the operator (e.g., supervisor) for cleaning machine training; as an indication of a driver assist function that causes the cleaning machine to change direction slightly; or a combination of both.

[0035] In an embodiment, the first intelligent module, the second intelligent module, or a combination of the two may perform one or more of the following steps. The intelligent module may perform mapping of the surrounding environment. The path of the cleaning machine may be recorded when the cleaning machine is in use. The presence of an object (e.g., non-human or human) may be detected. Collisions with objects (e.g., automatic stops) may be avoided. The intelligent module may be used to assist the operator in maximizing the amount of cleaning coverage of the floor. The intelligent module may be used to assist the operator in minimizing the amount of overlap of the cleaning area. The type of floor in the cleaning area or the cleaned area may be detected. In an embodiment, the type of floor may be detected by a sensor, an intelligent module, or a combination of the two. The degree of dirtiness in the cleaning area or the cleaned area may be detected. In an embodiment, the degree of dirtiness may be detected by a sensor, an intelligent module, or a combination of the two.

[0036] Cleaning settings can be automatically adjusted. In embodiments, cleaning settings can be automatically adjusted in response to a detected floor type, a detected level of soiling, or a combination of both. A connection can be made to a cloud (e.g., a cloud-based storage device and / or operating system) and data can be uploaded, data can be shared, or a combination of both, wherein the set of data represents the level of soiling, the floor type, or a combination of both.

[0037] The cleaning path may have errors due to sensor readings and processing by the computer. Data collected at different times on different cleaning paths can be combined to increase the understanding of the actual path and eliminate errors.

[0038] An improvement of the present disclosure is the modular nature of adding intelligence to the cleaning machine through intelligent modules. Existing autonomous platforms can perform some of the above actions, but these actions usually need to be fully integrated into the system of the cleaning machine. The present disclosure provides an additional intelligent module that can be added to the existing cleaning machine with minimal setup and calibration. In addition, compared with a fully autonomous system for a cleaning machine, existing hardware (such as off-the-shelf two-dimensional cameras and three-dimensional cameras) and computer platforms can be used to reduce the cost of the cleaning machine. In another embodiment, sensing and detection modes can also be incorporated into the cleaning machine of the present disclosure.

[0039] Deploying the present disclosure to multiple manual cleaning machines can result in a larger fleet than developing only automated cleaning machines. Manual fleets allow for large amounts of data to be collected for developing artificial intelligence and algorithms. Since the results of failures may not be critical to the function of the cleaning machine, the maturity of deploying additional algorithms may be lower than for automated machines. In such an embodiment, the quality and robustness of the final solution for both manual and automated cleaning can be improved.

[0040] The present disclosure allows the collection of data in the form of, for example, usage maps of where the cleaning machine is used with estimated efficiency and unique areas of cleaning; cleanliness data derived from actual sensor readings to demonstrate cleanliness; usage reports showing how well the operator is operating the cleaning machine (e.g., number of stops, percentage of overlap, time to completion, or a combination of the above). Adding this data available to the operator (e.g., product customer) can increase the value of the cleaning machine.

[0041] Additionally, operator assistance mechanisms may be used in conjunction with the cleaning machine and / or the smart module to help avoid damage to the equipment and surrounding environment (eg, a facility or objects thereof) and to help maximize the efficiency of the cleaning machine.

[0042] In another embodiment, data collected or generated according to the present disclosure can provide insights into the use of cleaning machines and operator efficiency. In this and other embodiments, the collected data can allow operators (e.g., customers) to make informed decisions about how to make the most of their cleaning machines and can use data-driven methods to prove the degree of cleaning. The present disclosure provides operator assistance technology that can save customers money by ensuring that cleaning machines and facilities are not damaged and provide the most effective cleaning methods, routes and / or strategies available.

[0043] The above detailed description includes reference to the accompanying drawings, which form a part of the detailed description. The accompanying drawings show specific embodiments in which the present invention can be practiced by way of illustration. These embodiments are also referred to as "embodiments" herein. These embodiments may include elements other than those elements shown or described. However, the inventors also contemplate embodiments in which only those elements shown or described are provided. In addition, the inventors also contemplate embodiments using any combination or arrangement of the elements shown or described (or one or more aspects thereof), whether for a specific embodiment (or one or more aspects thereof), or for other embodiments shown or described herein (or one or more aspects thereof).

[0044] In an embodiment, the first intelligent module, the second intelligent module, or a combination of the two can perform one or more of the following steps. The intelligent module can perform mapping of the surrounding environment (e.g., creating a map). The map can include the location of walls, doors, trash cans, and other immovable objects. The map also includes an estimated state history of the cleaning machine position, rate, acceleration, or a combination of the above. In addition or alternatively, a map can be created based on the outline of a building. In such an embodiment, the outline of the building can be sensed, input by a user, or a combination of the two.

[0045] In another embodiment, over time, as the first cleaning machine travels through the same area multiple times, the created map may be updated, enhanced, adjusted, or a combination thereof.

[0046] In another embodiment, the created map may be updated or enhanced over time as the second cleaning machine drives through the same area multiple times.

[0047] In an embodiment, the smart module may include a sensor for estimating the motion of the machine, such as an IMU, a wheel odometer, an optical flow, or a combination of the above. Motion estimation may be integrated, processed, or otherwise operated in a filter to estimate one or more states of the cleaning machine. The state of the cleaning machine may include a historical position. The cleaning machine may pass through an area many times over many days, and such information history may be processed using an artificial intelligence algorithm or a similar algorithm to increase the estimation of the traveled position. The position of the machine may be used to show the area that has been cleaned.

[0048] In another embodiment, when the cleaning machine passes through an area multiple times in multiple days, the second cleaning machine with the smart module may include a sensor for estimating the movement of the machine, such as an IMU, a wheel distance meter, an optical flow, or a combination of the above. The information history of the first cleaning machine, the second cleaning machine, or a combination of the two can be combined using an artificial intelligence algorithm or a similar algorithm to increase the estimation of the traveled position. The position can be used to show the area that has been cleaned.

[0049] Figure 1 A flow chart showing the steps of a method 100 for estimating the positioning of a cleaning machine. In particular, Figure 1 Steps 102 to 114 of method 100 are shown. In an embodiment, method 100 may include steps 102 to 170.

[0050] In an embodiment, regarding Figures 1 to 3 The described cleaning machine may include at least one of a first cleaning machine, a second cleaning machine, or a combination thereof. The first cleaning machine may include a first smart module and a first group of sensors having a first configuration. The second smart module may include a second group of sensors and a second smart module having a second configuration. The second group of sensors may include a two-dimensional camera, a three-dimensional camera, or a combination thereof. At least one of the first smart unit, the second smart unit, or a combination thereof may include at least one of an inertial measurement unit, a two-dimensional camera, a three-dimensional camera, a light detection and ranging device, a range meter, or a combination thereof.

[0051] Step 102 may include utilizing the smart module of the cleaning machine to perform mapping of the surrounding environment. Step 104 may include recording the path of the cleaning machine when the cleaning machine is in use. Step 106 may include connecting the cleaning machine to a cloud computer. Step 108 may include sharing data from the cleaning machine with the cloud computer. Step 110 may include estimating at least one of the state of the position, speed, acceleration, angular heading, rotation rate, acceleration rate, or a combination of the above-mentioned of the cleaning machine. Step 112 may include utilizing a filter to combine one or more sensor readings from a sensor, wherein the filter includes a Kalman filter, a marginalized particle filter, or a combination of the two. Step 114 may include utilizing a neural network to map a set of inputs of the cleaning machine about multiple cleaning cycles to a corrected output, wherein the set of inputs is from a cleaning device, a user, or a combination of the two.

[0052] Figure 2 Another flow chart of steps 116 to 138 of the method 100 of estimating the location of a cleaning machine is shown. In an embodiment, the method 100 may further include at least one of steps 116 to 138 or a combination thereof.

[0053] Step 116 may include transferring data from the second cleaning machine to the first cleaning machine.

[0054] Step 118 may include utilizing a neural network to map data from the first cleaning machine and the second cleaning machine to an estimated position of the first cleaning machine.

[0055] Step 120 may include at least one of step 122, step 124, or a combination of the above-mentioned. Step 122 may include mapping the data from the first cleaning machine, the second cleaning machine, or a combination of the two to an estimated position. In an embodiment, the position may be a historical position showing the path of the cleaning machine. Step 124 may include sending the estimated position back to the first cleaning machine, the second cleaning machine, or a combination of the two.

[0056] Step 126 may include at least one of step 128, step 130, step 132, or a combination thereof. Step 128 may include extracting landmarks using one or more cameras operatively connected to the cleaning machine, the smart module, or a combination of both. Step 130 may include determining in response to extracting the landmarks which room the cleaning machine is located. Step 132 may include applying a set of preselected cleaning settings, automatically calculated settings, or a combination of both.

[0057] Step 134 may include at least one of step 136, step 138, or a combination of the two. Step 136 may include analyzing the image of the surrounding environment to annotate the room type using artificial intelligence. Step 138 may include determining the frequency of cleaning the space in response to the annotated room type.

[0058] Figure 3 Another flow chart of steps 140 to 170 of the method 100 for estimating the location of a cleaning machine is shown. In an embodiment, the method 100 may further include at least one of steps 140 to 170 or a combination thereof.

[0059] Step 140 may include at least one of step 142, step 144, step 146, step 148, or a combination thereof. Step 142 may include creating a segmentation and annotation of the image. Step 144 may include generating a warning or safety control signal in response to the segmentation and annotation of the image. Step 146 may include overlaying the two-dimensional image with depth information. Step 148 may include limiting the speed of the cleaning machine when the cleaning machine approaches a specific object in response to creating the annotation of the image.

[0060] Step 150 may include at least one of step 152, step 154, step 156, or a combination of the foregoing. Step 152 may include using at least one of a sensor, an intelligent module, or a combination of the cleaning machine to detect at least one of the floor type of the floor, the degree of dirtiness of the floor, or a combination of the two. Step 154 ​​may include adjusting the cleaning settings in response to at least one of the detected floor type, the detected degree of dirtiness, or a combination of the two. Step 156 may include sharing data from the cleaning machine with a cloud computer. In an embodiment, the data may include data representing the detected floor type, the detected degree of dirtiness, or a combination of the two.

[0061] Step 158 may include at least one of step 160, step 162, step 164, step 166, or a combination thereof. Step 160 may include estimating the movement of the cleaning machine using a sensor. Step 162 may include processing the information history of the cleaning machine using an intelligent module of the cleaning machine. Step 164 may include increasing an estimate of the location of the cleaning machine in response to the processed information history. Step 166 may include creating a map of the cleaning area using an intelligent module.

[0062] Step 168 may include: estimating at least one of the position, velocity, acceleration and corresponding angular heading, rotation rate, acceleration rate, or a combination of the cleaning machine using an extended Kalman filter, a marginalized particle filter, or a combination of both; step 170; or a combination of both. Step 170 may include combining the sensor readings using an extended Kalman filter, a marginalized particle filter, or a combination of both.

[0063] Various Descriptions and Examples

[0064] Embodiment 1 may include or use subject matter such as a method for estimating the positioning of a cleaning machine, the method comprising: utilizing an intelligent module of the cleaning machine to perform mapping of the surrounding environment; recording the path of the cleaning machine while the cleaning machine is in use; connecting the cleaning machine to a cloud computer; sharing data from the cleaning machine with the cloud computer; and estimating at least one of the states of the cleaning machine's position, velocity, acceleration, angular heading, rotation rate, acceleration rate, or a combination thereof.

[0065] Embodiment 2 may include the subject matter of embodiment 1 or may optionally be combined with the subject matter of embodiment 1 to optionally include combining one or more sensor readings from the sensor using a filter, wherein the filter includes a Kalman filter, a marginalized particle filter, or a combination of both.

[0066] Embodiment 3 may include the subject matter of one or any combination of Embodiment 1 or Embodiment 2 or may optionally be combined with the subject matter of one or any combination of Embodiment 1 or Embodiment 2 to optionally include utilizing a neural network to map a set of inputs of a cleaning machine regarding multiple cleaning cycles to corrected outputs, wherein the set of inputs comes from a cleaning device, a user, or a combination of both.

[0067] Embodiment 4 may include the subject matter of one or any combination of embodiments 1 to 3 or may be optionally combined with the subject matter of one or any combination of embodiments 1 to 3 to optionally include: a first cleaning machine, which includes a first smart module with a first configuration and a first group of sensors; a second cleaning machine, which includes a second group of sensors and a second smart module with a second configuration; and wherein the second cleaning machine includes a second group of sensors, which includes a two-dimensional camera, a three-dimensional camera, or a combination of both.

[0068] Embodiment 5 may include or may optionally be combined with the subject matter of one or any combination of Embodiments 1 to 4 to optionally include transmitting data from the second cleaning machine to the first cleaning machine.

[0069] Example 6 may include the subject matter of one or any combination of Examples 1 to 5 or may optionally be combined with the subject matter of one or any combination of Examples 1 to 5 to optionally include utilizing a neural network to map data from the first cleaning machine and the second cleaning machine to an estimated position of the first cleaning machine.

[0070] Embodiment 7 may include or may optionally be combined with the subject matter of one or any combination of Embodiments 1 to 6 to optionally include mapping data from a first cleaning machine, a second cleaning machine, or a combination of both into an estimated location, wherein the location is a historical location showing the path traveled by the cleaning machine; and sending the estimated location back to the first cleaning machine, the second cleaning machine, or a combination of both.

[0071] Embodiment 8 may include or may optionally be combined with the subject matter of one or any combination of Embodiments 1 to 7 to optionally include extracting landmarks using one or more cameras operatively connected to a cleaning machine, a smart module, or a combination of both; determining in response to the extracted landmarks which room the cleaning machine is located; and applying a set of preselected cleaning settings, automatically calculated settings, or a combination of both.

[0072] Embodiment 9 may include the subject matter of one or any combination of embodiments 1 to 8 or may be optionally combined with the subject matter of one or any combination of embodiments 1 to 8 to optionally include utilizing artificial intelligence to analyze images of the surrounding environment to label room types; and determining the frequency of cleaning the space in response to the labeled room types.

[0073] Embodiment 10 may include the subject matter of one or any combination of Embodiments 1 to 9 or may be optionally combined with the subject matter of one or any combination of Embodiments 1 to 9 to optionally include creating segmentation and annotation of the image; and generating a warning or safety control signal in response to the segmentation and annotation of the image.

[0074] Embodiment 11 may include the subject matter of one or any combination of Embodiments 1 to 10 or may be optionally combined with the subject matter of one or any combination of Embodiments 1 to 10 to optionally include superimposing the two-dimensional image with depth information.

[0075] Embodiment 12 may include or may optionally be combined with the subject matter of one or any combination of Embodiments 1 to 11 to optionally include limiting the speed of the cleaning machine when the cleaning machine approaches a specific object in response to creating annotations to the image.

[0076] Embodiment 13 may include the subject matter of one or any combination of embodiments 1 to 12 or may be optionally combined with the subject matter of one or any combination of embodiments 1 to 12 to optionally include that the intelligent unit includes at least one of an inertial measurement unit, a two-dimensional camera, a three-dimensional camera, a light detection and ranging device, a rangefinder, or a combination of the above.

[0077] Embodiment 14 may include the subject matter of one or any combination of embodiments 1 to 13 or may be optionally combined with the subject matter of one or any combination of embodiments 1 to 13 to optionally include a method of cleaning using a cleaning machine, the method comprising: using at least one of the sensors, smart modules, or a combination of the cleaning machine to detect at least one of the floor type of the floor, the degree of dirtiness of the floor, or a combination of the two; adjusting the cleaning settings in response to at least one of the detected floor type, the detected degree of dirtiness, or a combination of the two; connecting the cleaning machine to a cloud computer; and sharing data from the cleaning machine with the cloud computer, wherein the data includes data representing the detected floor type, the detected degree of dirtiness, or a combination of the two.

[0078] Embodiment 15 may include the subject matter of one or any combination of embodiments 1 to 14 or may be optionally combined with the subject matter of one or any combination of embodiments 1 to 14 to optionally include a method for estimating the position of a cleaning machine, the method comprising: estimating the movement of the cleaning machine using a sensor; processing the information history of the cleaning machine using an intelligent module of the cleaning machine; increasing the estimate of the location where the cleaning machine has traveled in response to the processed information history; and creating a map of the cleaning area using the intelligent module.

[0079] Embodiment 16 may include the subject matter of one or any combination of embodiments 1 to 15 or may be optionally combined with the subject matter of one or any combination of embodiments 1 to 15 to optionally include utilizing an extended Kalman filter, a marginalized particle filter, or a combination of both to estimate the state of the cleaning machine's position, velocity, acceleration, and corresponding angular heading, rotation rate, acceleration rate, or a combination thereof.

[0080] Example 17 may include or may optionally be combined with the subject matter of one or any combination of Examples 1 to 16, optionally including combining sensor readings using an extended Kalman filter, a marginalized particle filter, or a combination of both.

[0081] Each of these non-limiting embodiments may exist independently, or may be arranged or combined in various ways with one or more of the other embodiments.

[0082] The above detailed description includes reference to the accompanying drawings, which form a part of the detailed description. The accompanying drawings show specific embodiments in which the present invention can be practiced by way of illustration. These embodiments are also referred to as "embodiments" herein. These embodiments may include elements other than those elements shown or described. However, the inventors also contemplate embodiments in which only those elements shown or described are provided. In addition, the inventors also contemplate embodiments using any combination or arrangement of the elements shown or described (or one or more aspects thereof), whether for a specific embodiment (or one or more aspects thereof), or for other embodiments shown or described herein (or one or more aspects thereof).

[0083] In one or more embodiments, the cleaning machine or the smart module includes one or more sensors, wherein the method further includes utilizing a filter to combine one or more sensor readings from the one or more sensors, wherein the filter includes a Kalman filter, a marginalized particle filter, or a combination of both.

[0084] In one or more embodiments, the method further comprises:

[0085] - Using a neural network implemented in the smart module and / or the cloud computer, a set of inputs of the cleaning machine regarding multiple cleaning cycles is mapped into a corrected output, wherein the set of inputs comes from the cleaning equipment and / or the smart module, the user, or a combination of the above.

[0086] In one or more embodiments, the method includes:

[0087] - using a first cleaning machine comprising:

[0088] i) a first smart module having a first configuration; and

[0089] ii) a first set of sensors; and

[0090] - using a second cleaning machine comprising:

[0091] i) a second set of sensors; and

[0092] ii) a second smart module having a second configuration; and

[0093] The second cleaning machine is used to collect data, and the data collected by the second cleaning machine is used to estimate the position of the first cleaning machine.

[0094] In one or more embodiments, the second set of sensors includes a two-dimensional camera, a three-dimensional camera, or a combination of both.

[0095] In one or more embodiments, the method further comprises:

[0096] - Transmitting the collected data from the second cleaning machine to the first cleaning machine and / or a cloud computer.

[0097] In one or more embodiments, the method further comprises:

[0098] In one or more embodiments, the method further comprises:

[0099] - mapping, using the cloud computer, data collected from a first cleaning machine, a second cleaning machine, or a combination of both, into a location estimate for the first cleaning machine, wherein the location is a location history showing a path traveled by the first cleaning machine; and

[0100] - sending the position estimate back to the first cleaning machine, the second cleaning machine or both the first cleaning machine and the second cleaning machine.

[0101] In one or more embodiments, the method further comprises:

[0102] - Extracting landmarks using one or more cameras connected to the cleaning machine and / or the smart module at the worksite;

[0103] - determining in response to said extracted landmarks, in which room the cleaning machine is located; and

[0104] - In response to the determination, applying a set of predetermined cleaning settings, automatically calculated cleaning settings, or a combination of both.

[0105] In one or more embodiments, the method further comprises:

[0106] - using the smart module and / or the cloud computer, preferably using artificial intelligence, to analyze the received surrounding environment image to label / identify the room type; and

[0107] -Determining how often to clean the space in response to the tagged / identified room type.

[0108] In one or more embodiments, the method further comprises:

[0109] - Using the smart module and / or the cloud computer to create image segmentation and annotation; and

[0110] - In response to the segmentation and annotation of the image, a warning or safety control signal is generated.

[0111] In one or more embodiments, the process of creating a segmentation and annotation of an image includes overlaying a two-dimensional image with depth information.

[0112] In one or more embodiments, the method further comprises:

[0113] - In response to the segmentation and labeling operations on the image, limiting the maximum speed of the cleaning machine when the cleaning machine approaches a specific object.

[0114] In one or more embodiments, the smart unit includes a sensor selected from at least one of an inertial measurement unit, a two-dimensional camera, a three-dimensional camera, a light detection and ranging device, a rangefinder, or a combination thereof.

[0115] In one or more embodiments, the method further comprises:

[0116] - using at least one of a sensor, an intelligent module, or a combination of the cleaning machine to detect at least one of a floor type of the floor, a degree of dirtiness of the floor, or a combination of the two;

[0117] - adjusting cleaning settings in response to at least one of a detected floor type, a detected level of soiling, or a combination of both; and

[0118] - Sharing collected data from the cleaning machine with a cloud computer, wherein the data includes data representative of the detected floor type, the detected soiling level, or a combination of both.

[0119] In one or more embodiments, the method further comprises:

[0120] - Use sensors to estimate the movement of cleaning machines;

[0121] -Use the intelligent module of the cleaning machine to process the information history of the cleaning machine;

[0122] - improving or correcting estimates of locations travelled by the cleaning machine in response to the processed information history; and

[0123] -Use smart modules to create maps of cleaning areas.

[0124] In one or more embodiments, the method further comprises:

[0125] - Using an extended Kalman filter, a marginalized particle filter, or a combination of both, estimate at least the current position of the cleaning machine, including at least the state of the cleaning machine's position, velocity, acceleration and corresponding angular heading, rotation rate, acceleration rate, or a combination thereof.

[0126] In one or more embodiments, the process of estimating the state of the cleaning machine's current position, location, velocity, acceleration, and corresponding angular heading, rotation rate, acceleration rate, or a combination thereof includes combining sensor readings with an extended Kalman filter, a marginalized particle filter, or a combination of both.

[0127] In this document, the use of the terms "a" or "an" is the same as the common use in patent documents, including one or more, and is independent of any other instances or uses of "at least one" or "one or more". In this document, unless otherwise stated, the term "or" is used to refer to a non-exclusive or, such as "A or B" includes "A but not B", "B but not A" and "A and B". In this document, the terms "including" and "wherein" are used as the plain English equivalents of the corresponding terms "comprising" and "wherein". In addition, in the following claims, the terms "including" and "comprising" are open-ended, that is, a system, device, article, composition, formula or process that includes elements other than the elements listed after the term in the claim is still deemed to fall within the scope of the claim. In addition, in the following claims, the terms "first", "second", and "third", etc. are used only as labels and are not intended to impose numerical requirements on their objects.

[0128] The above description is intended to be illustrative, not limiting. For example, the above embodiments (or one or more aspects thereof) may be used in combination with each other. Other embodiments may also be used by a person of ordinary skill in the art after reading the above description. The abstract is provided to comply with 37 CFR § 1.72 (b) so that the reader can quickly determine the nature of the technical disclosure. The premise of submitting the abstract is that the abstract is not used to interpret or limit the scope or meaning of the claims. In addition, in the above-mentioned specific embodiments, various features may be grouped together to simplify the disclosure. This should not be interpreted as not requiring that the disclosed features are essential to any claim. On the contrary, the creative subject matter may not lie in all the features of a particular disclosed embodiment. Therefore, the following claims are incorporated into the specific embodiments as examples or embodiments, wherein each claim exists independently as a separate embodiment, and it is considered that such embodiments can be combined with each other in various combinations or arrangements. The scope of the present invention should be determined with reference to the attached claims and the full range of equivalents enjoyed by these claims.

Claims

1. A method for estimating the position of a cleaning machine, the cleaning machine being adapted for manual operation, the method comprising: - Installing the smart module on the cleaning machine, or providing the cleaning machine with a pre-installed smart module; - recording the path of the cleaning machine in the surroundings of the cleaning machine while the cleaning machine is in use; - using the installed intelligent module to perform mapping of the surrounding environment; - Using the smart module to store the collected data; - Connecting the cleaning machine and / or the smart module to a cloud computer; - Sharing the collected data from the cleaning machine and / or the smart module with the cloud computer; as well as - Using the smart module or the cloud computer, estimate at least the current position of the cleaning machine, wherein the current position includes at least one of the status of the position, speed, acceleration, angular heading, rotation rate, acceleration rate, or a combination thereof of the cleaning machine.

2. The method according to claim 1, wherein: The cleaning machine or the smart module includes one or more sensors, wherein the method further includes combining one or more sensor readings from the one or more sensors using a filter, wherein the filter includes a Kalman filter, a marginalized particle filter, or a combination of both.

3. The method according to any one of claims 1 to 2, further comprising: - Using a neural network implemented in the smart module and / or the cloud computer, a set of inputs of the cleaning machine regarding multiple cleaning cycles is mapped into a corrected output, wherein the set of inputs comes from the cleaning equipment and / or the smart module, the user, or a combination of the above.

4. The method according to any one of claims 1 to 3, wherein: The method comprises: - using a first cleaning machine, said first cleaning machine comprising: i) a first smart module having a first configuration; and ii) a first set of sensors; and - using a second cleaning machine, said second cleaning machine comprising: i) a second set of sensors; and ii) a second smart module having a second configuration; and The second cleaning machine is used to collect data, and the data collected by the second cleaning machine is used to estimate the position of the first cleaning machine.

5. The method according to claim 4, wherein: The second set of sensors includes a two-dimensional camera, a three-dimensional camera, or a combination of the two.

6. The method according to any one of claims 4 to 5, further comprising transmitting the collected data from the second cleaning machine to the first cleaning machine and / or the cloud computer.

7. The method according to any one of claims 4 to 6, further comprising: - Mapping the data collected from the first cleaning machine and the second cleaning machine into an estimated position of the first cleaning machine using a neural network implemented in the smart module and / or the cloud computer.

8. The method according to any one of claims 4 to 7, further comprising: - using the cloud computer to map data collected from the first cleaning machine, from the second cleaning machine, or from a combination of the first cleaning machine and the second cleaning machine into an estimated location of the first cleaning machine, wherein the location is a historical location showing a path traveled by the first cleaning machine; and - sending the estimated position back to the first cleaning machine, to the second cleaning machine, or to both the first cleaning machine and the second cleaning machine.

9. The method according to any one of claims 1 to 8, further comprising: - extracting landmarks using one or more cameras operatively connected to the cleaning machine and / or the smart module; - determining in response to the extracted landmarks, in which room the cleaning machine is located; as well as - In response to the determination, applying a set of predetermined cleaning settings, automatically calculated cleaning settings, or a combination of both.

10. The method according to any one of claims 1 to 9, further comprising: - Analyzing the received image of the surrounding environment using the smart module and / or the cloud computer to mark / identify the room type. Preferably, analyzing the received image of the surrounding environment using the smart module and / or the cloud computer and applying artificial intelligence to mark / identify the room type. as well as -Determining how often to clean the space in response to the tagged / identified room type.

11. The method according to any one of claims 1 to 10, further comprising: - Using the smart module and / or the cloud computer to create image segmentation and annotation; as well as - In response to said segmentation and labeling of the image, generating a warning or safety control signal.

12. The method according to claim 11, wherein: The process of creating a segmentation and annotation of an image involves overlaying a two-dimensional image with depth information.

13. The method according to any one of claims 11 to 12, further comprising: - In response to the segmentation and labeling operations on the image, limiting the maximum speed of the cleaning machine when the cleaning machine approaches a specific object.

14. The method according to any one of claims 1 to 13, wherein: The smart unit includes a sensor selected from at least one of the following: an inertial measurement unit, a two-dimensional camera, a three-dimensional camera, a light detection and ranging device, a rangefinder, or a combination thereof.

15. The method according to any one of claims 1 to 14, further comprising: - using at least one of a sensor, an intelligent module, or a combination of a sensor and an intelligent module of the cleaning machine to detect at least one of a floor type of the floor, a degree of dirtiness of the floor, or a combination of the two; - adjusting cleaning settings in response to at least one of a detected floor type, a detected level of soiling, or a combination of both; as well as - sharing the collected data from the cleaning machines with the cloud computer, The data includes data representing the detected floor type, the detected degree of dirtiness, or a combination of the two.

16. The method according to any one of claims 1 to 15, further comprising: - using sensors to estimate the movement of the cleaning machine; - using the intelligent module of the cleaning machine to process the information history of the cleaning machine; - improving or correcting an estimate of the locations travelled by the cleaning machine in response to the processed information history; as well as -Use the smart module to create a map of the cleaning area.

17. The method according to claim 16, further comprising: - Using an extended Kalman filter, a marginalized particle filter, or a combination of the two, to estimate at least the current position of the cleaning machine, wherein the current position includes at least the state of the position, velocity, acceleration and corresponding angular heading, rotation rate, acceleration rate, or a combination of the above of the cleaning machine.

18. The method according to claim 17, wherein: The process of estimating the current position of the cleaning machine, that is, the process of estimating the position, speed, acceleration and corresponding angular heading, rotation rate, acceleration rate, or a combination thereof of the cleaning machine, includes: The sensor readings are combined using an extended Kalman filter, a marginalized particle filter, or a combination of both.

Images