Augmented reality cleaning system

Abstract

A system configured to track an area that has been cleaned with a cleaning appliance, the system comprising: a position determination module; a processor; and a display component, wherein: the position determination module is configured to determine position information indicative of a position of a portion of a surface cleaned by the cleaning appliance at a given time; generating a position data signal for encoding the position information; and the processor is configured to receive the position data signal from the position determination module and determine position information based on the position data signal; and generate instructions configured to cause the display component to display a visual representation of the position information.

Description

Technical Field

[0001] The present invention relates to a system for tracking an area that has been cleaned using a cleaning tool based on position information indicating the location of the cleaning tool at a given time. Equivalent methods and computer-implemented methods are also provided. Background Art

[0002] Currently, when a user is cleaning an area, it is often difficult to keep track of which areas have been cleaned during the cleaning process. For example, when vacuum cleaning, there is no reliable way to show the user which areas of the floor have and have not been cleaned. This is partly because, in most cases, dust particles are so small that they are difficult (if not impossible) to detect with the naked eye. A vacuum cleaner can leave streaks on a plush carpet to indicate where the vacuum has been, but this is unreliable and the problem is exacerbated in areas with hard or smooth surfaces.

[0003] This difficulty in tracking the area that has been cleaned often results in duplication of effort by the user and inefficiency of the automated system. It is therefore desirable to provide an improved means of tracking the area that has been cleaned using a cleaning implement. Summary of the invention

[0004] In general, the present invention relates to a method and system for tracking an area that has been cleaned using a cleaning appliance based on position information indicating the position of a portion of a surface cleaned by the cleaning appliance at a given time. In particular, the system is configured to generate a visual representation of the position information to a user.

[0005] More specifically, a first aspect of the present invention provides a system configured to track an area that has been cleaned using a cleaning tool, the system comprising: a position determination module; a processor; and a display component, wherein: the position determination module is configured to determine position information indicating the position of a portion of a surface cleaned by a cleaning tool at a given time; and generate a position data signal encoding the position information; and the processor is configured to receive the position data signal from the position determination module and determine the position information based on the position data signal; and generate instructions configured to cause the display component to display a visual representation of the position information.

[0006] The cleaning device is preferably configured to clean a surface. The cleaning device may be configured to clean a surface under the guidance of a user, such as by cleaning the surface when the user manually moves the device over the surface. Alternatively or in addition, the cleaning device may be configured to clean the surface in an automated manner. For example, the cleaning device may be configured to move and clean the surface under its own guidance by following an automated protocol. In a specific example, the cleaning device may be a robotic vacuum cleaner configured to move over a floor surface and clean dust particles from the surface as part of an automated cleaning program that may be defined by the user or automatically defined based on the environment in which the surface is located.

[0007] Preferably, the cleaning appliance is a user-operated vacuum cleaner configured to clean the surface by removing dust from the surface as the user moves the vacuum cleaner over the surface. However, the cleaning appliance may be any appliance suitable for cleaning or treating a surface, for example: a vacuum cleaner, a wet vacuum cleaner, a wet and dry vacuum cleaner, a polisher, a steam cleaner, a hard surface cleaner or a carpet cleaner.

[0008] In other words, the cleaning appliance is configured to perform a certain cleaning action, such as vacuuming, dust removal, wiping, etc. In a preferred case, the cleaning action is vacuuming, wherein dust particles on the floor are entrained in an airflow, and the airflow causes the dust particles to be removed from the floor by suction and deposited in a container forming part of the cleaning appliance, in which application, the cleaning appliance may be a vacuum cleaner. In some cases, the container may be removed from the vacuum cleaner. During this process, the dust particles move along the airflow path. In terms of its nature as a vacuum cleaner, it is understood that the vacuum cleaner includes a device for generating a vacuum or negative pressure, thereby generating a suction force, which causes air to flow from the outside of the robot vacuum cleaner into the airflow path. The device for generating a vacuum or negative pressure may be referred to as a vacuum generating component, and may include a motor. The speed of the motor (e.g., revolutions per minute) is preferably proportional to or related to the suction force. The higher the motor speed, the stronger the suction force. The speed of the motor may be controlled by controlling the power supply to the motor, wherein the greater the power provided to the motor, the faster the speed, and vice versa. The motor may have a plurality of predetermined power levels, each corresponding to a predetermined motor speed. Preferably, there are three predetermined power levels corresponding to three different motor speeds.

[0009] At any given time, the vacuum cleaner may be configured to remove dust particles from a surface (e.g., a floor). Specifically, the area of ​​the surface from which dust particles are removed at a given time may be an area of ​​the floor below a cleaning area defining assembly of the vacuum cleaner. Obviously, the cleaning area defining assembly is preferably located at the base of the vacuum cleaner (herein, the term "base" is used to refer to the side or surface of the vacuum cleaner that the vacuum cleaner faces or engages during a cleaning operation). The cleaning area defining assembly preferably includes a housing having an open base, wherein at least the outer wall of the housing defines the edge of the cleaning area.

[0010] The housing preferably comprises an outer shell having a lower edge, the lower edge being arranged to face the surface in use. Preferably, when the vacuum cleaner is placed on the surface and the base faces the surface, the lower edge is arranged to be spaced from the surface. This spacing ensures that air can enter the housing from the outside. In some cases, the housing may contain a brush. In use, when the vacuum cleaner moves on the floor, the brush can be configured to disturb or agitate dust located on the surface, thereby causing the dust to be entrained in the moving air and enter the airflow path. In a preferred case, the brush can be configured to rotate in use, improving the degree to which dust particles are disturbed, thereby ensuring that more dust particles can be entrained in the airflow and can therefore be sucked up. Such a housing may be referred to as a brush bar. The lower edge of the brush bar housing may define a generally rectangular cleaning area of ​​the surface from which dust is removed at any given time. This may be referred to as a "brush bar area" in this article. The rotation of the brush may be controlled by a motor. The speed of the motor may be controlled by controlling the power supply to the motor, wherein the greater the power provided to the motor, the faster the speed, and vice versa. The motor may have a plurality of predetermined power levels, each corresponding to a predetermined motor speed. Preferably, there are three predetermined power levels corresponding to three different motor speeds.

[0011] Here, "airflow path" refers to the route that moving air takes from the outside to the container, for example, below the gap defined by the floor and the lower edge of the outer wall of the housing / casing, and then through the vacuum cleaner into the container. The airflow path preferably includes an air duct located between the housing and the container and in fluid communication therewith.

[0012] A first aspect of the present invention requires a position determination module, which is configured to determine the position information indicating the position of a part of the surface cleaned by the cleaning appliance at a given time. Preferably, the position determination module is configured to determine the position information at a predetermined time interval. The time interval is preferably a regular time interval, which means that each time interval is the same duration. The time interval can have a duration suitable for application of the present invention. For example, the time interval can have a duration of one second, more than one second (for example two seconds, five seconds, ten seconds or more), less than one second (for example half a second, a quarter second or less). The duration of the time interval can be configured to provide more or less continuous position monitoring to the cleaning appliance. For example, by reducing the duration of the time interval to an appropriate amount of time, the position information can be a continuous or nearly continuous set of time series data.

[0013] In some aspects of the present invention, the duration of the time interval can be adjusted based on the moving speed of the cleaning tool on the cleaned surface. For example, when the cleaning tool moves slowly on the surface, the duration of the time interval can be increased without significantly reducing the accuracy of the position information, thereby reducing the computing power required by the processor at this time. In addition, when the cleaning tool moves quickly on the surface, the duration of the time interval can be reduced to improve the accuracy of the position information at this time. The terms "fast" and "slow" provided above can be interpreted as referring to "higher than a predetermined speed threshold" and "lower than a predetermined speed threshold" respectively. The speed threshold can be defined at the time of manufacture, defined by the user, or automatically defined by the system based on how to use the cleaning tool.

[0014] As a working example of the system in use, a user can start cleaning a surface by positioning a cleaning tool at a first position on a surface at a first time point. The user can also start the system of the present invention, or when the cleaning tool is in use, the system can start automatically. The position determination module of the system determines the position information of the first position indicating that the cleaning tool is located on the surface at the first time point. The user can then move the cleaning tool on the surface to achieve the cleaning of the surface. When the user moves the cleaning tool, the cleaning tool will arrive at the second position on the surface at a second time point, and the first and second time points are separated by a single time interval. At the second time point, or after the time interval has passed, the position determination module can be configured to determine the position information of the second position indicating that the cleaning tool is located on the surface at the second time point. When the surface is cleaned, the process can be repeated at multiple positions and time intervals, thereby generating a series of individual position data signals illustrating that the position of the cleaning tool on the surface varies over time.

[0015] As described above, the cleaning appliance may include a cleaning component that defines the portion of the surface that is cleaned at any given time. Returning to the example in which the cleaning appliance is a vacuum cleaner, the cleaning component may be a base portion of the vacuum cleaner, the housing of which defines the cleaning area cleaned by the vacuum cleaner or the portion of the surface that is cleaned. In another example, the cleaning appliance may be a steam cleaner with a steam head, which is a component from which steam is ejected to clean the surface, wherein the steam head defines the portion of the surface that is cleaned at any given time. The position determination module may be configured to determine the position of the cleaning component (e.g., a vacuum base portion or a steam head) within its surroundings, and generate position information based on the position of the cleaning component within its surroundings. In other words, the position determination module may determine the position information of the cleaning component of the cleaning device, rather than determining the position information of the entire cleaning device, thereby improving the accuracy of tracking the area that has been cleaned by the cleaning device.

[0016] The first aspect of the present invention also requires a processor, which is configured to receive a position data signal from a position determination module, and determines the position information based on the position data signal. The processor is also configured to generate an instruction configured to make a display component display a visual representation of the position information. Preferably, the instruction is configured to make the display component display the visual representation of the position information in real time. The term "real time" can be interpreted as referring to the visual representation of the position information when the cleaning utensil is in the display position or is displayed substantially at the same time. In other words, the term "real time" can be interpreted as referring to the visual representation of the position information during the actual time of the cleaning surface. For example, the visual representation of the position information can be displayed at the display component in a sufficiently short amount of time, such as less than one second, so that the visual representation of the position information is almost immediately displayed to the user as feedback to the cleaning process. In other words, when the cleaning utensil moves on the surface, and accordingly when the position data signal generated by the position determination module is updated, the instruction generated by the processor can be updated to make the display display the visual representation of the position information when the position information is updated.

[0017] A display component for showing a visual representation of position data may be included in a device separate from the cleaning appliance, or the display component may be included in the cleaning appliance itself. According to one aspect of the invention, the system may include a mobile device, which includes a position determination module; a processor; and a display component. Preferably, the mobile device is mountable to the cleaning appliance so that when a user uses the cleaning appliance to clean a surface, the display component of the mobile device and therefore the visual representation of the position data is visible to the user. The mobile device may include one or more of the following: a smart phone; a touch screen device; a tablet device; a smart watch, etc. According to another aspect of the invention, the cleaning appliance may include: a position determination module; a processor; and a display component. In addition, the display component may be arranged on the cleaning appliance so that when a user uses the cleaning appliance to clean a surface, the display component of the cleaning appliance and therefore the visual representation of the position data is visible to the user.

[0018] The manner in which the location determination module can determine location information is now discussed.

[0019] In a first example, the system also includes a camera configured to capture video data of its surroundings. In this example, the position determination module is configured to determine the position information based at least on the captured video data. The term "video data" may include a time series of individual frames or a continuous stream of image frames captured by the camera at each time interval. In other words, the video data may include a series of individual images captured by the camera. The camera may be suitable for capturing a single image at each time interval. Alternatively, the camera may capture a continuous image stream, but only a single frame is selected at each time interval to be transmitted to the position determination module. The camera may be configured to work in the visible spectrum or any other suitable spectrum, such as infrared.

[0020] The camera can be configured to capture video data at predetermined time intervals. The time interval is preferably a regular time interval, which means that each time interval is the same duration. The time interval can have a duration suitable for application of the present invention. For example, the time interval can have a duration of one second, more than one second (e.g., two seconds, five seconds, ten seconds or more), less than one second (e.g., half a second, a quarter second or less). The duration of the time interval can be configured to provide more or less continuous position monitoring to cleaning equipment. For example, by reducing the duration of the time interval to an appropriate amount of time, the video data can be a continuous or nearly continuous set of time series data.

[0021] In some aspects of the present invention, the duration of the time interval can be adjusted based on the moving speed of the cleaning tool on the cleaned surface. For example, when the cleaning tool moves slowly on the surface, the duration of the time interval can be increased without significantly reducing the accuracy of the position information, thereby reducing the computing power required by the processor at this time. In addition, when the cleaning tool moves quickly on the surface, the duration of the time interval can be reduced to improve the accuracy of the position information at this time. The terms "fast" and "slow" provided above can be interpreted as "higher than a predetermined speed threshold" and "lower than a predetermined speed threshold" respectively. The speed threshold can be defined at the time of manufacture, defined by the user, or automatically defined by the system based on how to use the cleaning tool.

[0022] As a working example of the system in use, a user can start cleaning a surface by positioning a cleaning tool at a first position on a surface at a first time point. The user can also start the system of the present invention, or when the cleaning tool is in use, the system can be automatically started. The camera can be configured to capture video data from a first position, and the position determination module determines the position information of the first position indicating that the cleaning tool is located on the surface at a first time point based on the captured video data. The user can then move the cleaning tool on the surface to achieve the cleaning of the surface. When the user moves the cleaning tool, the cleaning tool will arrive at a second position on the surface at a second time point, and the first and second time points are separated by a single time interval. At the second time point, or after the time interval has passed, the camera can be configured to capture additional video data, and the position determination module determines the position information of the second position indicating that the cleaning tool is located on the surface at a second time point based on additional video data. When the surface is cleaned, the process can be repeated at multiple positions and time intervals, thereby generating a series of individual position data signals illustrating that the position of the cleaning tool on the surface changes over time.

[0023] Preferably, the display component is configured to display the captured video data, and the instructions are configured to cause the display component to superimpose the visual representation of the position information on the display of the captured video data, so that each site of the visual representation of the position information is superimposed on the corresponding site of the captured video data. The term "corresponding site" can be interpreted as meaning that a given point in real space (i.e., the surrounding of the cleaning appliance) has the same site position in the captured video data and the visual representation of the position information. In other words, the site in the captured video data can correspond to the site in the visual representation of the position information, and the two sites can correspond to the same point in real space.

[0024] According to various aspects of the present invention, the visual representation of the position data can take different forms. For example, the visual representation of the position data can include coordinates or distances moved from an origin, which can be defined at the starting point of the cleaning tool for cleaning a surface. Alternatively, the user can manually define or redefine the origin. For example, the user can define the origin when they start cleaning a first surface with the cleaning tool, and then redefine the origin when they start cleaning a different second surface. In a practical example, this may occur when the user moves between rooms.

[0025] In another example, the visual representation of the position data may include one or more colors for indicating the position of the cleaning device on the surface. In this case, the parameters of the color may change as the position of the cleaning device moves on the surface being cleaned. The parameters of the color may include one or more of the following: opacity; hue; saturation; brightness, etc. For example, in the case where the visual representation of the position data includes a single color, as the cleaning device passes through a given position on the cleaned surface one or more times, the color may become more opaque, more saturated and / or brighter, which means that the opaque, saturated and / or bright color on the display can indicate the cleaning area of ​​the surface. Alternatively, again in the case where the visual representation of the position data includes a single color, when the cleaning device passes through a given position on the cleaned surface one or more times, the color may become less opaque, less saturated and / or darker, which means that the semi-transparent, unsaturated and / or darker color on the display can indicate the cleaning area of ​​the surface. In another example, in the case where the visual representation of the position data includes multiple colors, the hue of the color may change as the cleaning device passes through a given position on the cleaned surface one or more times, which means that a given color that can be predetermined or set by a user can indicate the cleaning area of ​​the surface.

[0026] In another example, the visual representation of the position data can include a visual pattern for representing the position of the cleaning device on the surface. In this case, the parameters of the pattern can change as the position of the cleaning device moves on the surface being cleaned. The parameters of the pattern can include one or more of the following: opacity; size; orientation; repetition of the pattern; color (which can include the sub-parameters of hue, brightness and saturation discussed above) and the like. For example, when the user starts the cleaning appliance, the entire surface can be assigned to a given pattern, and when the cleaning appliance moves on the surface to clean the surface, the pattern can be changed or removed. In a specific example, as the cleaning device moves one or more times on a given position, the visual representation of the position data can show that the pattern fades (i.e., opacity is reduced).

[0027] As described above, where the system also includes a camera to capture video data of the surroundings of the cleaning appliance, visual representations of the position information, such as those discussed above, may be superimposed on the captured video data.

[0028] Taking a vacuum cleaner as an example of a cleaning device for cleaning a floor, the camera can be mounted so that the camera's field of view includes the cleaning head of the vacuum cleaner and at least the immediate vicinity of the cleaning head. The field of view and focal length of the camera lens can be selected according to the cleaning device used and the surface being cleaned. The display component can be mounted in a position that is easily visible to the user, such as near the handle of the cleaning device.

[0029] When the vacuum cleaner is initialized for the first time to prepare for cleaning the surface, the display component can display the video data captured by the camera and the visual representation of the position data superimposed thereon. In a specific example, the position data can be a color or pattern to indicate the position where the cleaning head of the vacuum cleaner has contacted the surface to be cleaned. In such an example, when the vacuum cleaner is initialized for the first time to prepare for cleaning the surface, the display component can only show the captured video data. When the user then moves the vacuum cleaner on the surface to start cleaning, a visual representation of the position data can be generated and superimposed on the captured video data as a colored or patterned path showing where the cleaning head has been. Specifically, at each point of the video data captured by the camera around the cleaning device, the position determination module can determine the position data indicating the position of the surface portion cleaned by the cleaning appliance at a given time, and a visual representation (i.e., a color or pattern) can be generated and superimposed on the video data to be displayed to the user. In this way, over time, when the cleaning appliance moves on the surface, a tracking record of the position occupied by the cleaning appliance since the start of the cleaning process will be presented to the user.

[0030] Alternatively, when the vacuum cleaner is first initialized to prepare to clean a surface, the display component may show the captured video data with a color or pattern layer provided across the surface to be cleaned. When the user then moves the vacuum cleaner over the surface to begin cleaning, a visual representation of the position data may be generated and superimposed on the captured video data as a clear path in the color or pattern layer showing where the cleaning head has been since the start of the cleaning process.

[0031] When cleaning a surface, particularly when vacuuming a surface such as a carpet, it is often desirable to make multiple passes of the cleaning implement over the surface to ensure that the surface has been sufficiently and thoroughly cleaned. The present invention may provide a method of tracking multiple passes of a cleaning implement over a surface, as described below.

[0032] As described above, when the user then moves the vacuum cleaner over the surface to start cleaning, a visual representation of the location data may be generated and superimposed on the captured video data as a colored or patterned path showing where the cleaning head has been. The user may then move the vacuum cleaner so that the subsequent path of the cleaning head at least partially overlaps the initial path marked by the visual representation of the location data, for example by first pushing the vacuum cleaner away from them and then pulling the vacuum cleaner back along the same path. In this case, the visual representation of the location data in the overlapping area may be changed to indicate that the cleaning appliance has passed through this area of ​​the surface multiple times. For example, the visual representation of the location data may be a color with an opacity of 20%. Thus, when the cleaning appliance passes over a portion of the surface for the first time, the visual representation will show a colored path with an opacity of 20%. When the user next moves the cleaning appliance over the area so that the cleaning head of the cleaning appliance passes over the existing colored path, the visual representation of this next pass of the cleaning appliance may simply be added to the existing visual representation, meaning that a portion of the visual representation has an opacity of 40%, indicating where the cleaning appliance has made two passes. This can be repeated until 100% opacity of the visual representation is reached, which can indicate that the desired number of passes of the cleaning tool has been fully achieved. In this example, the user can adjust the opacity of the visual representation to reflect the desired number of passes of the cleaning tool to be achieved when cleaning the surface.

[0033] Alternatively, or in addition to the opacity of the visual representation, any other suitable parameter of the visual representation may be adjusted with each pass of the cleaning tool to indicate the number of passes performed by the cleaning tool. For example, when the visual representation includes colors to be superimposed on the captured video data, multiple passes of the cleaning tool over a portion of the surface may be indicated by an increase / decrease in saturation, a change in hue, and / or an increase / decrease in brightness of the color of the visual representation.

[0034] In another aspect, the system further comprises a light detection and ranging (LiDAR) unit configured to capture mapping data of its surroundings. In this example, the position determination module is configured to determine the position information based at least on the captured mapping data.

[0035] A LiDAR unit is a component used to determine the range or distance between the unit and another object. A LiDAR unit works by generating a light beam (preferably using a laser) and aiming the light at an object or surface. The LiDAR unit is adapted to detect light reflected by an object and measure the time it takes for the light to propagate from the LiDAR unit to the object and back, which may be referred to as a time-of-flight measurement. Based on the time-of-flight measurement and the known speed of light, the LiDAR unit can determine the distance or range between the LiDAR unit and the object or surface that reflected the emitted light. By performing multiple measurements in the environment, such as by scanning a laser around the LiDAR unit, the layout of the area can be mapped. The accuracy of the mapping can be further improved by repeating the above process for the LiDAR unit from multiple locations within the environment.

[0036] The LiDAR unit may include one or more of the following components. The LiDAR unit may include a laser, a sensor, and an actuated mirror. The laser may be configured to emit light with a wavelength range of 500nm to 1600nm, and preferably within the range of 600nm to 1000nm. The laser may be power-limited so that the laser is safe for the user's eyes. Depending on the application of the cleaning appliance, the laser may be operated in a pulsed or continuous manner. Preferably, the laser is a 600nm to 1000nm laser operated in a pulsed manner. The pulse frequency may be selected according to the application of the cleaning appliance. For example, the pulse frequency may be 1Hz, greater than 1Hz (e.g., 2Hz, 5Hz, 10Hz, 100Hz or greater) or less than 1Hz (e.g., 0.5Hz, 0.25Hz or less). The pulse frequency may be configured to provide more or less continuous position monitoring to the cleaning appliance. For example, by increasing the pulse frequency to an appropriate frequency, the captured mapping data and position information may be a continuous or nearly continuous set of time series data.

[0037] In some aspects of the present invention, the pulse frequency can be adjusted based on the moving speed of the cleaning tool on the cleaned surface. For example, when the cleaning tool moves slowly on the surface, the pulse frequency can be reduced without significantly reducing the accuracy of the position information, thereby reducing the computing power required by the processor at this time. In addition, when the cleaning tool moves quickly on the surface, the pulse frequency can be increased to improve the accuracy of the position information at this time. The terms "fast" and "slow" provided above can be interpreted as "higher than a predetermined speed threshold" and "lower than a predetermined speed threshold", respectively. The speed threshold can be defined at the time of manufacture, defined by the user, or automatically defined by the system based on how the cleaning tool is used.

[0038] As a working example of the system in use, a user can start cleaning a surface by positioning a cleaning tool at a first position on the surface at a first point in time. The user can also start the system of the present invention, or the system can automatically start when the cleaning tool is in use. The LiDAR unit can be configured to perform one or more measurements to capture mapping data, and the position determination module of the system determines the position information of the first position on the surface indicating that the cleaning tool is located at the first point in time based on the mapping data. The user can then move the cleaning tool on the surface to achieve the cleaning of the surface. When the user moves the cleaning tool, the cleaning tool will arrive at a second position on the surface at a second point in time, and the first and second points in time are separated by a single time interval defined by the pulse frequency of the laser of the LiDAR unit. At the second point in time, the position determination module determines the position information of the second position on the surface indicating that the cleaning tool is located at the second point in time based on the captured mapping data from the LiDAR unit captured at the second point in time. When the surface is cleaned, the process can be repeated at multiple positions and time intervals, thereby generating a series of separate mapping data sets and thus position data signals that illustrate the position of the cleaning tool on the surface over time.

[0039] The laser and the actuated mirror can be arranged so that the generated light beam hits the actuated mirror, which is suitable for directing the laser beam to the vicinity of the LiDAR unit. The actuated mirror is a mirror, i.e. a reflective surface, which can be moved by an actuating device such as an electric motor. Preferably, the actuated mirror is configured to rotate around at least one axis, and in some cases, around two orthogonal axes. The actuated mirror may include any reflective surface suitable for reflecting the laser beam, such as a plane mirror, a polygonal mirror, a prism, etc. The actuating device may include any actuator suitable for moving the mirror, in particular rotating it around an axis. For example, the actuator may be an electric motor, a stepper electric motor, a micro-electromechanical machine, etc.

[0040] The sensor of the LiDAR unit may be any sensor suitable for detecting reflected light. Preferably, the sensor is a photoelectric sensor or photodetector, which is sensitive to the wavelength of the light generated by the laser, for example 600nm to 1000nm. The photoelectric sensor may be a solid-state photodetector, such as a photodiode or an avalanche photodiode, or a photomultiplier.

[0041] The LiDAR unit may also include a processing unit adapted to determine the time of flight of the reflected laser beam detected by the sensor and derive the range or distance of the reflection point from the LiDAR unit. Alternatively, this function may be performed by a position determination module or processor of the system of the present invention.

[0042] The LiDAR unit can be mounted on a cleaning head of a cleaning appliance to determine the distance between the cleaning head and objects or surfaces around the cleaning head. Alternatively, the LiDAR unit can be mounted on the cleaning appliance so that the cleaning head of the cleaning appliance is within the field of view of the LiDAR unit. In this case, the known distance to the cleaning head can be used to determine the distance between the cleaning head and objects or surfaces in the surrounding environment using appropriate trigonometric functions.

[0043] The above-mentioned LiDAR unit can be used in combination with the above-mentioned camera device. For example, the captured mapping data and the captured video data can be used in combination with each other by the position determination module to determine the position information. In particular, the captured mapping data can be used to provide a distance measurement to an object or surface identified in the captured video data, thereby improving the accuracy of the determined position information, and therefore improving the accuracy of tracking the surface cleaned by the cleaning appliance.

[0044] On the other hand, the system also includes an inertial measurement unit (IMU), which is configured to detect the change of the motion and orientation of the cleaning tool, and generates IMU data in response to the detection. In this example, the position determination module is configured to determine the position information based on at least the IMU data. The IMU may include one or more components for detecting the motion and orientation change of the cleaning tool. For example, the IMU may include an accelerometer, which may be used to detect the acceleration of the cleaning tool in a given direction. In addition, the accelerometer may be configured to determine the orientation of the cleaning tool by measuring the gravity effect on the accelerometer. Alternatively or in addition, the IMU may include a gyroscope sensor. The gyroscope sensor may be suitable for measuring the orientation of the cleaning tool. The orientation of the cleaning tool can be measured in any number of directions or planes. For example, the orientation can be measured in a Cartesian coordinate system with three orthogonal axes. Alternatively, the orientation can be measured as pitch, yaw and roll, which can be defined relative to the position where the cleaning tool is first activated or relative to a predetermined reference position. The IMU may include any number of accelerometers and / or gyroscopes. For example, the IMU may include a single accelerometer or gyroscope, an accelerometer and a gyroscope, three accelerometers (each accelerometer is arranged to measure the acceleration of the cleaning appliance in one of three orthogonal directions), etc. The IMU can be located at any suitable location on the cleaning appliance. For example, the IMU can be located in the cleaning head of the cleaning appliance, which means that when the cleaning head moves over the surface being cleaned, the IMU data will be directly related to the movement and / or orientation of the cleaning head.

[0045] IMU data can be used to determine position information as follows. When the cleaning appliance is first started, the IMU unit can be activated, and when the cleaning appliance is stationary, a reference IMU data set is acquired. When the user moves the cleaning device to start the cleaning process, the IMU unit will detect the acceleration of the cleaning device in a given direction. Based on the acceleration signal and the time between the acceleration changes, the IMU unit or the position determination module can determine the distance that the cleaning appliance moves on the surface being cleaned. Therefore, the position determination module can track the movement of the cleaning appliance on the surface being cleaned.

[0046] The IMU unit may be configured to capture position IMU data at predetermined time intervals. The time interval is preferably a regular time interval, which means that each time interval is the same duration. The time interval may have a duration suitable for application of the present invention. For example, the time interval may have a duration of one second, more than one second (e.g., two seconds, five seconds, ten seconds or more), less than one second (e.g., half a second, a quarter second or less). The duration of the time interval may be configured to provide more or less continuous position monitoring to the cleaning appliance. For example, by reducing the duration of the time interval to an appropriate amount of time, the IMU data may be a continuous or nearly continuous set of time series data.

[0047] In some aspects of the present invention, the duration of the time interval can be adjusted based on the moving speed of the cleaning tool on the cleaned surface. For example, when the cleaning tool moves slowly on the surface, the duration of the time interval can be increased without significantly reducing the accuracy of the position information, thereby reducing the computing power required by the processor at this time. In addition, when the cleaning tool moves quickly on the surface, the duration of the time interval can be reduced to improve the accuracy of the position information at this time. The terms "fast" and "slow" provided above can be interpreted as "higher than a predetermined speed threshold" and "lower than a predetermined speed threshold" respectively. The speed threshold can be defined at the time of manufacture, defined by the user, or automatically defined by the system based on how to use the cleaning tool.

[0048] As a working example of the system in use, a user can start cleaning a surface by positioning a cleaning tool at a first position on a surface at a first time point. The user can also start the system of the present invention, or the system can automatically start when the cleaning tool is in use. The IMU unit can be configured to capture a reference IMU data set of a stationary cleaning tool, and the position determination module of the system determines the position information of the first position on the surface at the first time point based on the reference IMU data set. The user can then move the cleaning tool on the surface to achieve the cleaning of the surface. When the user moves the cleaning tool, the cleaning tool will arrive at a second position on the surface at a second time point, and the first and second time points are separated by a single time interval. At the second time point, or after the time interval has passed, the IMU unit captures additional IMU data, and the position determination module determines the position information of the second position on the surface at the second time point based on additional IMU data. When the surface is cleaned, the process can be repeated at multiple positions and time intervals, thereby generating a series of individual position data signals illustrating the position of the cleaning tool on the surface changing over time.

[0049] The above-mentioned IMU unit can be used in combination with the above-mentioned camera and / or the above-mentioned LiDAR unit. For example, the captured IMU data and the captured mapping data and / or the captured video data can be used in combination with each other by the position determination module to determine the position information. In particular, the captured IMU data can be used to provide motion and / or orientation information on the cleaning head of the cleaning appliance, which can be used to inform how the perspective of an object or surface in the field of view of the captured video data and / or the captured mapping data changes, thereby improving the accuracy of the determined position information and thus improving the accuracy of tracking the surface cleaned by the cleaning appliance.

[0050] According to one aspect of the present invention, the processor can be configured to determine one or more operating characteristics of the cleaning appliance, wherein the instructions are further configured to display a visual representation of the operating characteristics of the cleaning appliance. Preferably, the one or more operating characteristics include one or more of the following: motor speed; operating mode; stroke speed; speed of the cleaning appliance relative to the surface being cleaned; multiple pass counts; and data related to particles collected by the cleaning appliance, which include one or more of the following: particle weight; particle size; and particle type. The visual representation of the operating characteristics of the cleaning appliance can be displayed by a display component in combination with a visual representation of position information. For example, the visual representation of the operating characteristics of the cleaning appliance can be overlaid on the visual representation of the position information, or displayed in a separate window on the display component. Alternatively, the visual representation of the operating characteristics of the cleaning appliance can be displayed on a display component separated from the display component showing the visual representation of the position information.

[0051] In the case where the cleaning appliance includes multiple cleaning modes, the visual representation of the operating characteristics of the cleaning appliance can include the current cleaning mode. The visual representation of the operating characteristics of the cleaning appliance can also include other available cleaning modes in a visual arrangement different from the current cleaning mode, so as to highlight which mode the cleaning appliance is currently in, while still showing all or some available modes to the user. As mentioned above, the visual representation of the positional information can provide the indication of when the same part of the surface has been cleaned multiple times or multiple passes by the cleaning appliance. The visual representation of the operating characteristics of the cleaning appliance can include a digital count of the number of times the area below the cleaning head of the cleaning appliance has been cleaned or passed, which can be referred to as multiple passes counting.

[0052] The cleaning appliance may include one or more sensors to detect other operating parameters of the cleaning appliance. For example, if the cleaning appliance includes a motor, such as a motor for driving the vacuum generating component of a vacuum cleaner, the cleaning appliance may include a rotation sensor to detect the motor speed. Similarly, if the cleaning appliance includes a rotating component, such as a rotating brush head or a rotating polishing head, the cleaning appliance may include a rotation sensor to detect the travel speed of the rotating component. In addition, the cleaning appliance may include a speed sensor for detecting the speed of the cleaning appliance relative to the surface being cleaned. The speed sensor may be a mechanical speed sensor or an optical speed sensor.

[0053] In addition, the operating parameters of the cleaning appliance may include data related to particles collected by the cleaning appliance. Preferably, the data related to particles collected by the cleaning appliance include one or more of: particle weight; particle size; and particle type. The cleaning appliance may include a piezoelectric sensor adapted to generate a plurality of electrical signals in response to a corresponding plurality of vibrations, each vibration being caused by a dust particle impacting a detection area of ​​the cleaning appliance.

[0054] The piezoelectric sensor preferably comprises a piezoelectric crystal. Those skilled in the art will appreciate that piezoelectric crystals have the property of accumulating charge in the material in response to an applied mechanical stress. This means that a voltage can be generated in response to an impact that causes vibrations in the piezoelectric crystal. Therefore, the electrical signal can be in the form of a voltage signal, although equivalently, a current signal or a power signal can also be used. The electrical signal preferably includes a graph of the electrical characteristic over time (or includes data representing the graph, for example by sampling the voltage across the piezoelectric signal at multiple time points). Preferably, a high sampling rate is used, at least 1000 measurements per second, more preferably at least 5000 measurements per second. More preferably, the sampling rate is at least 10,000 measurements per second, and even more preferably, about 15,000 measurements per second.

[0055] The particle size classes may include one or more of the following: ultrafine dust particles, fine dust particles, medium dust particles, and coarse dust particles. The following table illustrates the upper and lower ranges of particle sizes within each class.

[0056] grade Lower limit (μm) Upper limit (μm) Ultrafine 5 to 15 45 to 135 thin 45 to 135 90 to 270 medium 90 to 270 250 to 750 thick 250 to 750 1000 to 3000

[0057] Of course, in each case, the lower limit must be lower than the upper limit. In preferred cases, the following upper and lower limits can be used.

[0058] grade Lower limit (μm) Upper limit (μm) Ultrafine 10 90 thin 90 180 medium 180 500 thick 500 2000

[0059] Here, the term "dust particle size" is defined according to industry standards. Specifically, dolomite particles are used to define "particle size". The particles are assumed to be spherical in nature with an average density of 2.7 g / cm 3 According to this definition, the term "particle size" refers to the diameter of such particles. This is in accordance with the industrial standard IEC-60312 relating to household vacuum cleaners.

[0060] It will be appreciated that by monitoring the piezoelectric signal over time, the number of particles by weight cleaned by the cleaning device can be determined.

[0061] The cleaning device may also include a sensor device adapted to determine the type of particles cleaned by the cleaning device. For example, the sensor device may include an optical sensor for determining the type of particles cleaned by the cleaning device, such as a spectrometer or a camera, which may identify the particle type by image recognition.

[0062] The examples given above are described in the context of a single cleaning device. However, the system can be adapted to work with multiple cleaning appliances to track the areas that have been cleaned using each cleaning appliance. For example, a robot vacuum cleaner can automatically clean a floor surface, but due to the path limitations of the robot vacuum cleaner, a portion of the floor (e.g., at the edge of the room) may not be cleaned by the robot vacuum cleaner. The robot vacuum cleaner may include some or all of the above systems to track which areas of the floor surface it has cleaned. This information may be transmitted to a second cleaning appliance, such as a vacuum cleaner operated by a user, and the visual representation of the position information may include an indication of the surface portion that has been cleaned by the robot vacuum cleaner. The user can then use this information to guide the second cleaning appliance to complete the surface cleaning in a more efficient manner. The visual representation of the position information may be different for each cleaning appliance, or may be the same. The visual representation of the position information of each cleaning appliance may be shown on the display component at the same time, or may be selectively shown to the user, such as by the user selecting which (which) cleaning appliances they wish to view.

[0063] A first aspect of the present invention focuses on a system for tracking an area that has been cleaned using a cleaning appliance. A corresponding second aspect of the present invention provides a method for tracking an area that has been cleaned using a cleaning appliance, the method comprising: determining, by a position module, position information indicating the position of a portion of a surface cleaned by a cleaning appliance at a given time; generating a position data signal encoding the position information; determining, by a processor, the position information based on the position data signal; and generating instructions configured to cause a display component to display a visual representation of the position information. It is easy to understand that the method according to the second aspect of the present invention achieves the same technical effect as the system of the first aspect of the present invention.

[0064] Optional features of the first aspect of the invention are equally applicable to the second aspect of the invention, unless clearly incompatible, or the context clearly dictates otherwise. Various optional features are explicitly listed below, but it should be noted that the features listed above still apply.

[0065] The method may also include, by the position determination module, determining the position information at predetermined time intervals. The time interval is preferably a regular time interval, which means that each time interval is the same duration. The time interval may have a duration suitable for application of the present invention. For example, the time interval may have a duration of one second, more than one second (for example two seconds, five seconds, ten seconds or more), less than one second (for example half a second, a quarter second or less). The duration of the time interval may be configured to provide more or less continuous position monitoring to cleaning equipment. For example, by reducing the duration of the time interval to an appropriate amount of time, the position information may be a continuous or nearly continuous set of time series data.

[0066] In some aspects of the present invention, the duration of the time interval can be adjusted based on the moving speed of the cleaning tool on the cleaned surface. For example, when the cleaning tool moves slowly on the surface, the duration of the time interval can be increased without significantly reducing the accuracy of the position information, thereby reducing the computing power required by the processor at this time. In addition, when the cleaning tool moves quickly on the surface, the duration of the time interval can be reduced to improve the accuracy of the position information at this time. The terms "fast" and "slow" provided above can be interpreted as "higher than a predetermined speed threshold" and "lower than a predetermined speed threshold" respectively. The speed threshold can be defined at the time of manufacture, defined by the user, or automatically defined by the system based on how to use the cleaning tool.

[0067] As a working example of the system in use, a user can start cleaning a surface by positioning a cleaning tool at a first position on the surface at a first time point. The user can also start the system of the present invention, or the system can automatically start when the cleaning tool is in use. The position determination module of the system determines position information, and the position information indicates that the cleaning tool is located at the first position on the surface at this first time point. The user can then move the cleaning tool on the surface to achieve the cleaning of the surface. When the user moves the cleaning tool, the cleaning tool will arrive at the second position on the surface at a second time point, and the first and second time points are separated by a single time interval. At the second time point, or after the time interval has passed, the position determination module determines the position information indicating that the cleaning tool is located at the second position on the surface at this second time point. When the surface is cleaned, the process can be repeated at multiple positions and time intervals, thereby generating a series of individual position data signals illustrating that the position of the cleaning tool on the surface changes over time.

[0068] The method may further include obtaining one or more of: video data via a camera as described above; mapping data via a LiDAR unit as described above; and IMU data via an IMU unit as described above. The position determination module may be adapted to determine position data based on the captured video data, the captured mapping data, and / or the captured IMU data.

[0069] The method may also include generating instructions to cause the display component to display a visual representation of the position information in real time. The term "real time" may be interpreted as meaning that the visual representation of the position information is displayed while the cleaning device is in the display position. In other words, when the cleaning device moves on the surface, and accordingly when the position data signal generated by the position determination module is updated, the instructions generated by the processor may be updated to cause the display to show a visual representation of the position information when the position information is updated.

[0070] As described above, the method may include capturing video data around the cleaning appliance by a camera. The term "video data" may include a time series of individual frames or continuous stream image frames captured by the camera at each time interval. In other words, the video data may include a series of individual images captured by the camera. The camera may be adapted to capture a single image at each time interval. Alternatively, the camera may capture a continuous stream of images, but only a single frame is selected at each time interval to be transmitted to the position determination module. The camera may be configured to work in the visible spectrum or any other suitable spectrum, such as infrared.

[0071] The method may also include displaying the captured video data by a display component, and superimposing the visual representation of the position information on the display of the captured video data, so that each site of the visual representation of the position information is superimposed on the corresponding site of the captured video data. The term "corresponding site" may be interpreted as meaning that a given point in real space (i.e., the surrounding of the cleaning device) has the same site position in the captured video data and the visual representation of the position information. In other words, the site in the captured video data may correspond to the site in the visual representation of the position information, and the two sites may correspond to the same point in real space.

[0072] According to various aspects of the present invention, the visual representation of the position data can take different forms. For example, the visual representation of the position data can include coordinates or distances moved from an origin, which can be defined at the starting point of the cleaning tool for cleaning a surface. Alternatively, the user can manually define or redefine the origin. For example, the user can define the origin when they start cleaning a first surface with the cleaning tool, and then redefine the origin when they start cleaning a different second surface. In a practical example, this may occur when the user moves between rooms.

[0073] In another example, the visual representation of the position data may include one or more colors for indicating the position of the cleaning device on the surface. In this case, the parameters of the color may change as the position of the cleaning device moves on the surface being cleaned. The parameters of the color may include one or more of the following: opacity; hue; saturation; brightness, etc. For example, in the case where the visual representation of the position data includes a single color, as the cleaning device passes through a given position on the cleaned surface one or more times, the color may become more opaque, more saturated and / or brighter, which means that the opaque, saturated and / or bright color on the display can indicate the cleaning area of ​​the surface. Alternatively, again in the case where the visual representation of the position data includes a single color, when the cleaning device passes through a given position on the cleaned surface one or more times, the color may become less opaque, less saturated and / or darker, which means that the semi-transparent, unsaturated and / or darker color on the display can indicate the cleaning area of ​​the surface. In another example, in the case where the visual representation of the position data includes multiple colors, the hue of the color may change as the cleaning device passes through a given position on the cleaned surface one or more times, which means that a given color that can be predetermined or set by a user can indicate the cleaning area of ​​the surface.

[0074] In another example, the visual representation of the position data can include a visual pattern for representing the position of the cleaning device on the surface. In this case, the parameters of the pattern can change as the position of the cleaning device moves on the surface being cleaned. The parameters of the pattern can include one or more of the following: opacity; size; orientation; repetition of the pattern; color (which can include the sub-parameters of hue, brightness and saturation discussed above) and the like. For example, when the user starts the cleaning appliance, the entire surface can be assigned to a given pattern, and when the cleaning appliance moves on the surface to clean the surface, the pattern can be changed or removed. In a specific example, as the cleaning device moves one or more times on a given position, the visual representation of the position data can show that the pattern fades (i.e., opacity is reduced).

[0075] As described above, where the system also includes a camera to capture video data about the cleaning appliance, visual representations of the position information, such as those discussed above, may be superimposed on the captured video data.

[0076] Taking a vacuum cleaner as an example of a cleaning device for cleaning a floor, the camera can be mounted so that the camera's field of view includes the cleaning head of the vacuum cleaner and at least the immediate vicinity of the cleaning head. The field of view and focal length of the camera lens can be selected according to the cleaning device used and the surface being cleaned. The display component can be mounted in a position that is easily visible to the user, such as near the handle of the cleaning device.

[0077] When the vacuum cleaner is initialized for the first time to prepare for cleaning the surface, the display component can display the video data captured by the camera and the visual representation of the position data superimposed thereon. In a specific example, the position data can be a color or pattern to indicate the position where the cleaning head of the vacuum cleaner has contacted the surface to be cleaned. In such an example, when the vacuum cleaner is initialized for the first time to prepare for cleaning the surface, the display component can only show the captured video data. When the user then moves the vacuum cleaner on the surface to start cleaning, a visual representation of the position data can be generated and superimposed on the captured video data as a colored or patterned path showing where the cleaning head has been. Specifically, at each point of the video data captured by the camera around the cleaning device, the position determination module can determine the position data indicating the position of the surface portion cleaned by the cleaning appliance at a given time, and a visual representation (i.e., a color or pattern) can be generated and superimposed on the video data to be displayed to the user. In this way, over time, when the cleaning appliance moves on the surface, a tracking record of the position occupied by the cleaning appliance since the start of the cleaning process will be presented to the user.

[0078] Alternatively, when the vacuum cleaner is first initialized to prepare to clean a surface, the display component may show the captured video data with a color or pattern layer provided across the surface to be cleaned. When the user then moves the vacuum cleaner over the surface to begin cleaning, a visual representation of the position data may be generated and superimposed on the captured video data as a clear path in the color or pattern layer showing where the cleaning head has been since the start of the cleaning process.

[0079] When cleaning a surface, particularly when vacuuming a surface such as a carpet, it is often desirable to make multiple passes of the cleaning implement over the surface to ensure that the surface has been sufficiently and thoroughly cleaned. The present invention may provide a method of tracking multiple passes of a cleaning implement over a surface, as described below.

[0080] As described above, when the user then moves the vacuum cleaner over the surface to start cleaning, a visual representation of the location data may be generated and superimposed on the captured video data as a colored or patterned path showing where the cleaning head has been. The user may then move the vacuum cleaner so that the subsequent path of the cleaning head at least partially overlaps the initial path marked by the visual representation of the location data, for example by first pushing the vacuum cleaner away from them and then pulling the vacuum cleaner back along the same path. In this case, the visual representation of the location data in the overlapping area may be changed to indicate that the cleaning appliance has passed through this area of ​​the surface multiple times. For example, the visual representation of the location data may be a color with an opacity of 20%. Thus, when the cleaning appliance passes over a portion of the surface for the first time, the visual representation will show a colored path with an opacity of 20%. When the user next moves the cleaning appliance over the area so that the cleaning head of the cleaning appliance passes over the existing colored path, the visual representation of this next pass of the cleaning appliance may simply be added to the existing visual representation, meaning that a portion of the visual representation has an opacity of 40%, indicating where the cleaning appliance has made two passes. This can be repeated until 100% opacity of the visual representation is reached, which can indicate that the desired number of passes of the cleaning tool has been fully achieved. In this example, the user can adjust the opacity of the visual representation to reflect the desired number of passes of the cleaning tool to be achieved when cleaning the surface.

[0081] Alternatively, or in addition to the opacity of the visual representation, any other suitable parameter of the visual representation may be adjusted with each pass of the cleaning tool to indicate the number of passes performed by the cleaning tool. For example, when the visual representation includes colors to be superimposed on the captured video data, multiple passes of the cleaning tool over a portion of the surface may be indicated by an increase / decrease in saturation, a change in hue, and / or an increase / decrease in brightness of the color of the visual representation.

[0082] According to one aspect of the present invention, the method may also include determining one or more operating characteristics of the cleaning appliance, and generating instructions to cause a display component to display a visual representation of the operating characteristics of the cleaning appliance. Preferably, the one or more operating characteristics include one or more of the following: motor speed; operating mode; stroke speed; speed of the cleaning appliance relative to the surface being cleaned; multiple pass counts; and data related to particles collected by the cleaning appliance, which include one or more of the following: particle weight; particle size; and particle type. The visual representation of the operating characteristics of the cleaning appliance can be displayed by the display component in combination with the visual representation of the position information. For example, the visual representation of the operating characteristics of the cleaning appliance can be overlaid on the visual representation of the position information, or displayed in a separate window on the display component. Alternatively, the visual representation of the operating characteristics of the cleaning appliance can be displayed on a display component separated from the display component showing the visual representation of the position information.

[0083] With regard to the nature of the visual representation of position information, the optional features set out above also apply to the second aspect of the invention.

[0084] A second aspect of the invention focuses on a method for tracking areas that have been cleaned using a cleaning appliance. A corresponding third aspect of the invention provides a computer program product comprising instructions which, when executed by a computer, cause the computer to perform the method steps described above in relation to the second aspect of the invention.

[0085] Furthermore, a corresponding fourth aspect of the present invention provides a computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to perform the method steps described above with respect to the second aspect of the present invention.

[0086] The present invention includes any combination of the described aspects and preferred features unless such a combination is expressly impermissible or explicitly avoided. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] Embodiments of the present invention will now be described with reference to the accompanying drawings, in which:

[0088] - Figure 1A It is a high-level system diagram including mobile equipment and cleaning equipment.

[0089] - Figure 1B A cleaning appliance is shown, highlighting a user-operated vacuum cleaner as an example of a cleaning appliance and a smartphone as a mobile device.

[0090] - Figure 2 is a schematic diagram showing components of a mobile device.

[0091] - Figure 3 is a schematic diagram showing components of a cleaning appliance.

[0092] - Figure 4 is a schematic diagram showing components of a position determination module of a cleaning appliance or mobile device.

[0093] - Figure 5 is a schematic diagram showing components of a sensor array of a cleaning device.

[0094] - Figure 6 is a flow chart illustrating the high-level steps performed by the system of the present invention.

[0095] - Fig. 7A , 7B , 7C and 7D are flow charts illustrating the generation of a visual representation of location information.

[0096] - Figures 8A to 8E is a schematic diagram of an example of a visual representation of location information that may be displayed to a user.

[0097] - Figures 9A to 9C is a schematic diagram of how a visual representation of position information may be superimposed onto captured video data. DETAILED DESCRIPTION

[0098] Various aspects and embodiments of the present invention will now be discussed with reference to the accompanying drawings. Other aspects and embodiments will be apparent to those skilled in the art. All documents mentioned herein are incorporated herein by reference.

[0099] Figure 1A It is a high-level schematic diagram of a system 1, which can be used to perform, for example, the method of the first aspect of the present invention. System 1 includes up to three components: an optional mobile device 10 and a cleaning appliance 20. These components can be interconnected, as shown by the lines connecting them. Specifically, the mobile device 10 and the cleaning appliance 20 can be connected to each other via one or more networks (not shown), preferably a wireless network, such as a Wi-Fi network or a cellular network. In some cases, these three components can all be connected by the same network, and in other cases, each pair of components can be connected by a separate corresponding network. In other cases, two pairs of components can be connected by a first network, and the third pair of components can be connected by a second network. Figure 1B An example of a cleaning appliance 20 and a mobile device 10 is shown, which may be used in embodiments of the present invention and will be discussed in greater detail later.

[0100] Now refer to Figures 1B to 5 The components constituting the mobile device 10 and the cleaning device 20 are described respectively, and then reference is made to Figures 6 to 9C Methods that can be performed by the corresponding components are described in detail.

[0101] Figure 2An example of a mobile device 10 that can be used in the method according to the present invention is shown. The mobile device 10 is preferably a handheld device, such as a smart phone or a tablet computer, on which applications configured to perform various aspects of the present invention are stored. The mobile device 10 includes a cleaning device interface module 102 for interfacing with a cleaning device. The mobile device 10 also includes a display component 104, an input receiving component 106 (such as a touch screen, which can be integrated with the display component 104), a processor 108 and a memory 110. The mobile device also includes a position determination module 114, which is configured to determine the position information indicating the position of a portion of the surface cleaned by the cleaning device 20 at a given time. The functions of these components will be described in more detail later in this application with reference to the various methods that can be performed according to the present invention.

[0102] Figure 3 2 is a schematic diagram of a cleaning device 20 that can be used in the method according to the present invention. Figure 2 When used in combination with the mobile device described above, the cleaning appliance 20 further comprises a mobile device interface module 202 for interfacing with the mobile device 1 .

[0103] In the case where no mobile device 10 is used in conjunction with a cleaning appliance, the cleaning appliance 20 itself may include a location determination module 201 configured to determine location information indicating the location of a portion of a surface cleaned by the cleaning appliance 20 at a given time. The cleaning device 20 will also include a processor 204, which includes an instruction generation module 210, the purpose of which is to receive a location data signal from the location determination module and generate instructions configured to cause the display component 216 to display a visual representation of the location information contained in the location data signal. The functions of these modules will be described in detail later in this application.

[0104] Figure 4 Shows Figure 3 Position determination module 201 (which may also correspond to Figure 2Detailed schematic diagram of the position determination module 114) and the position capture unit 2011. The position capture unit 2011 is a unit, sensor or module that captures data related to the surroundings of the cleaning appliance, which can then be interpreted by the position determination module to determine the position information indicating the position of a portion of the surface cleaned by the cleaning appliance at a given time. For example, the position capture unit may include one or more of the following: a camera configured to capture video data as described above; a LiDAR unit configured to capture mapping data as described above; and an IMU unit configured to capture IMU data as described above. The position determination module 201 can then determine the position data based on the captured video data, the captured mapping data and / or the captured IMU data. The position determination module can receive the captured video data, the captured mapping data and / or the captured IMU data at the position capture unit interface module 2012. The position information is then determined from the captured data by the position information determination module 2014, and the position data signal generation module 2016 generates a position data signal to be sent to the processor 204.

[0105] Back to Figure 3 The cleaning appliance may also include a sensor array 218 for determining operating parameters of the cleaning device. The sensor array may include any of the above sensors. The operating parameters of the cleaning appliance are determined by the operating parameter determination module 2182, which may then be provided to the processor 204 via the processor interface module 2184 so that the processor generates instructions via the instruction generation module 212 to cause the display component to display a visual representation of the operating parameters.

[0106] Back again Figure 3 The robotic vacuum cleaner 20 also includes a memory 206 that includes a buffer 214 .

[0107] Methods that may be performed according to some aspects of the present invention are now described in detail. Figure 6is a flow chart illustrating the high-level steps performed by the above-described system, and in particular, the position determination module and processor of the cleaning appliance 20 or mobile device 10. Steps S600 to S606 occur during a cleaning operation of the cleaning appliance 20, such as when a user uses the vacuum cleaner to clean the floor of a house or other building according to some cleaning plan. In step S600, location information indicating the location of a portion of a surface cleaned by the cleaning appliance at a given time is determined by the position determination module. In step S602, a position data signal encoding the location information is generated by the position determination module. The location data signal is then sent to a processor (step not shown), and in step S604, location information is determined by the processor based on the position data signal. In step S606, instructions are generated by the processor that are configured to cause a display component to display a visual representation of the location information. The instructions can then be sent to the display component so that the user can observe the visual representation. Now referring to Fig. 7A , 7B , 7C and 7D describe methods by which this can be performed.

[0108] Fig. 7A is a flow chart showing the steps performed by the above system when generating a visual representation of location information. In particular, Fig. 7A The illustrated example shows a method for generating a visual representation of location information when the system includes a camera. For example, the system may include a cleaning appliance 20 and a mobile device 10 mounted thereon, such as Figure 2 As shown, wherein the mobile device is a smartphone with a camera. In step S700, the camera captures video data around the cleaning device. The camera can operate as described above. In step S702, the captured video data can be used to determine location information indicating the location of a portion of the surface cleaned by the cleaning device at a given time. For example, the captured video data can be received by a processor of the mobile device, and the processor can then apply various image recognition and / or image processing algorithms to the captured video data to determine location information indicating the location of a portion of the surface cleaned by the cleaning device at a given time. For example, the captured video data may include a cleaning head of the cleaning device, in which case the known distance between the camera and the cleaning head can be used to interpret the distance between the cleaning head and other objects or surfaces in the camera's field of view. In step S704, an instruction is generated by the processor, the instruction being configured to cause a display component (e.g., a screen of a mobile device) to display a visual representation of the location information. As described above, an instruction can be generated to cause a visual representation of the location information to be displayed. In step S706, an instruction is sent to the display component to display a visual representation of the location information to the user. As described above, a visual representation of the location information can be displayed. Reference below Figures 8A to 8E Further discussion of display of visual representations of location information is provided.

[0109] In addition to the captured data, which are the captured LiDAR data and the captured IMU data, Figure 7B and 7C and Fig. 7A In each respective case, LiDAR data and IMU data may be captured and interpreted as described above. Fig.7D and Fig. 7A The system is similar to the one described above, except for the fact that, in addition to the video data captured from the camera of the system, the system also includes a LiDAR unit and / or an IMU unit for the captured LiDAR data and / or IMU data, respectively. The LiDAR data and / or IMU data can be combined with the captured video data to determine the position information as described above. The camera, LiDAR unit, and IMU unit can be located at any point in the system. For example, the camera and IMU unit can be incorporated into a mobile device, such as a smartphone, and the LiDAR unit can be directly incorporated into the cleaning appliance at the cleaning head.

[0110] Figures 8A to 8E A schematic diagram of a visual representation of location information according to various aspects of the invention is shown. The above system can be adapted to utilize any or all of the various visual representations described herein. As described above, according to various aspects of the invention, the visual representation of location data can take different forms.

[0111] Fig. 8A A schematic diagram showing the field of view of a camera capturing video data, such as mounted to a Figure 2 The field of view of the mobile device 10 of the cleaning device 20 is shown with a visual representation of the position information overlaid thereon. Figures 9A to 9C The overlay of the visual representation of the position information on the captured video data is described in further detail.

[0112] exist Fig. 8A In the example shown, the depicted field of view includes a cleaning head 805 of a cleaning device, a floor surface 810 as a cleaned surface, and a wall surface 815. Fig. 8A A visual representation of position information as the tracked position of a cleaning head of a cleaning device over time is shown in the form of regions 820a, 820b and 825, which represent the area of ​​a single pass by the cleaning head and the area of ​​multiple passes (particularly two passes) by the cleaning head, respectively. Although the current field of view is displayed to the user, the full extent of the visual representation may not fit within the field of view visible to the camera, for example when the surface being cleaned is the floor of a large room. However, the visual representation, such as the tracked position of the cleaning head on the surface, can persist outside the current field of view so that when a given area that has been cleaned re-enters the field of view of the camera, the visual representation previously generated for the area is shown on the display component. Fig. 8AIn the specific example shown, the visual representation of the position information includes a color track overlaid on the captured video data to indicate where the cleaning head of the cleaning device has been since the cleaning surface was started. As described above, when the position of the cleaning device moves on the surface being cleaned, the parameters of the color will change. The parameters of the color may include one or more of the following: opacity; hue; saturation; brightness, etc. For example, in the case where the visual representation of the position data includes a single color, as the cleaning device passes through a given position on the cleaned surface one or more times, the color may become more opaque, more saturated and / or brighter, which means that the opaque, saturated and / or bright colors on the display can indicate the cleaning area of ​​the surface. Alternatively, again in the case where the visual representation of the position data includes a single color, when the cleaning device passes through a given position on the cleaned surface one or more times, the color may become less opaque, less saturated and / or darker, which means that the semi-transparent, unsaturated and / or darker colors on the display can indicate the cleaning area of ​​the surface. In another example, where the visual representation of the position data includes multiple colors, the hue of the colors may change as the cleaning tool passes over a given location on the cleaned surface one or more times, meaning that a given color, which may be predetermined or set by a user, may indicate a cleaning area of ​​the surface.

[0113] Back to Fig. 8A In the specific example shown, the user has activated the cleaning appliance, and performed two movements of the cleaning head 805 over the floor surface 810. In the first movement, the user has pushed the cleaning appliance to define a first area 820a of the visual representation. In the second movement, the user has pulled the cleaning appliance to define a second area 820b of the visual representation. For example, the visual representation of the location information may be blue with an opacity of 20%, meaning that areas 820a and 820b would be blue tracks with an opacity of 20%, illustrating where the cleaning head 805 has passed over the floor surface 810 being cleaned. When performing the second movement, the user has moved the cleaning head 805 over a portion of the first area 820a, thereby forming an overlapping area 825 between the two areas. The overlapping area has multiple passes of the cleaning head 805, while areas 820a and 820b only have a single pass of the cleaning head. In Fig. 8A In the example of , the overlapping region 825 will simply have a composite visual representation of the position information, meaning that the overlapping region will be shown as blue with an opacity of 40%. The composite of the visual representation in overlapping regions such as 825 need not be linear. For example, the overlapping region may be shown as blue with an opacity of 30% for two passes, 35% for three passes, and so on.

[0114] In addition to the fact that the overlapping region 830 is shown in a different color than regions 820a and 820b, Figure 8BShown with Fig. 8A Example of a visual representation of the same location information. For example, regions 820a and 820b may be shown as blue with an opacity of 20%, and the overlapping region may be shown as orange with an opacity of 20%.

[0115] The visual representation of the position information is based on the fact that the operating parameters of the cleaning appliance are divided into sub-areas, except when the cleaning head 805 appears in said sub-area. Figure 8C Shown with Fig. 8A Examples of visual representations of the same location information. For example, an operational parameter may be the amount of particles detected in a given sub-region, which may indicate that a particular sub-region contains a greater amount of dust than other sub-regions. Figure 8C In the specific example of , the first area is divided into sub-areas 840a, 842a, and 844a, the second area is divided into sub-areas 842b and 844b, and the overlapping area is divided into sub-areas 852 and 854. Figure 8C In the specific example shown, the visual representation includes: green with an opacity of 20% for sub-areas with light dust; orange with an opacity of 20% for sub-areas with medium dust; and red with an opacity of 20% for sub-areas with heavy dust. The term "light dust" may be understood to mean that the measured particle amount / size is below a predetermined lower limit. The term "medium dust" may be understood to mean that the measured particle amount / size is between a predetermined lower limit and a predetermined upper limit. The term "large dust" may be understood to mean that the measured particle amount / size is above a predetermined upper limit. The thresholds may be set by the manufacturer, the user, or automatically set and adjusted by the system based on learned cleaning routines performed using the cleaning appliance. See Figure 8C , sub-region 840a is a sub-region with a small amount of dust, and thus can be shown as a green region with an opacity of 20%. Sub-regions 842a and 842b are sub-regions with a medium amount of dust, and thus can be shown as an orange region with an opacity of 20%, and the corresponding overlapping region 852 can have an orange opacity of 40%. Sub-regions 844a and 844b are sub-regions with a medium amount of dust, and thus can be shown as a red region with an opacity of 20%, and the corresponding overlapping region 854 can have a red opacity of 40%.

[0116] Besides the fact that the visual representation of the position information changes depending on the operating mode of the cleaning device, Fig.8D Shown with Fig. 8A Examples of visual representations of the same position information. For example, if the cleaning appliance is a wet and dry vacuum cleaner, the cleaning appliance may have a first dry cleaning mode and a second wet cleaning mode. Fig.8DIn the particular example shown, the visual representation includes a first area 860 indicating where the cleaning head 805 moves when the cleaning device is in the first cleaning mode and a second area 865 indicating where the cleaning head 805 moves when the cleaning device is in the second cleaning mode. For example, the first area can have a blue color and the second area can have a red color. The overlapping area 870, which can have a purple color, for example, indicates a surface area that has been through the first and second cleaning modes.

[0117] In addition to the visual representation of the location information including a color or pattern layer 880 disposed over the entire floor surface 810 to be cleaned, Fig. 8E Shown with Fig. 8A Example of a visual representation of the same position information. When the user then moves the vacuum cleaner over the surface to begin cleaning, a visual representation of the position data can be generated and superimposed on the captured video data as a clear path in the color or pattern layer showing where the cleaning head has been since the start of the cleaning process. In other words, as the user moves the cleaning head 805 over the surface 810, the color or pattern layer 880 can be completely or partially removed from the displayed image shown to the user. In this way, the user can achieve complete cleaning of the surface by "removing" all of the color or pattern layers.

[0118] Figures 9A to 9C A schematic diagram demonstrating superimposing a visual representation of position information onto captured video data is shown. Fig.9A A schematic diagram showing the field of view of a camera capturing video data, e.g. Figure 2 The field of view of the mobile device 10 is shown. The depicted field of view includes a first point 900 and a cleaning head 905 of a cleaning appliance. Fig. 9B A schematic diagram of a visual representation of position information of a cleaning tool, which position information may be generated as described above, is shown. The visual representation of the position information of the cleaning tool comprises a second location 910 corresponding to the same real spatial location as the first location 900 . Fig. 9C shows an overlay on the captured video data ( Fig. 9C Cleaning tools (shown) Fig. 9B Schematic diagram of a visual representation of position information of FIG. 1 ) such that a first location 900 and a second location 910 overlap each other.

[0119] The features disclosed in the preceding description, or in the following claims, or in the accompanying drawings, expressed in their specific form or as means for performing the disclosed functions, or as methods or processes for obtaining the disclosed results, may, where appropriate, be used alone or in any combination of these features to implement the invention in its different forms.

[0120] Although the present invention has been described in conjunction with the above exemplary embodiments, many equivalent modifications and variations will be apparent to those skilled in the art when this disclosure is given. Therefore, the above exemplary embodiments of the present invention are considered to be illustrative rather than restrictive. Various changes may be made to the described embodiments without departing from the spirit and scope of the present invention.

[0121] For the avoidance of any doubt, any theoretical explanations provided herein are intended to enhance the reader's understanding. The inventors do not wish to be bound by any of these theoretical explanations.

[0122] Any section headings used herein are for organizational purposes only and are not to be construed as limiting the subject matter described.

[0123] Throughout the specification, including the following claims, unless the context requires otherwise, the words "comprises" and "comprising" and variations will be understood to imply the inclusion of stated integers or steps or groups of integers or steps but not the exclusion of any other integers or steps or groups of integers or steps.

[0124] It must be noted that, as used in the specification and the appended claims, the singular forms "a", "an", and "the" include plural referents unless the context clearly dictates otherwise. Ranges may be expressed herein as from "about" one particular value, and / or to "about" another particular value. When such a range is expressed, another embodiment includes from the one particular value and / or to the other particular value. Similarly, when a value is expressed as an approximation by using the antecedent "about," it will be understood that the particular value forms another embodiment. The term "about" in relation to a numerical value is optional and means, for example, + / - 10%.

Claims

1. A system configured to track an area that has been cleaned using a cleaning device, the system comprising: Position determination module; processor; as well as Display components, where: The position determination module is configured to determine position information indicative of the position of a portion of the surface being cleaned by the cleaning appliance at a given time; and generate a position data signal encoding the position information; and The processor is configured to receive the position data signal from the position determination module and determine the position information based on the position data signal; and generate instructions configured to cause the display component to display a visual representation of the position information.

2. The system of claim 1, wherein: The location determination module is configured to determine the location information at predetermined time intervals.

3. A system according to any one of claims 1 to 2, wherein: The instructions are configured to cause the display component to display a visual representation of the position information in real time.

4. A system according to any preceding claim, wherein: The system also includes a camera configured to capture video data of its surroundings; and The position determination module is configured to determine the position information based at least on captured video data.

5. The system of claim 4, wherein: The display component is configured to display the captured video data; and The instructions are configured to cause the display component to superimpose the visual representation of the position information over a display of the captured video data such that each location of the visual representation of the position information is superimposed over a corresponding location of the captured video data.

6. A system according to any preceding claim, wherein: The system also includes a light detection and ranging (LiDAR) unit configured to capture mapping data of its surroundings; and The position determination module is configured to determine the position information based at least on captured mapping data.

7. A system according to any preceding claim, wherein: The system also includes an inertial measurement unit (IMU) configured to detect changes in motion and orientation of the cleaning implement and generate IMU data in response to the detection; and The position determination module is configured to determine the position information based at least on IMU data.

8. A system according to any preceding claim, wherein: The cleaning implement includes a cleaning member that defines the portion of the surface that is cleaned at any given time; and The position determination module is configured to determine the position of the cleaning member within its surroundings and generate the position information based on the position of the cleaning member within its surroundings.

9. A system according to any preceding claim, wherein: The processor is configured to determine one or more operating characteristics of the cleaning appliance; The instructions are further configured to cause the display to be a visual representation of an operating characteristic of the cleaning appliance.

10. A system according to any preceding claim, wherein: The one or more operating characteristics include one or more of: motor speed; operating mode; stroke speed; speed of the cleaning implement relative to the surface being cleaned; multiple pass counts; and data related to particles collected by the cleaning implement, including one or more of: particle weight; particle size; and particle type.

11. A system according to any preceding claim, further comprising a mobile device comprising: The location determination module; the processor; as well as The display component.

12. The system of claim 11, wherein: The system also includes the cleaning appliance; and The mobile device can be mounted on a cleaning appliance.

13. The system according to any one of claims 1 to 10, further comprising the cleaning appliance comprising: The location determination module; the processor; as well as The display component.

14. A system according to any preceding claim, wherein: The cleaning appliance is a surface cleaning appliance.

15. The system of claim 14, wherein: The floor cleaning appliance is a vacuum cleaner, a wet vacuum cleaner, a dry and wet vacuum cleaner, a polisher, a steam cleaner, a hard surface cleaner or a carpet cleaner.

16. A method for tracking an area that has been cleaned using a cleaning appliance, the method comprising: determining, by a position determination module, position information indicative of a position of a portion of a surface being cleaned by a cleaning appliance at a given time; generating a position data signal encoding position information; determining, by a processor, position information based on the position data signal; and Instructions configured to cause a display component to display a visual representation of the position information are generated.

Images