Cleaning support system and cleaning support program

Abstract

To achieve appropriate support of housework by the embodiment of housework such as display of regions to be cleaned and its performance.SOLUTION: A cleaning support system including a user terminal 1 used by a user of a cleaner and a cleaning support device 3 includes: an imaging part 12 for imaging the neighborhood of a suction port of the cleaner; a position estimation part 32 for estimating the position of the cleaner or the user terminal; a moving locus estimation part 33 for estimating a moving locus of the suction port according to the movement of the cleaner; a region characteristics information specification part 34 for specifying region characteristics information indicating the characteristics on the cleaning of a region to be cleaned by the cleaner according to the moving locus; a display information creation part 35 for creating display information by superposing the region characteristics information on image data captured by the imaging part; and a display part 18 for displaying the display information.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a so-called home appliance, particularly a vacuum cleaner, and information processing technology for managing the same. For this reason, the present invention particularly relates to technology for assisting cleaning. [Background technology]

[0002] Currently, household appliances such as vacuum cleaners are used for housework. As such household appliances, so-called IoT (Internet of Things) household appliances have been proposed. With IoT household appliances, it is possible to use information processing technology to assist with housework such as cleaning. For example, a technology for grasping the cleaning status of a vacuum cleaner has been proposed. Patent Document 1 discloses the following configuration for a vacuum cleaner system 1 including a vacuum cleaner 2 and an information terminal device 3. "The vacuum cleaner 2 includes a suction port 13, a fan 14 for sucking dirt from the suction port 13, a dirt sensor 15 for detecting dirt sucked from the suction port 13 by the fan 14, and a (first) transmission unit for transmitting detection data of the dirt sensor to a predetermined destination. The information terminal device 3 includes a receiving unit for receiving the detection data from the (first) transmission unit, an alarm unit for issuing an alarm based on the detection data received by the receiving unit, and a (second) transmission unit for transmitting the detection data received by the receiving unit to a predetermined server." [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2021-176423 A Summary of the Invention [Problem to be solved by the invention]

[0004] With the recent advancement of AR (Augmented Reality) technology, a virtual world can be superimposed on the real world and displayed. This allows for the support of various tasks. Therefore, the present invention aims to realize appropriate support for housework, such as cleaning, by embodying the housework (for example, visualizing the housework). [Means for solving the problem]

[0005] In order to solve the above problem, in the present invention, a trajectory image showing the trajectory of the suction nozzle and area characteristic information showing the cleaning characteristics of the cleaning area to be cleaned are superimposed on an image captured near the suction nozzle of a vacuum cleaner.

[0006] More specifically, the cleaning support system includes at least a user terminal used by a vacuum cleaner user, and has an imaging unit that images the area around the suction nozzle of the vacuum cleaner, a position estimation unit that estimates the position of the vacuum cleaner or the user terminal, a movement trajectory estimation unit that estimates the movement trajectory of the suction nozzle in accordance with the movement of the vacuum cleaner, an area characteristic information identification unit that identifies area characteristic information indicating cleaning characteristics of the cleaning area of ​​the vacuum cleaner in accordance with the movement trajectory, a display information creation unit that creates display information by superimposing the area characteristic information on image data captured by the imaging unit, and a display unit that displays the display information.

[0007] The present invention also includes a cleaning assistance program that causes a user terminal used by a user of a vacuum cleaner, which is a computer, to function as an imaging unit that images the area near the suction nozzle of the vacuum cleaner, a position estimation unit that estimates the position of the vacuum cleaner or the user terminal, a movement trajectory estimation unit that estimates the movement trajectory of the suction nozzle in accordance with movement of the vacuum cleaner, an area characteristic information identification unit that identifies area characteristic information indicating cleaning characteristics of the cleaning area of ​​the vacuum cleaner in accordance with the movement trajectory, a display information creation unit that creates display information by superimposing the area characteristic information on image data captured by the imaging unit, and a display unit that displays the display information.

[0008] Furthermore, the present invention also includes the above-mentioned cleaning support system, cleaning support program, each device or subsystem constituting the system, and a cleaning support method using the system or subsystems. Effect of the Invention

[0009] According to the present invention, appropriate assistance for housework can be realized by embodying the housework. [Brief description of the drawings]

[0010] [Figure 1] 1 is a perspective view showing a state in which a stick vacuum cleaner according to an embodiment of the present invention is stored in a charging stand in a stick state. FIG. [Diagram 2] FIG. 1 is a diagram showing a state in which a user terminal 1 is installed on a stick vacuum cleaner 2 in one embodiment of the present invention. [Diagram 3] 1 is a system configuration diagram of a cleaning assistance system according to an embodiment of the present invention; [Figure 4] 1 is a functional block diagram of a user terminal 1 in one embodiment of the present invention. [Diagram 5] 1 is a hardware configuration diagram of a user terminal 1 in one embodiment of the present invention. [Figure 6] FIG. 1 is a functional block diagram of a stick vacuum cleaner 2 in one embodiment of the present invention. [Figure 7] 1 is a hardware configuration diagram of a cleaning support device 3 in one embodiment of the present invention. [Figure 8] FIG. 3 is a diagram showing cleaner management information 361 used in one embodiment of the present invention. [Figure 9] FIG. 3 is a diagram showing map information 362 used in one embodiment of the present invention. [Figure 10] 10 is a flowchart showing a processing flow of cleaning support processing in one embodiment of the present invention. [Figure 11] 11 is a flowchart showing a process of identifying area characteristic information in the first embodiment. [Figure 12] FIG. 2 is a diagram showing the display contents on the user terminal 1 in the first embodiment. [Figure 13] 13 is a flowchart showing a process of identifying area characteristic information in the second embodiment. [Figure 14A] FIG. 11 is a diagram showing the display content on the user terminal 1 before cleaning in the second embodiment. [Figure 14B] FIG. 11 is a diagram showing the display contents on the user terminal 1 during and after cleaning in the second embodiment. [Figure 15] 13 is a flowchart showing a process of identifying area characteristic information in the third embodiment. [Figure 16] FIG. 13 is a diagram showing the display contents on the user terminal 1 in the third embodiment. [Figure 17] FIG. 13 is a diagram showing display contents combining area characteristic information of Examples 1 to 3. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] An embodiment of the present invention will be described below. In this embodiment, a trajectory image showing the trajectory of the suction nozzle and area characteristic information showing the characteristics of the cleaning of the cleaning area to be cleaned are superimposed on an image captured around the suction nozzle of a vacuum cleaner, thereby supporting the user in cleaning. Details of this embodiment will be described below with reference to the drawings.

[0012] 1 is a perspective view showing a stick vacuum cleaner 2 according to this embodiment in a stick state stored in a charging stand 26. Here, the stick vacuum cleaner 2 is configured to include a vacuum cleaner body 21, a handle 22, an extension tube 23, and a vacuum cleaner head 24. The vacuum cleaner body 21 and the vacuum cleaner head 24 are connected via the extension tube 23. The vacuum cleaner head 24 also has a suction port for sucking in dust via the extension tube 23.

[0013] The vacuum cleaner body 21 is provided with a handle 22 that is held by a user. A terminal holder 20 is provided below the handle 22 on the vacuum cleaner body 21. The terminal holder 20 holds a user terminal 1 such as a smartphone, and may be configured to be separable from the vacuum cleaner body 21, or may be configured to be integrated with the vacuum cleaner body 21.

[0014] In this embodiment, the cleaner head 24, i.e., the vicinity of the suction nozzle, is photographed by the user terminal 1 held in the terminal holder 20. In this embodiment, at least a part of the cleaning support process, such as photographing, is executed by the user terminal 1.

[0015] The stick vacuum cleaner 2 can be changed to various states for cleaning, such as a handheld state and a stick state. The charging stand 26 in which the stick vacuum cleaner 2 is stored stores the stick vacuum cleaner 2 in the stick state, and is configured with a base member 25, three stand members 28, and a holder member 27. The charging stand 26 can charge the stick vacuum cleaner 2.

[0016] The stick vacuum cleaner 2 in this embodiment may execute at least some of the functions of the cleaning support process. In this case, it is preferable that the stick vacuum cleaner 2 has a communication function with the user terminal 1, etc. However, the stick vacuum cleaner 2 does not need to have a function for executing the cleaning support process or a communication function.

[0017] Next, Fig. 2 is a diagram showing a state in which the user terminal 1 is installed on the stick vacuum cleaner 2 in this embodiment. In this embodiment, as shown in Fig. 2, the user terminal 1 is held in a terminal holder 20 connected to the vacuum cleaner body 21. At this time, the user terminal 1 is held in the terminal holder 20 so that it faces the direction of the vacuum cleaner head 24 and can capture an image of the periphery of the vacuum cleaner head 24, that is, the vicinity of the suction port. It is preferable that the orientation of the terminal holder 20 is variable and therefore the direction in which the user terminal 1 faces can also be changed accordingly. Furthermore, the camera of the user terminal 1 captures an image of the floor surface to be cleaned by the stick vacuum cleaner 2.

[0018] In this embodiment, a trajectory image showing the trajectory of the suction nozzle (cleaner head 24) and area characteristic information showing characteristics regarding the cleaning of the cleaning area to be cleaned are superimposed on the image around the suction nozzle captured by the user terminal 1, and this is displayed on the user terminal 1. As a result, the user can confirm the contents. Note that the area characteristic information may include a virtual barrier showing the boundary of the cleaning area, the amount of accumulated dirt, the cleaning range, and the like. These will be described later.

[0019] Next, the system configuration of this embodiment will be described. FIG. 3 is a system configuration diagram of the cleaning support system in this embodiment. In FIG. 3, a user terminal 1 is attached to a stick vacuum cleaner 2 using a terminal holder 20. The user terminal 1 is connected to a cleaning support device 3 via a router 4 or a network 5. The user terminal 1 may be connected to the network 5 via the router 4, or may be connected via another device, or may be directly connected to the network 5. This cleaning support device 3 can be realized by a computer such as a server managed by the manufacturer of the stick vacuum cleaner 2. In this embodiment, the user terminal 1 and the cleaning support device 3 cooperate to execute the cleaning support process. The configuration of each of these devices and information used in the cleaning support process will be described below.

[0020] First, Fig. 4 is a functional block diagram of the user terminal 1 in this embodiment. The user terminal 1 is a computer used by a user, and can be realized by a smartphone, tablet, PC, etc., and executes a part of the cleaning support process in this embodiment. Note that the user terminal 1 may execute all of the cleaning support process. In this case, the cleaning support device 3 can be omitted.

[0021] 4, the user terminal 1 has a communication unit 11, an image capturing unit 12, a self-position estimation unit 13, a movement trajectory estimation unit 14, an area characteristic information identification unit 15, a display information creation unit 16, an input unit 17, a display unit 18, a positioning unit 19, and a storage unit 190. First, the communication unit 11 has a function of communication via the router 4 and the network 5 and a short-distance wireless communication function. Also, the image capturing unit 12 is realized by a camera or the like, and captures an image of the surroundings, in particular, the periphery of the vacuum cleaner head 24, and acquires the image data.

[0022] Furthermore, the self-position estimation unit 13 estimates the position of the user terminal 1 or the stick vacuum cleaner 2. To this end, the self-position estimation unit 13 uses the positioning results of the positioning unit 19 described below. Furthermore, the movement trajectory estimation unit 14 estimates the movement trajectory of the cleaner head 24 (suction nozzle) of the stick vacuum cleaner 2. In this embodiment, to this end, image data acquired by the imaging unit 12 is used, but the positioning results of a positioning unit provided in the cleaner head 24 may also be used. Note that the self-position estimation unit 13 can be referred to as a position estimation unit that estimates the position of the user terminal 1 or the stick vacuum cleaner 2.

[0023] Furthermore, the area characteristic information identifying unit 15 identifies area characteristic information indicating characteristics regarding cleaning in the corresponding cleaning area. To this end, the area characteristic information identifying unit 15 identifies area characteristic information according to the movement trajectory estimated by the movement trajectory estimating unit 14. The area characteristic information identifying unit 15 may identify area characteristic information according to the position of the user terminal 1 or the stick vacuum cleaner 2 estimated by the self-position estimating unit 13.

[0024] Furthermore, the display information creation unit 16 creates display information using the image data acquired by the photographing unit 12 and the area identification information identified by the area characteristic information identification unit 15. Furthermore, the input unit 17 accepts various operations from the user. Furthermore, the display unit 18 displays the display information and the like. Furthermore, the input unit 17 and the display unit 18 may be configured as one unit, such as a touch panel.

[0025] Furthermore, positioning unit 19 measures physical quantities relating to position such as radio wave intensity, and can be realized by a GPS sensor, a gyro sensor, an acceleration sensor, etc. Storage unit 190 stores cleaner management information 191 and map information 192. Each of these pieces of information will be described later.

[0026] Next, an implementation example of the user terminal 1 will be described. As described above, the user terminal 1 can be realized by various computers such as a smartphone, a tablet, a mobile phone, or a PC. Fig. 5 is a hardware configuration diagram of the user terminal 1 in this embodiment. In Fig. 5, the user terminal 1 has a camera 101, a touch panel 102, a processing device 103, a communication device 104, a storage device 105, and a positioning sensor 111, which are connected to each other via a communication path.

[0027] First, camera 101 is an implementation example of image capturing unit 12, and has an image capturing function. Touch panel 102 is configured to serve as input unit 17 and display unit 18 in Fig. 4, and receives operations from a user and displays various information such as AR information. Note that touch panel 102 may be configured as an input device and an output device.

[0028] The processing device 103 can be realized by a processor such as a CPU (Central Processing Unit), and executes calculations according to a cleaning assistance program 106 stored in a storage device 105, which will be described later.

[0029] The cleaning support program 106 is composed of a self-location estimation module 107, a movement trajectory estimation module 108, an area characteristic information identification module 109, and a display information creation module 110 for each function. Each of these modules may be realized as an individual program or a partial combination. Furthermore, the cleaning support program 106 may be realized as one function of an integrated management application that manages multiple home appliances.

[0030] Here, the configuration shown in FIG. 4, which performs the same functions as each module, is as follows. Self-location estimation module 107: Self-location estimation unit 13 Trajectory estimation module 108: Trajectory estimation unit 14 Area characteristic information identification module 109: Area characteristic information identification unit 15 Display information creation module 110: Display information creation unit 16 Therefore, the processing device 103 executes the processes of the self-position estimation unit 13, the movement trajectory estimation unit 14, the area characteristic information identification unit 15, and the display information creation unit 16 in accordance with the cleaning support program 106. The cleaning support program 106 can be stored in the storage device 105 described later or other storage media.

[0031] 4, and has a function of communication via the router 4 and the network 5 and a short-distance wireless communication. The storage device 105 corresponds to the storage unit 190 in FIG. 4, and stores the above-mentioned cleaning assistance program 106, cleaner management information 191, and map information 192. For this reason, the storage device 105 may be realized by a main storage device such as a memory and a secondary storage device (storage medium) that is a so-called storage. The secondary storage device may be realized by an external HDD (Hard Disk Drive), SSD (Solid State Drive), memory card, or the like. The positioning sensor 111 corresponds to the positioning unit 19 in FIG. 4, and measures the position of the user terminal 1.

[0032] Next, the configuration related to information processing of the stick vacuum cleaner 2 will be described. Fig. 6 is a functional block diagram of the stick vacuum cleaner 2 in this embodiment. In Fig. 6, the stick vacuum cleaner 2 has a control unit 201, a communication unit 204, a display operation unit 205, an operation unit 206, and a positioning unit 207. These are connected to each other via a communication path. Each component will be described below.

[0033] First, the control unit 201 creates a control signal for realizing the function of the stick vacuum cleaner 2 for cleaning, controls the operation of the stick vacuum cleaner 2, and executes other information processing. For this purpose, the control unit 201 has a control signal creation unit 202 and a self-position estimation unit 203.

[0034] The control signal generating unit 202 generates a control signal for executing a cleaning function in accordance with an operation from the user to the display operation unit 205 or a control command from the user terminal 1. Furthermore, the self-position estimating unit 203 estimates the position of the stick vacuum cleaner 2. For this purpose, the positioning result of the positioning unit 207 described later is used.

[0035] The control unit 201 can be realized by an MPU (Micro-Processing Unit). Therefore, each of the control unit 201, the control signal generation unit 202, and the self-position estimation unit 203 can be realized by dedicated hardware or software such as a program.

[0036] The communication unit 204 also has a function of communicating via the router 4 and the network 5 and a short-distance wireless communication function. The communication unit 204 notifies the user terminal 1 and the cleaning support device 3 of the estimated position information of the stick vacuum cleaner 2, and receives operation commands and display information from the user terminal 1 and the cleaning support device 3.

[0037] Furthermore, the display operation unit 205 receives operations for the stick vacuum cleaner 2 from the user and displays display information. Furthermore, the operation unit 206 operates according to a control signal generated by the control signal generation unit 202, or operates according to a user's operation. The positioning unit 207 measures a physical quantity related to a position, such as radio wave intensity, and can be realized by a GPS sensor, a gyro sensor, an acceleration sensor, or the like. Note that the self-position estimation unit 203 and the positioning unit 207 may be omitted. Furthermore, the communication unit 204 may also be omitted.

[0038] Next, returning to Fig. 3, the cleaning support device 3 will be described. In Fig. 3, the cleaning support device 3 has a communication unit 31, a position estimation unit 32, a movement trajectory estimation unit 33, an area characteristic information identification unit 34, a display information creation unit 35, and a storage unit 36. First, the communication unit 31 connects to the user terminal 1 via the network 5. Note that the communication unit 31 may communicate with the communication unit 204 of the stick vacuum cleaner 2.

[0039] Furthermore, the position estimation unit 32 estimates the position of the stick vacuum cleaner 2 or the user terminal 1. To this end, the position estimation unit 32 uses the positioning results of the positioning unit 19 and / or the positioning unit 207. Furthermore, the movement trajectory estimation unit 33 to the display information creation unit 35 have the same functions as the movement trajectory estimation unit 14 to the display information creation unit 16. Furthermore, the storage unit 36 ​​stores vacuum cleaner management information 361 and map information 362.

[0040] Next, a hardware configuration as an implementation example of the cleaning support device 3 will be described. The cleaning support device 3 can be realized by a computer such as a server or a cloud system. Fig. 7 is a hardware configuration diagram of the cleaning support device 3 in this embodiment. In Fig. 7, the cleaning support device 3 has a processing device 301, a communication device 302, a memory 303, and a secondary storage device 304, which are connected to each other via a communication path.

[0041] First, the processing device 301 can be realized by a processor such as a CPU, and executes calculations according to a cleaning support program 305 stored in a secondary storage device 304 described later. Also, the communication device 302 corresponds to the communication unit 31 in Fig. 3, connects to the network 5, and communicates with other devices such as the user terminal 1.

[0042] 3. The memory 303 and the secondary storage device 304 expand the cleaning support program 305 stored in the secondary storage device 304 and information used for processing by the processing device 301. The secondary storage device 304 can be realized by a so-called storage. The secondary storage device 304 stores the cleaning support program 305, vacuum cleaner management information 361, and map information 362. The secondary storage device 304 may be realized by various storage media such as an external HDD (Hard Disk Drive), SSD (Solid State Drive), or memory card, or may be realized by a device separate from the cleaning support device 3, such as a file server.

[0043] Here, cleaning support program 305 is composed of a position estimation module 306, a movement trajectory estimation module 307, an area characteristic information identification module 308, and a display information creation module 309 for each function. Each of these modules may be realized as an individual program or a partial combination. Also, cleaning support program 305 may be realized as one function of an integrated management application that manages multiple home appliances.

[0044] Here, the configuration shown in FIG. 3, which performs the same functions as each module, is as follows. Position estimation module 306: Position estimation unit 32 Trajectory estimation module 307: Trajectory estimation unit 33 Area characteristic information identification module 308: Area characteristic information identification unit 34 Display information creation module 309: Display information creation unit 35 Therefore, the processing device 301 executes the processes of the position estimation unit 32 , the movement trajectory estimation unit 33 , the area characteristic information identification unit 34 , and the display information creation unit 35 in accordance with the cleaning assistance program 305 .

[0045] Here, the cleaning support program 305 executes the same processes as the cleaning support program 106 of the user terminal 1. Therefore, at least some of the overlapping functions (modules, parts) can be omitted. The cleaning support program 305 can be stored in the secondary storage device 304 or other storage media.

[0046] Next, the cleaner management information 361 and the map information 362 will be described. FIG. 8 is a diagram showing the cleaner management information 361 used in this embodiment. The cleaner management information 361 is information for managing cleaners such as the stick cleaner 2, and is used to execute the cleaning support process. For this reason, the cleaner management information 361 has items of movement history and area characteristic information for each user or cleaner. The movement history indicates the movement history of the relevant cleaner. Specifically, it includes the date and time and the corresponding location information.

[0047] The area characteristic information is information indicating the characteristics of the cleaning of the cleaning area of ​​the corresponding vacuum cleaner, and in this embodiment, a virtual barrier, a dust accumulation amount, and a cleaning range are used. At least one of these area characteristic information may be used. Here, the virtual barrier is information indicating the boundary of the cleaning area. The virtual barrier indicates its position and a unit area such as a room in which it exists.

[0048] The amount of dust accumulation is information indicating the amount of dust accumulation (accumulated amount) for each area. The amount of dust accumulation indicates the amount of dust accumulation for each position, and the positions can be expressed as coordinates or unit areas. The cleaning range is information indicating the areas to be cleaned by the vacuum cleaner and the cleaning results. The cleaning range indicates the positions of the areas to be cleaned or the areas that have been cleaned, and the positions can be expressed as coordinates or unit areas. The details of these area characteristic information will be described in each embodiment below.

[0049] Note that the cleaner management information 191 has the same items as the cleaner management information 361, but is information limited to the cleaner owned by the user of the corresponding user terminal 1. In addition, in the cleaner management information 361 and the cleaner management information 191, the movement history and the area characteristic information may be managed as independent information.

[0050] 9 is a diagram showing map information 362 used in this embodiment. The map information 362 is information showing the positional relationship of a place of use where a vacuum cleaner such as the stick vacuum cleaner 2 is used, such as a house or an office. For this reason, the map information 362 has items of user and layout corresponding to the place of use. The user indicates a user of a vacuum cleaner such as the stick vacuum cleaner 2 at the place of use, for example, a resident there. Note that the user item can be omitted.

[0051] The layout indicates the layout of sections such as the floor plan of the relevant usage location, and includes location information such as coordinates. In the example of FIG. 9, the coordinates of each section (room) are shown, but the format is not limited. In addition to the coordinates of the relevant room, the layout also includes the installation positions of obstacles (such as home appliances and furniture) in the room (area). Note that the above items are merely examples, and are not limited to these, and some of them may be omitted. Note that the map information 192 has the same items as the map information 362, but is information limited to the user of the relevant user terminal 1.

[0052] The above is the explanation of the information used in this embodiment, and the cleaning support process in this embodiment will be described next. Fig. 10 is a flowchart showing the processing flow of the cleaning support process in this embodiment. Note that, in the following, this flowchart will be described with the configurations shown in Figs. 3, 4, and 6 as the processing subject.

[0053] First, in step S1, the stick vacuum cleaner 2 is started up. That is, the stick vacuum cleaner 2 executes a cleaning function. Then, in step S4, the photographing unit 12 starts photographing the periphery of the cleaner head 24, that is, the vicinity of the suction port.

[0054] Furthermore, in step S3, in response to the start of step S1, the self-position estimation unit 13 of the user terminal 1 continuously estimates the position of the user terminal 1 or the stick vacuum cleaner 2. Here, since the user terminal 1 is attached to the terminal holder 20, the self-position estimation unit 13 can estimate the position of the user terminal 1 or the stick vacuum cleaner 2 by using a physical quantity related to the position measured by the positioning unit 19. The self-position estimation unit 13 registers this position history in the vacuum cleaner management information 361 as a movement history.

[0055] Furthermore, in step S4, movement trajectory estimation unit 14 estimates the movement trajectory of cleaner head 24 (suction nozzle) based on image data captured by imaging unit 12. Then, in step S5, area characteristic information identification unit 15 identifies area characteristic information corresponding to the estimated movement trajectory from cleaner management information 361. Here, at least one of a virtual barrier, a dust accumulation amount, and a cleaning range can be used as the area characteristic information.

[0056] In step S6, the display information creation unit 16 creates display information by superimposing the image captured by the image capture unit 12 and the area identification information identified by the area characteristic information identification unit 15. In step S7, the display unit 18 displays the created display information. At this time, information indicating the movement trajectory may be included in the display information. Note that the creation of this superimposed display information is preferably performed using so-called AR (Augmented Reality).

[0057] In step S8, the display information creation unit 16 determines whether the area characteristic information has changed due to the movement of the stick vacuum cleaner 2 associated with cleaning. If the area characteristic information has changed (YES), the process proceeds to step S9. If the area characteristic information has not changed (NO), the process proceeds to step S10.

[0058] In step S9, the display information creation unit 16 changes the display information according to the changed area characteristic information. In step S7, the display unit 18 displays this. In step S10, the display unit 18 continues to display the display information. This concludes the description of this embodiment. According to this embodiment, the embodiment of cleaning can be realized by using the display information including the movement trajectory and the area characteristic information. In this embodiment, the user terminal 1 executes the cleaning support process shown in FIG. 10, but at least a part of it may be executed by the stick vacuum cleaner 2 or the cleaning support device 3. For example, the self-location estimation in step S2 and the display of step S7 may be executed by the stick vacuum cleaner 2, or at least a part of step S2 and steps S3 to S6 may be executed by the cleaning support device 3. In this case, step S2 and steps S3 to S6 are executed by any one of the self-location estimation unit 203, the position estimation unit 32 to the display information creation unit 35. Hereinafter, each embodiment including a specific example of the area characteristic information will be described. In each embodiment, the explanation will be centered on step S5 (processing for identifying area characteristic information) and display examples in FIG. EXAMPLES

[0059] First, the first embodiment is an example in which a virtual barrier is used as the area characteristic information. The virtual barrier is information indicating a boundary that separates an area to be cleaned by the stick vacuum cleaner 2 from an area that is not cleaned. Therefore, by displaying this on the captured image data, the user can intuitively grasp the area to be cleaned. The process of the first embodiment will be described below.

[0060] 11 is a flowchart showing the process of identifying area characteristic information in the embodiment 1. In step S501, the area characteristic information identifying unit 15 estimates the room in which the user terminal 1 or the stick vacuum cleaner 2 is located, which is estimated in step S3, from the map information 192. Here, the room is an example of a unit area, and other unit areas may be used.

[0061] In step S502, the area characteristic information identifying unit 15 identifies a virtual barrier existing in the estimated room from the cleaner management information 361. This step is a look-ahead process for extracting the virtual barrier in step S504. Therefore, this step may be omitted.

[0062] Furthermore, in step S503, area characteristic information identification unit 15 estimates a position (within the field of view) in the image data captured in step S2. This can estimate a position within a predetermined range from the position estimated in step S2 as a position within the field of view. Furthermore, in step S504, area characteristic information identification unit 15 extracts an extracted virtual barrier within the field of view from vacuum cleaner management information 361. At this time, area characteristic information identification unit 15 extracts a virtual barrier at a position of area characteristic information corresponding to a position within the field of view.

[0063] Furthermore, in step S505, the area characteristic information identifying unit 15 determines whether an obstacle is present within the field of view. To this end, the area characteristic information identifying unit 15 may perform image processing on the image data captured in step S2, or may use the map information 192. As a result, if there is an obstacle (YES), the process proceeds to step S506. If there is no obstacle (NO), the process proceeds to step S508.

[0064] Furthermore, in step S506, the area characteristic information identification unit 15 creates a temporary virtual barrier to avoid the obstacle. To this end, the area characteristic information identification unit 15 identifies a plurality of positions (coordinates) between the obstacle and the cleaner head 24, and creates a temporary virtual barrier that connects these positions.

[0065] Furthermore, in step S507, the area characteristic information identification unit 15 merges, i.e., connects, the temporary virtual barrier and the extracted virtual barrier extracted in step S504. As a result, in step S508, the area characteristic information identification unit 15 identifies the virtual barrier merged in step S507 or the virtual barrier extracted in step S504 as the virtual barrier to be displayed. Then, in step S6, the display information creation unit 16 creates display information by superimposing the identified virtual barrier on the image data captured in step S2.

[0066] When the display information is displayed, steps S801 and S901 are executed as steps S8 and S9. First, in step S801, the display information creation unit 16 uses image processing technology to determine whether the cleaner head 24 in the image data is in proximity to (including contact with) the virtual barrier. If it is in proximity (YES), the process proceeds to step S901. If it is not in proximity (NO), the process proceeds to step S10 and the display continues.

[0067] Then, in step S901, the display information creating unit 16 may display this alert on the display unit 18, or may output it as a sound.

[0068] Here, the display contents in the first embodiment will be described. Fig. 12 is a diagram showing the display contents on the user terminal 1 in the first embodiment. In Fig. 12, a virtual barrier 18-1 is displayed on the display unit 18 of the user terminal 1, superimposed on image data 18-2. In this case, the area above the virtual barrier 18-1 indicates an area that does not need to be cleaned, and the area below it indicates an area that should be cleaned.

[0069] The image data 18-2 also shows the captured vacuum cleaner head and extension tube (vacuum cleaner 18-4), with an arrow 18-3 in front of them to guide cleaning. This arrow 18-3 indicates the direction to avoid virtual barrier 18-1. As the field of view of the imaging unit 12 changes, the display position of virtual barrier 18-1 on display unit 18 also changes.

[0070] This concludes the description of Example 1, which allows the user to intuitively grasp the area to be cleaned. At least a part of the processing of Example 1 may be executed by the stick vacuum cleaner 2 or the cleaning support device 3, as described in the embodiment. EXAMPLES

[0071] The second embodiment is an example in which the amount of accumulated dirt is used as the area characteristic information. Here, the amount of accumulated dirt is history information indicating the amount of dirt in past cleaning results, and is information indicating the amount of dirt (dust) sucked up for each position. By displaying such an amount of accumulated dirt on the image data, it is possible to notify the user of the location to be cleaned with priority. The process of the second embodiment will be described below.

[0072] 13 is a flowchart showing the process of identifying area characteristic information in Example 2. In step S511, the area characteristic information identifying unit 15 estimates the room in which the user terminal 1 or the stick vacuum cleaner 2 estimated in step S3 is located, from the map information 192 (similar to step S501).

[0073] In step S512, the area characteristic information identifying unit 15 identifies the estimated amount of dust accumulation at each position in the room from the cleaner management information 361. This step is a look-ahead process for extracting the amount of dust accumulation in step S514. Therefore, this step can also be omitted.

[0074] Also, in step S513, similar to step S503, area characteristic information identifying unit 15 estimates a position in the image data (within the field of view) captured in step S2. Then, in step S514, area characteristic information identifying unit 15 extracts the amount of dirt accumulation at each position within the field of view from cleaner management information 361. Then, in step S6, display information creating unit 16 creates display information by superimposing an accumulation icon corresponding to the extracted amount of dirt accumulation on the image data captured in step S2.

[0075] When the display information is displayed, steps S811 and S911 are executed as steps S8 and S9. First, in step S811, the display information creation unit 16 uses an image processing technique to determine whether the cleaner head 24 in the image data has passed over the pile icon. If the cleaner head 24 has passed over the pile icon (YES), the process proceeds to step S911. If the cleaner head 24 has not passed over the pile icon (NO), the process proceeds to step S10 to continue the display.

[0076] In step S911, the display information creating unit 16 changes the display format of the passed pile icon to indicate that cleaning has been performed.

[0077] Here, the display contents in the second embodiment will be described, including the change in step S911. FIG. 14A is a diagram showing the display contents on the user terminal 1 before cleaning in the second embodiment. In FIG. 14A, the pile icons 18-5, 18-6, and 18-7 are displayed on the display unit 18 of the user terminal 1, superimposed on the image data 18-2. In this case, the pile icons 18-5, 18-6, and 18-7 are displayed in a display format according to their pile amounts. In FIG. 14A, the pile amounts are indicated by their shading. That is, the pile icons 18-7, 18-6, and 18-5 are indicated in the order of the pile amounts of dirt. For this reason, it is desirable to give priority to cleaning the location of the pile icon 18-7. For this reason, the display unit 18 displays an arrow 18-3 guiding cleaning.

[0078] As a result, the user operates the stick vacuum cleaner 2 to clean in the direction of the pile icon 18-7. FIG. 14B is a diagram showing the display contents on the user terminal 1 during or after cleaning in the second embodiment. As a result of the above-mentioned cleaning, the location of the pile icon 18-7 is cleaned. As a result, the cleaning priority of the location of the pile icon 18-7 where the cleaner head 24 has passed, that is, the location that has been cleaned, is lowered. As a result, the color of the pile icon 18-8 is displayed lighter than that of the pile icon 18-7. Furthermore, the abnormal pile icon can also be displayed in a heat map format according to the amount of pile.

[0079] This concludes the description of the second embodiment. According to the second embodiment, the location to be cleaned with priority can be presented to the user. EXAMPLES

[0080] The third embodiment is an example in which a cleaning range is used as the area characteristic information. Here, the cleaning range is information indicating the area to be cleaned by the stick vacuum cleaner 2 and the cleaning record. Here, the area to be cleaned may be a place that has not been cleaned many times in the past or a place set by the user. By displaying such a cleaning range on the image data, the user can be notified of the place to be preferentially cleaned. The process of the third embodiment will be described below.

[0081] 15 is a flowchart showing the process of identifying area characteristic information in Example 3. In step S521, the area characteristic information identifying unit 15 estimates the room in which the user terminal 1 or the stick vacuum cleaner 2 estimated in step S3 is located, from the map information 192 (similar to steps S501 and S511).

[0082] In step S522, the area characteristic information identifying unit 15 identifies the estimated cleaning area in the room from the cleaner management information 361. This step is a look-ahead process for extracting the cleaning area in step S524. Therefore, this step can also be omitted.

[0083] Furthermore, in step S523, similarly to steps S503 and S513, area characteristic information identifying unit 15 estimates a position in the image data (within the field of view) captured in step S2. Then, in step S524, area characteristic information identifying unit 15 extracts the cleaning range within the field of view from cleaner management information 361.

[0084] In step S525, the area characteristics information identification unit 15 compares the estimated movement trajectory with the extracted cleaning area. In step S526, the area characteristics information identification unit 15 changes the display format of the cleaning area that overlaps with the movement trajectory. In step S527, the area characteristics information identification unit 15 calculates points as an incentive according to the cleaning performance of the cleaning area.

[0085] Here, the display contents in the third embodiment will be described, including the calculation of points. FIG. 16 is a diagram showing the display contents on the user terminal 1 in the third embodiment. In FIG. 16, the cleaning ranges 18-9, 18-10, and 18-11 are displayed superimposed on the image data 18-2 on the display unit 18 of the user terminal 1. In the example of FIG. 16, the cleaning range 18-9 is indicated by a star icon, which serves as an incentive for children and the like to clean. That is, by cleaning the cleaning range indicated by the star icon, the star icon is erased. Furthermore, by erasing the star icon, that is, by cleaning the cleaning range 18-9, points are calculated. Points for each position are recorded in the area characteristic information of the cleaner management information 191, and points are calculated based on this. In addition, it is desirable to display the size of the star icon in a size according to the points.

[0086] The priority of cleaning for cleaning areas 18-10 and 18-11 is indicated by the shading of the areas. In the example of Fig. 16, cleaning area 18-10 is indicated by the shading of the area, and has a higher priority than cleaning area 18-11. This priority may be determined according to the status of cleaning. In other words, cleaning has been performed in the location indicated by cleaning area 18-11, so cleaning area 18-10 is indicated by the higher priority (cleaning area 18-11 is indicated by the lower priority).

[0087] This concludes the description of the third embodiment. According to the third embodiment, the user can be presented with a priority on the places to be cleaned, and an incentive for cleaning can be given. The present invention is not limited to the first to third embodiments. For example, in the second and third embodiments, the amount of the piled-up icon and the cleaning range are shown by shading, but they may be distinguished by color. The shading and color distinction may be set by the user's operation using the input unit 17. For example, the color can be set according to the user's preference. Furthermore, the area characteristic information of the first to third embodiments may be displayed in combination at least.

[0088] Fig. 17 is a diagram showing display contents combining area characteristic information of Examples 1 to 3. In Fig. 17, the virtual barrier 18-1 of Example 1, the pile icons 18-5, 18-6, and 18-7 of Example 2, and the cleaning range 18-9 of Example 3 on the image data 18-2 are displayed. The display format and the like of these display contents are changed as described in the above-mentioned Examples 1 to 3. Also, the above-mentioned display unit 18, display operation unit 205, communication unit 11, and communication unit 31 can be regarded as output units that output display information. [Explanation of symbols]

[0089] 1...user terminal, 11...communication unit, 12...photography unit, 13...self-position estimation unit, 14...movement trajectory estimation unit, 15...area characteristic information identification unit, 16...display information creation unit, 17...input unit, 18...display unit, 19...positioning unit, 190...storage unit, 191...vacuum cleaner management information, 192...map information, 2...stick vacuum cleaner, 20...terminal holder, 21...vacuum cleaner body, 22...handle, 23...extension tube, 24...vacuum cleaner head, 25...base member, 26...charging stand , 27...holder member, 28...stand member, 201...control unit, 202...control signal generation unit, 203...self-position estimation unit, 204...communication unit, 205...display operation unit, 206...operation unit, 207...positioning unit, 3...cleaning support device, 31...communication unit, 32...position estimation unit, 33...movement trajectory estimation unit, 34...area characteristic information identification unit, 35...display information generation unit, 36...storage unit, 361...vacuum cleaner management information, 362...map information, 4...router, 5...network

Claims

1. A cleaning support system including at least a user terminal used by a user of a vacuum cleaner, An imaging unit that images the vicinity of the suction nozzle of the vacuum cleaner; a position estimation unit that estimates a position of the vacuum cleaner or the user terminal; a movement trajectory estimation unit that estimates a movement trajectory of the suction nozzle in response to a movement of the vacuum cleaner; an area characteristic information identification unit that identifies area characteristic information indicating characteristics of cleaning of a cleaning area of ​​the cleaner according to the movement trajectory; a display information generating unit that generates display information by superimposing the area characteristic information on image data captured by the imaging unit; A cleaning assistance system having a display unit that displays the display information.

2. The cleaning support system according to claim 1 , A cleaning support system in which the area characteristic information includes at least one of a virtual barrier indicating the boundary of the cleaning area, a dust accumulation amount indicating the amount of dust accumulated in each area, and a cleaning range indicating the location of the area to be cleaned or the area that has been cleaned.

3. The cleaning support system according to claim 1 , The area characteristic information is a virtual barrier indicating a boundary of a cleaning area, the image data includes a cleaner head of a vacuum cleaner including the suction nozzle; The display information creation unit outputs an alert when a vacuum cleaner head in the image data comes close to the virtual barrier.

4. The cleaning support system according to claim 3, the area characteristic information identifying unit creates a temporary virtual barrier to avoid an obstacle when an obstacle is present in the image data; and The display information creation unit creates the virtual barrier by connecting an extracted virtual barrier extracted from vacuum cleaner management information and the temporary virtual barrier.

5. The cleaning support system according to claim 1 , the area characteristic information is a dust accumulation amount indicating a dust accumulation amount for each area, the image data includes a cleaner head of a vacuum cleaner including the suction nozzle; The display information creation unit creates an accumulation icon indicating the amount of accumulated dirt, and changes the display format of the accumulation icon when a cleaner head in the image data passes over the accumulation icon.

6. The cleaning support system according to claim 1 , The area characteristic information is a cleaning range indicating the location of an area to be cleaned or an area that has been cleaned, The area characteristic information identifying unit compares the movement trajectory with the cleaning area, and changes a display format of any cleaning area that overlaps with the movement trajectory.

7. A user terminal used by a user of a computer, which is a vacuum cleaner, An imaging unit that images the vicinity of the suction nozzle of the vacuum cleaner; a position estimation unit that estimates a position of the vacuum cleaner or the user terminal; a movement trajectory estimation unit that estimates a movement trajectory of the suction nozzle in response to a movement of the vacuum cleaner; an area characteristic information identification unit that identifies area characteristic information indicating characteristics of cleaning of a cleaning area of ​​the cleaner according to the movement trajectory; a display information generating unit that generates display information by superimposing the area characteristic information on image data captured by the imaging unit; A cleaning assistance program that functions as a display unit that displays the display information.

8. The cleaning support program according to claim 7 , A cleaning assistance program in which the area characteristic information includes at least one of a virtual barrier indicating the boundary of the cleaning area, a dust accumulation amount indicating the amount of dust accumulated in each area, and a cleaning range indicating the location of the area to be cleaned or the area that has been cleaned.

9. The cleaning support program according to claim 7 , The area characteristic information is a virtual barrier indicating a boundary of a cleaning area, the image data includes a cleaner head of a vacuum cleaner including the suction nozzle; The display information creation unit is a cleaning assistance program that outputs an alert when a vacuum cleaner head in the image data comes close to the virtual barrier.

10. The cleaning support program according to claim 9 , the area characteristic information identifying unit creates a temporary virtual barrier to avoid an obstacle when an obstacle is present in the image data; and The display information creation unit creates the virtual barrier by connecting an extracted virtual barrier extracted from vacuum cleaner management information and the temporary virtual barrier.

11. The cleaning support program according to claim 7 , the area characteristic information is a dust accumulation amount indicating a dust accumulation amount for each area, the image data includes a cleaner head of a vacuum cleaner including the suction nozzle; The display information creation unit creates an accumulation icon indicating the amount of accumulated dirt, and changes the display format of the accumulation icon when a cleaner head in the image data passes over the accumulation icon.

12. The cleaning support program according to claim 7 , The area characteristic information is a cleaning range indicating the location of an area to be cleaned or an area that has been cleaned, The area characteristic information identifying unit compares the movement trajectory with the cleaning area, and changes a display format of any cleaning area that overlaps with the movement trajectory.

Images