Cleaning Management Program

Abstract

To facilitate the creation of a cleaning area map in cleaning. [Solution] A cleaning management program that causes a user terminal, which is a computer attached to the terminal holder of a vacuum cleaner, to function as an input unit that accepts multiple specified positions from a user and a cleaning area map creation unit that creates a cleaning area map using lines connecting the specified positions, wherein the cleaning area map creation unit controls the execution of an end instruction to complete the cleaning area map and the determination of self-intersections between the final line segment identified among the line segments in response to the end instruction and other line segments in cooperation with each other.

Description

[Technical Field]

[0001] The present invention relates to so-called home appliances, particularly vacuum cleaners, and information processing technology for managing the same. [Background technology]

[0002] Currently, household appliances such as vacuum cleaners are used for housework. So-called IoT (Internet of Things) appliances have been proposed for these appliances. IoT appliances connect to networks such as the Internet and provide various functions and services, such as operational control. For example, Patent Document 1 describes the following in order to "check the status of dust suction": "Vacuum cleaner system 1 includes vacuum cleaner 2 and information terminal device 3. Vacuum cleaner 2 includes suction nozzle 13, fan 14 that sucks dust through suction nozzle 13, dust sensor 15 that detects dust sucked through suction nozzle 13 by fan 14, and (first) transmitter that transmits detection data from the dust sensor to a predetermined destination. Information terminal device 3 includes a receiver that receives the detection data from (first) transmitter, a notification unit that issues a notification based on the detection data received by the receiver, and a (second) transmitter that transmits the detection data received by the receiver to a predetermined server" (abstract). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-176423 Summary of the Invention [Problem to be solved by the invention]

[0004] Here, housework, including cleaning, is time-consuming, so various support functions are required. For this reason, there is a need to support housework by visualizing the housework that has been done and the housework that needs to be done. In particular, there is a need to more easily create a cleaning area map that serves as a guide for cleaning.

[0005] Therefore, the present invention aims to realize various functions and services related to practical home appliances, particularly vacuum cleaners, that can assist users with housework such as cleaning, and in particular to more easily create a cleaning area map for cleaning. [Means for solving the problem]

[0006] In order to solve the above problems, the present invention performs processing for visualizing housework. In a representative embodiment, the present invention specifies a cleaning area map showing a cleaning area by specifying multiple line segments connecting vertices at specified positions and creating a cleaning area map using an area (e.g., a polygon) defined by the specified multiple line segments. The present invention controls execution of an end instruction to complete the cleaning area map and a determination of self-intersections (also called self-crossings) between the final line segment specified in response to the end instruction and other line segments in cooperation with each other.

[0007] More specifically, this is a cleaning management program that causes a user terminal, which is a computer attached to the terminal holder of a vacuum cleaner, to function as an input unit that accepts multiple specified positions from the user and a cleaning area map creation unit that creates a cleaning area map using lines connecting the specified positions, and the cleaning area map creation unit is a cleaning management program that controls the execution of an end instruction to complete the cleaning area map and the determination of self-intersections between the final line segment identified among the lines in response to the end instruction and other line segments in cooperation with each other.

[0008] Furthermore, the present invention also includes the above-mentioned cleaning management program, a storage medium for storing the program, information processing devices such as a user terminal or a cleaning management device, and a cleaning management method executed by the cleaning management system and each device or subsystem that constitutes the system. [Effects of the Invention]

[0009] According to the present invention, a cleaning area map for cleaning can be created more easily. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a perspective view showing a state in which a stick vacuum cleaner 2 in a stick state is stored in a charging stand according to an embodiment of the present invention. [Figure 2] 1 is a diagram showing a state in which a user terminal 1 is installed on a stick vacuum cleaner 2 according to an embodiment of the present invention. [Figure 3] FIG. 2 is a functional block diagram of a user terminal 1 according to the first embodiment. [Figure 4] FIG. 2 is a hardware configuration diagram of a user terminal 1 in the first embodiment. [Figure 5] FIG. 10 is a diagram showing user management information 112 used in the first embodiment. [Figure 6] FIG. 10 is a diagram showing cleaning management information 113 used in the first embodiment. [Figure 7] FIG. 10 is a diagram showing cleaning area map information 114 used in the first embodiment. [Figure 8] FIG. 10 is a diagram showing guidance information 115 used in the first embodiment. [Figure 9A] 10 is a flowchart showing a process flow for cleaning in the first embodiment. [Figure 9B] 10 is a flowchart showing a process flow for cleaning in the first embodiment. [Figure 10] 11 is a diagram showing a display screen 1101 of an output unit 110 that displays terms of use in the first embodiment. FIG. [Figure 11A] FIG. 11 is a diagram showing a display screen 1102a of the output unit 110 for registering the stick vacuum cleaner 2 in the first embodiment. [Figure 11B] FIG. 11 is a diagram showing a display screen 1102b of the output unit 110 for registering the stick vacuum cleaner 2 in the first embodiment. [Figure 11C] 11 is a diagram showing a display screen 1102c of the output unit 110 for registering the stick vacuum cleaner 2 in the first embodiment. FIG. [Figure 12A] FIG. 11 is a diagram showing a display screen 1103a of the output unit 110 for registering related parties in the first embodiment. [Figure 12B] FIG. 11 is a diagram showing a display screen 1103b of the output unit 110 for registering related parties in the first embodiment. [Figure 12C] FIG. 11 is a diagram showing a display screen 1103c of the output unit 110 for registering related parties in the first embodiment. [Figure 13] FIG. 11 is a diagram showing a display screen 1104 of the output unit 110 that displays instructions for use in the first embodiment. [Figure 14A] FIG. 11 is a diagram showing a display screen 1105a (first example) for explaining the creation of a map in the first embodiment. [Figure 14B] FIG. 11 is a diagram showing a display screen 1105b (second example) for explaining the creation of a map in the first embodiment. [Figure 14C] FIG. 11 is a diagram showing a display screen 1105c (second example) for explaining the creation of a map in the first embodiment. [Figure 15A] FIG. 11 is a diagram showing a display screen 1106a (first example) for creating a map in the first embodiment. [Figure 15B] FIG. 11 is a diagram showing a display screen 1106b (first example) for creating a map in the first embodiment. [Figure 15C] FIG. 11 is a diagram showing a display screen 1106c (first example) for creating a map in the first embodiment. [Figure 15D] FIG. 11 is a diagram showing a display screen 1106d (first example) for creating a map in the first embodiment. [Figure 15E] FIG. 11 is a diagram showing a display screen 1106e (first example) for creating a map in the first embodiment. [Figure 15F] FIG. 11 is a diagram showing a display screen 1106f (second example) for creating a map in the first embodiment. [Figure 15G] FIG. 11 is a diagram showing a display screen 1106g (second example) for creating a map in the first embodiment. [Figure 16A] FIG. 11 is a diagram showing a home screen 1107a (first example) in the first embodiment. [Figure 16B] FIG. 11 is a diagram showing a home screen 1107b (first example) in the first embodiment. [Figure 16C] FIG. 11 is a diagram showing a home screen 1107c (second example) in the first embodiment. [Figure 17A] FIG. 11 is a diagram showing a map selection screen 1108a (first example) in the first embodiment. [Figure 17B] FIG. 11 is a diagram showing a terminal attachment instruction screen 1108b (first example) in the first embodiment. [Figure 17C] FIG. 11 is a diagram showing a player map selection screen 1108c (second example) in the first embodiment. [Figure 17D] FIG. 11 is a diagram showing a player map selection result screen 1108d (second example) in the first embodiment. [Figure 18] 10 is a flowchart showing details of the cleaning-related information display process (step S22) in the first embodiment. [Figure 19A] FIG. 11 is a diagram showing a display screen 1109a (using a game function) of cleaning-related information in the first embodiment. [Figure 19B] FIG. 11 is a diagram showing a display screen 1109b (using a game function) of cleaning-related information in the first embodiment. [Figure 19C] FIG. 11 is a diagram showing a display screen 1109c (using a game function) of cleaning-related information in the first embodiment. [Figure 19D] FIG. 11 is a diagram showing a display screen 1109d (using a game function) of cleaning-related information in the first embodiment. [Figure 20A] FIG. 11 is a diagram showing a display screen 1110a (game function not used) in the process of ending cleaning in the first embodiment. [Figure 20B] FIG. 11 is a diagram showing a display screen 1110b (game function not used) in the process of ending cleaning in the first embodiment. [Figure 20C] FIG. 11 is a diagram showing a display screen 1110c (game function not used) in the process of ending cleaning in the first embodiment. [Figure 20D] FIG. 11 is a diagram showing a display screen 1110d (game function not used) in the process of ending cleaning in the first embodiment. [Figure 21] 10 is a flowchart showing details of the cleaning-related information display process (step S22) in the second embodiment. [Figure 22A] FIG. 11 is a diagram showing a display screen 1111a (using a game function) of cleaning-related information in the second embodiment. [Figure 22B] FIG. 11 is a diagram showing a display screen 1111b (using a game function) of cleaning-related information in the second embodiment. [Figure 23A] FIG. 11 is a diagram showing a display screen 1112a of cleaning-related information (without using the game function) in the second embodiment. [Figure 23B] FIG. 11 is a diagram showing a display screen 1112b of cleaning-related information (without using the game function) in the second embodiment. [Figure 24] FIG. 11 is a system configuration diagram of a cleaning management system according to a third embodiment. [Figure 25] FIG. 10 is a hardware configuration diagram of a cleaning management device 3 in a third embodiment. [Figure 26] FIG. 11 is a diagram showing user management information 310 used in the third embodiment. [Figure 27] FIG. 11 is a diagram showing cleaning management information 311 used in the third embodiment. [Figure 28] FIG. 11 is a diagram showing cleaning area map information 312 used in the third embodiment. [Figure 29] FIG. 11 is a diagram showing guidance information 313 used in the third embodiment. [Figure 30] 11 is a flowchart showing a process flow for cleaning in the third embodiment. [Figure 31] 10 is a diagram schematically illustrating a side view of a stick vacuum cleaner 2 in Example 4 when a terminal holder 20 is attached thereto. FIG. [Figure 32] 31 showing a state in which the terminal holder 20 in the fourth embodiment is attached. FIG. [Figure 33] 10 is a diagram showing a first example of a mounting portion 200 in the fourth embodiment. FIG. [Figure 34] 10 is a diagram showing a second example of the attachment portion 200 in the fourth embodiment. FIG. [Figure 35]10 is a diagram schematically illustrating a side view of a stick vacuum cleaner 2 in Example 5 when a terminal holder 20 is attached thereto. FIG. [Figure 36] 10 is an external view showing a state in which a terminal holder 20 and a handy brush 25 are connected in a fifth embodiment. FIG. [Figure 37] 36 is an enlarged view of FIG. 35 showing a state in which the terminal holder 20 in the fifth embodiment is attached. FIG. [Figure 38] FIG. 13 is a perspective view showing a state in which a terminal holder 20-10 in a sixth embodiment is connected to a handy brush 25. [Figure 39A] 13 is a side view showing a state in which a terminal holder 20-10 in Example 6 is connected to a handy brush 25. FIG. [Figure 39B] 13 is a top view showing a state in which a terminal holder 20-10 in Example 6 is connected to a handy brush 25. FIG. [Figure 39C] 13 is a bottom view showing a state in which a terminal holder 20-10 in Example 6 is connected to a handy brush 25. FIG. [Figure 40A] 13 is a rear view showing a state in which the terminal holder 20-10 in the sixth embodiment is connected to the handy brush 25. FIG. [Figure 40B] FIG. 40B is a cross-sectional view taken along line CC of FIG. 40A in the sixth embodiment. [Figure 41] FIG. 20 is a diagram for explaining a map creation process in the seventh embodiment. [Figure 42] 13 is a flowchart illustrating details of a display process in the eighth embodiment. [Figure 43] 13 is a flowchart illustrating details of a display process in a specific example 1 of the eighth embodiment. [Figure 44A] FIG. 11 is a diagram showing a display screen 1113a in a specific example 1 of the eighth embodiment. [Figure 44B] FIG. 11 is a diagram showing a display screen 1113b in a specific example 1 (modification) of the eighth embodiment. [Figure 44C] FIG. 11 is a diagram showing transitions of display screens 1113c to 1113e in a specific example 1 (modification) of the eighth embodiment. [Figure 45]FIG. 11 is a diagram showing a display screen 1113a including a peripheral display area in a specific example 1 of an eighth embodiment. [Figure 46] FIG. 20 is a diagram illustrating an example of controlling whether or not to display an operating map icon 1114a-1 in the eighth embodiment. [Figure 47] FIG. 13 is a diagram showing a drive instruction display screen in the eighth embodiment. [Figure 48] FIG. 11 is a diagram showing a display screen 1115a displaying the end of cleaning in the eighth embodiment. [Figure 49] FIG. 11 is a diagram showing a display screen 1116a for displaying the cleaning end including points (number of icons) in the eighth embodiment. [Figure 50A] 13 is a flowchart showing the process flow of a map creation process in the ninth embodiment. [Figure 50B] 13 is a flowchart showing the processing flow of the cleaning management process (normal mode) in the ninth embodiment. [Figure 50C] 13 is a flowchart showing the processing flow of cleaning management processing (simple mode) in Example 9. [Figure 51] FIG. 13 is a diagram illustrating the concept of drawing a path in the ninth embodiment. [Figure 52] FIG. 11 is a diagram showing a display screen 1106h on which a movement trajectory is drawn in the ninth embodiment. [Figure 53] FIG. 23 is a diagram showing a cleaning area map in a tenth embodiment. [Figure 54] 19 is a flowchart showing a cleaning area map creation process in a specific example 1 of the tenth embodiment. [Figure 55] 19 is a flowchart showing a cleaning area map creation process in a specific example 2 of the tenth embodiment. [Figure 56] 19 is a flowchart showing a cleaning area map creation process in a specific example 3 of the tenth embodiment. [Figure 57] FIG. 23 is a diagram showing distortion of a cleaning area map 1106i-1 in the eleventh embodiment. [Figure 58] FIG. 23 is a diagram showing the downward rightward slope (distortion) of the upper side of the cleaning area map 1106i-1 in the eleventh embodiment. [Figure 59A] FIG. 20 is a diagram showing the result of correcting the downward tilt (distortion) of the upper side of the cleaning area map 1106i-1 in the eleventh embodiment. [Figure 59B] FIG. 23 is a diagram showing a grid map 1106i-1a in the eleventh embodiment. [Figure 59C] FIG. 20 is a diagram showing a dot map 1106i-1b in the eleventh embodiment. [Figure 60] FIG. 20 is another diagram showing that the downward rightward slope (distortion) of the upper side of the cleaning area map 1106i-1 in the eleventh embodiment has been corrected. [Figure 61A] FIG. 11 is a diagram showing a display screen 1106j of cleaning-related information in the eleventh embodiment. [Figure 61B] FIG. 11 is a diagram showing a display screen 1106k of cleaning-related information in the eleventh embodiment. [Figure 62A] FIG. 23 is a diagram showing a display screen 1106l (part 1) for adjusting a movement trajectory in the twelfth embodiment. [Figure 62B] FIG. 23 is a diagram showing a display screen 1106l (part 2) for adjusting a movement trajectory in the twelfth embodiment. [Figure 62C] FIG. 23 is a diagram showing a display screen 1106l (part 3) for adjusting a movement trajectory in the twelfth embodiment. [Figure 62D] FIG. 23 is a diagram showing a display screen 1106l (part 4) for adjusting a movement trajectory in the twelfth embodiment. [Figure 63] FIG. 23 is a diagram showing a display screen 1106m during cleaning management processing in the thirteenth embodiment. [Figure 64A] FIG. 23 is a diagram for explaining the arrangement of the virtual camera in the thirteenth embodiment. [Figure 64B] FIG. 23 is a diagram for explaining the arrangement of the virtual camera in the height direction in the thirteenth embodiment. [Figure 65A] FIG. 11 is a diagram showing a display screen 1106m in which an actual trajectory 1106m-3 in the thirteenth embodiment extends beyond an actual map 1106m-2. [Figure 65B] FIG. 23 is a diagram for explaining determination of protrusion of an actual trajectory in the thirteenth embodiment. [Figure 65C]23A and 23B are diagrams illustrating adjustment processing for a protruding actual trajectory in the thirteenth embodiment. [Figure 66] FIG. 11 shows a display screen 1106n for switching between the game mode and the normal mode. [Figure 67A] FIG. 11 is a diagram (part 1) showing screen transitions of a display screen 1106o during map creation processing. [Figure 67B] FIG. 11 is a diagram (part 2) showing screen transitions of a display screen 1106o during map creation processing. [Figure 67C] FIG. 11 is a diagram (part 3) showing screen transitions of a display screen 1106o during map creation processing. [Figure 68A] FIG. 11 is a diagram (part 1) showing screen transitions of a display screen 1106p for cleaning after the map creation process. [Figure 68B] FIG. 11 is a diagram (part 2) showing screen transitions of a display screen 1106p for cleaning after the map creation process. [Figure 68C] FIG. 11 is a diagram (part 3) showing screen transitions of a display screen 1106p for cleaning after the map creation process. DETAILED DESCRIPTION OF THE INVENTION

[0011] An embodiment of the present invention will be described below. In this embodiment, a stick vacuum cleaner, which is an example of a manual vacuum cleaner operated by a user, will be used as the vacuum cleaner. Furthermore, the stick vacuum cleaner of this embodiment may or may not have a communication function with other devices such as a smartphone or a router. A configuration for realizing cleaning management processing related to such a stick vacuum cleaner will be described below.

[0012] 1 is a perspective view showing a stick vacuum cleaner 2 according to this embodiment stored in a charging stand 26 in a stick state. The stick vacuum cleaner 2 is comprised of a vacuum cleaner body 21, a handle 22, an extension tube 23, and a vacuum cleaner head 24. The vacuum cleaner body 21 and the vacuum cleaner head 24 are connected via the extension tube 23, which is a pipe through which sucked dust passes.

[0013] The vacuum cleaner body 21 is also provided with a handle 22 that is held by the user. A handy brush 25 can be attached to the vacuum cleaner body 21 below the handle 22, and a vacuum cleaner terminal holder (hereinafter referred to as terminal holder 20) is connected to the handy brush 25. The terminal holder 20 holds a user terminal 1 such as a smartphone, and may be configured to be separable from the vacuum cleaner body 21, or may be configured as an integrated unit. Note that using the handy brush 25 to attach the terminal holder 20 and the stick vacuum cleaner 2 is just one example, and they may also be realized as separate structures, as in Example 4 described below. In this embodiment, at least a part of the cleaning management process is executed by the user terminal 1 held in the terminal holder 20. This function (processing flow) will be described in each example.

[0014] Furthermore, the stick vacuum cleaner 2 can be changed into various usage states such as a handheld state and a stick state for cleaning. The charging base 26 in which the stick vacuum cleaner 2 is stored stores the stick vacuum cleaner 2 in a stick state, and is configured with a base member 29, three stand members 28, and a holder member 27. This charging base 26 can charge the stick vacuum cleaner 2.

[0015] The stick vacuum cleaner 2 may be provided with a label that records identification information for identifying itself and that can be photographed and read by the user terminal 1. For example, a label such as an AR marker or nameplate can be provided on the cleaner head 24 that faces the cleaning surface, such as the floor. This label records identification information for identifying the stick vacuum cleaner 2. The identification information may be information that individually identifies the stick vacuum cleaner 2, or information that identifies the type, such as its model. The following describes how the user terminal 1 is installed in the terminal holder 20 connected to the handheld brush 25.

[0016] FIG. 2 is a diagram showing a state in which a user terminal 1 is installed on a stick vacuum cleaner 2 in this embodiment. In this embodiment, as shown in FIG. 2, the user terminal 1 is held in a terminal holder 20 connected to a handheld brush 25. The handheld brush 25 is connected to the vacuum cleaner main body 21. In this case, the back of the user terminal 1 faces the cleaner head 24, and the screen side faces the user's head. As a result, the user can check cleaning information displayed on the user terminal 1. For example, AR information showing the cleaning status of the stick vacuum cleaner 2 and a cleaning area map showing a unit cleaning area such as a room or floor are displayed. More preferably, these are displayed on a single screen. As an example of this display on a single screen, the cleaning area map is superimposed on the AR information. In this way, a unit cleaning area is an area that constitutes a cleaning area to be cleaned, such as a house, and the unit cleaning area may be the cleaning area.

[0017] Furthermore, when a sign is provided, the user terminal 1 is held in the terminal holder 20 so that its own camera can capture an image of it. It is desirable that the orientation of the terminal holder 20 is variable, and that the direction in which the user terminal 1 faces can also be changed accordingly. Furthermore, the camera of the user terminal 1 captures an image of the floor surface to be cleaned by the stick vacuum cleaner 2.

[0018] In this embodiment, the cleaning management process is executed under conditions such as the user's operation on the user terminal 1 and an identification process using the sign identification information photographed by the user terminal 1. That is, the display of the above-mentioned AR information and cleaning area map is executed. In the following Examples 1 to 3, which are specific examples of this embodiment, the display process of the AR information and cleaning area map will be described in detail. Furthermore, in Examples 4 to 6, the structure of the terminal holder 20 will be described in detail. Furthermore, in Example 7, a modified example of the map creation process will be described. Furthermore, in Example 8, an example of the display form of AR information and cleaning-related information including AR icons and an example of dynamically changing the display form will be described. Note that it is possible to combine at least two of each example. [Example]

[0019] Example 1 is an example in which AR information and a cleaning area map are displayed by a single user terminal 1 as part of a cleaning management process. Example 1 also shows an example in which cleaning can be performed like a game. FIG. 3 is a functional block diagram of the user terminal 1 in Example 1. The user terminal 1 includes a communication unit 101, an imaging unit 102, a cleaner identification unit 103, an operation status detection unit 104, a cleaning area map creation unit 105, an AR information creation unit 106, a cleaning management unit 107, a history information creation unit 108, an input unit 109, an output unit 110, and a memory unit 111.

[0020] First, the communication unit 101 has a function of communicating via a router or a network and a function of short-distance wireless communication. The photographing unit 102 is realized by a camera or the like and captures an image of the surroundings.

[0021] Furthermore, the vacuum cleaner identification unit 103 identifies the stick vacuum cleaner 2 using the image captured by the imaging unit 102. In Examples 1 and 2, the image of the sign unit captured by the imaging unit 102 is used to identify the product information recorded on the sign unit, and this is used to identify the vacuum cleaner. Note that an AR (Augmented Reality) marker, a two-dimensional code, or a nameplate can be used as the sign unit. Here, the nameplate records the product information and a two-dimensional code in which this information is recorded in a visible format. Note that the vacuum cleaner identification unit 103 may perform identification by user operation or by communicating with the stick vacuum cleaner 2. In the former case, product information for one or more stick vacuum cleaners 2 is stored in advance in the memory unit 111 of the user terminal 1, and the product information is identified by the user selecting from this.

[0022] Alternatively, the label unit may be omitted and the stick vacuum cleaner 2 may be identified using the shape of the vacuum cleaner head 24 photographed by the photographing unit 102. When the shape of the vacuum cleaner head 24 is used, the vacuum cleaner identification unit 103 will identify the type of stick vacuum cleaner 2. Furthermore, an RFID (Radio Frequency Identification) tag on which identification information is recorded may be used as the label unit. In this case, an RFID reader is provided in the user terminal 1 to read the identification information.

[0023] Furthermore, the operation status detection unit 104 detects the operation status of the stick vacuum cleaner 2, such as identifying cleaned positions, including the movement trajectory of the stick vacuum cleaner 2, using images of the floor surface captured by the image capture unit 102. For example, the operation status detection unit 104 calculates a coverage ratio, which indicates the ratio of cleaned positions to a unit cleaning area, as the operation status. Furthermore, the AR information creation unit 106 creates AR information using images of the floor surface, etc., captured by the image capture unit 102. More preferably, the AR information creation unit 106 creates AR information using the cleaned areas identified by the operation status detection unit 104. Here, the AR information is information created using augmented reality technology and is information indicating the cleaning status of the cleaned areas, etc. It is desirable that cleaning-related information including the AR information be provided to related parties, including users. Here, the cleaning-related information may be the AR information itself, or additional information may be added to it. Furthermore, the AR information may be a captured image (including a still image or a video), an animation indicating the cleaned areas, or text indicating the cleaned areas.

[0024] The AR information creation unit 106 also creates AR information including AR icons whose display is controlled in accordance with cleaning performed with the stick vacuum cleaner 2. Furthermore, the AR information creation unit 106 creates AR information that distinguishes between positions that have been cleaned with the stick vacuum cleaner 2 and positions that have not been cleaned. The cleaning management unit 107 then displays the boundary between the cleaned positions and the uncleaned positions using a gradation.

[0025] Furthermore, the cleaning management unit 107 executes cleaning management processing for cleaning performed by the stick vacuum cleaner 2 and preparation processing for this. First, as part of the cleaning management processing, the cleaning management unit 107 controls, that is, causes the output unit 110 to display cleaning-related information including AR information and a cleaning area map. At this time, the cleaning management unit 107 causes the AR information and the cleaning area map to be displayed within one screen of the output unit 110. At this time, the cleaning management unit 107 displays AR information including AR icons. Then, the cleaning management unit 107 displays map icons, which are displayed in conjunction with the display of the AR icons, in the cleaning area map. Furthermore, the cleaning management unit 107 changes the shapes of the AR icons and various map icons.

[0026] Furthermore, the preparation process includes the cleaning management unit 107 displaying an instruction to attach the user terminal 1 to the terminal holder 20, in other words, an attachment instruction and an attachment confirmation button for the stick vacuum cleaner 2, on the output unit 110. Furthermore, as part of the preparation process, the cleaning management unit 107 displays an instruction to select a cleaning area map of a unit area to be cleaned from a plurality of cleaning area maps stored in the memory unit 111 described below on the output unit 110.

[0027] Furthermore, the history information creation unit 108 creates history information indicating the cleaning history of the stick vacuum cleaner 2, using the operating status of the stick vacuum cleaner 2 detected by the operating status detection unit 104. In this embodiment, cleaning results in the cleaning management information 113 are created as the history information. Then, the cleaning management unit 107 displays the history information on the output unit 110. At this time, it is preferable that the cleaning management unit 107 displays the history information on the same screen as the cleaning-related information. Furthermore, the cleaning management unit 107 can also display the history information for a predetermined period of time. Furthermore, the history information creation unit 108 creates multiple pieces of history information for each cleaning area map, and the cleaning management unit 107 displays two or more pieces of the multiple pieces of history information in an overlapping manner on the output unit 110. Then, the cleaning management unit 107 switches between and displays these two or more pieces of history information depending on the tab selected. Note that the history information includes cleaning results including the movement trajectory of the vacuum cleaner.

[0028] The input unit 109 also accepts various operations from the user. The output unit 110 also outputs AR information, a cleaning area map, and the like. The input unit 109 and the output unit 110 may be integrated into one unit, such as a touch panel. The storage unit 111 also stores user management information 112, cleaning management information 113, cleaning area map information 114, guidance information 115, and game information 116. These pieces of information will be explained after the implementation example of the user terminal 1.

[0029] Next, an implementation example of the user terminal 1 will be described. The user terminal 1 can be realized by various computers such as a smartphone, tablet, mobile phone, or PC. FIG. 4 is a hardware configuration diagram of the user terminal 1 in Example 1. In FIG. 4, the user terminal 1 has a camera 11, a touch panel 12, a processing device 13, a communication device 14, and a storage device 15, which are connected to each other via a communication path.

[0030] First, camera 11 is an example of the imaging unit 102 and has an imaging function. Touch panel 12 is configured to serve both as input unit 109 and output unit 110 in FIG. 3, and receives user operations and displays various information such as AR information and a cleaning area map. Note that touch panel 12 may be configured as an input device and an output device.

[0031] Furthermore, the processing device 13 can be realized by a processor such as a CPU (Central Processing Unit), and executes calculations according to a cleaning management program 16 stored in a storage device 15, which will be described later.

[0032] Cleaning management program 16 is composed of a vacuum cleaner identification module 161, an operating status detection module 162, a cleaning area map creation module 163, an AR information creation module 164, a cleaning management module 165, and a history information creation module 166, each for one of its functions. Note that each of these modules may be implemented as an individual program or a partial combination. In particular, AR information creation module 164 may be implemented as a program with an augmented reality function separate from home appliance management. Furthermore, cleaning management program 16 may be implemented as one function of an integrated management application that manages multiple home appliances.

[0033] Here, the configuration shown in FIG. 3, which performs the same functions as each module, is as follows. Vacuum cleaner identification module 161: Vacuum cleaner identification unit 103 Operation status detection module 162: Operation status detection unit 104 Cleaning area map creation module 163: Cleaning area map creation unit 105 AR information creation module 164: AR information creation unit 106 Cleaning management module 165: Cleaning management section 107 History information creation module 166: History information creation unit 108 Therefore, the processing device 13 executes the processes of the vacuum cleaner identification unit 103, the operation status detection unit 104, the cleaning area map creation unit 105, the AR information creation unit 106, the cleaning management unit 107, and the history information creation unit 108 in accordance with the cleaning management program 16. The cleaning management program 16 can be stored in the storage device 15 described below or other storage media.

[0034] 3, and has a function of communicating via a router or a network and a short-range wireless communication. Storage device 15 corresponds to storage device 111 in FIG. 3, and stores cleaning management program 16, user management information 112, cleaning management information 113, cleaning area map information 114, guidance information 115, and game information 116. Therefore, storage device 15 may be realized by a main storage device such as a memory and a secondary storage device (storage medium) that is a so-called storage. The secondary storage device may be realized by an external hard disk drive (HDD), solid state drive (SSD), memory card, or the like.

[0035] Here, the user management information 112, cleaning management information 113, cleaning area map information 114, guidance information 115, and game information 116 will be described. FIG. 5 is a diagram showing the user management information 112 used in the first embodiment. The user management information 112 is information about the user of the user terminal 1 and related parties. Therefore, as shown in FIG. 5, the user management information 112 has the following items: user / related parties, points, address, contact information, and owned vacuum cleaner. Here, the user / related parties are information that identifies the user, such as the name and membership number of the user or related party. Furthermore, the points indicate the points or score that the user or related party has obtained through the game function. Note that the points may be stored as cleaning management information 113, which will be described later. In this case, the cleaning record items are managed for each user or related party, and the points are stored corresponding to the user or related party.

[0036] The address indicates the address of the user. The contact information indicates the contact information of the user, and may be a telephone number, email address, or the like. The owned vacuum cleaner is information that identifies the home appliance used by the user or related parties, i.e., the vacuum cleaner. Here, the user management information 112 can be used to manage and authenticate users in the cleaning management program 16. The user management information 112 may be omitted.

[0037] 6 is a diagram showing cleaning management information 113 used in Example 1. The cleaning management information 113 is information for managing the cleaning results of the stick vacuum cleaner 2. Therefore, as shown in FIG. 6, the cleaning management information 113 has the following items for each vacuum cleaner: user / person in charge, date and time, AR information, and cleaning results (cleaned position).

[0038] Here, the vacuum cleaner is used by the user and identifies the vacuum cleaner for which the presentation information is to be displayed, and product information such as identification information can be used. In the first embodiment, a stick vacuum cleaner 2 is recorded. If a user uses multiple vacuum cleaners, there will be multiple records in the cleaning management information 113.

[0039] Furthermore, the vacuum cleaner ID is identification information that identifies the vacuum cleaner in question. This vacuum cleaner ID is, for example, information equivalent to the identification information recorded in the label unit. Furthermore, the user / related parties refer to the user and related parties who perform the cleaning and who may be the recipients of the presented information. Typical examples of these users and related parties are the family or cohabitants of the user of the user terminal 1 (A in the figure) who is the main cleaner. Note that if a commercial vacuum cleaner is used as the target vacuum cleaner, the user's colleagues, managers, customers, etc. can be registered. However, these related parties are merely examples, and any person can be registered by the user's operation, etc. Furthermore, when it comes to users / related parties, it is desirable to distinguish users from other related parties. Note that related parties may be limited to users.

[0040] The date and time indicates the date and time when cleaning was performed or the date and time when the AR information was created. The AR information indicates a log of AR information created by the AR information creation unit 106. Note that this item may be omitted due to storage capacity limitations. The cleaning history (cleaned position) indicates the position cleaned by the corresponding stick vacuum cleaner 2 based on the AR information. Therefore, the cleaning history (cleaned position) has the items of coordinates and floor. Here, the coordinates are coordinates that indicate the area cleaned by the stick vacuum cleaner 2, i.e., the trajectory of movement. Furthermore, the floor indicates the floor that has been cleaned (or the floor that has not been cleaned) according to the coordinates. Here, the floor is an example of a unit cleaning area, and a room may also be used.

[0041] FIG. 7 is a diagram showing cleaning area map information 114 used in Example 1. The cleaning area map information 114 is map information that defines the layout of a place of use where the stick vacuum cleaner 2 is used, such as a house or an office, i.e., each unit cleaning area that makes up the place of use. For this reason, the cleaning area map information 114 has items for floor and coordinates that correspond to the place of use. The floor is an example of a unit cleaning area, and the coordinates are items that indicate the coordinates of the corresponding floor, i.e., location information. Each record in the cleaning area map information 114 indicates a cleaning area map for each unit cleaning area, such as a floor or a room.

[0042] FIG. 8 is a diagram showing guidance information 115 used in the first embodiment. The guidance information 115 is information showing various kinds of guidance, such as terms of use and guidance on how to use the device, including user operations, for executing the cleaning management process and preparation process in the first embodiment. For this reason, the guidance information 115 shows the operation details for each function for executing the cleaning management process and preparation process. For this reason, when the functions of the first embodiment are implemented by an app, the functions can be implemented as information about the app. In the first embodiment, the functions used are terms of use, map creation, cleaning (display of cleaning-related information), cleaning track (movement track), history information, maintenance, and support. Each of these functions is associated with a guidance content. Note that the cleaning track (movement track) and history information may be treated as a single function.

[0043] Here, the guidance content for each function will be explained. The terms of use prescribe terms of use that require consent from the user at the time of first use. Map creation indicates the procedure for creating cleaning area map information 114 for each unit cleaning area. Furthermore, cleaning (display of cleaning-related information) indicates the procedure for displaying cleaning-related information when cleaning. Furthermore, the movement trajectory and history information during cleaning indicate the procedure for displaying the movement trajectory during cleaning and the cleaning history, which are examples of cleaned positions, as cleaning-related information. Furthermore, maintenance indicates the procedure for performing various maintenance such as maintenance, replacing consumables, and repairs. Furthermore, support indicates the procedure for receiving support for repairs and purchasing consumables from the manufacturer or retailer of the stick vacuum cleaner 2. Note that each of these functions may be the manual for the corresponding vacuum cleaner, i.e., information summarizing how to use it, or a separate manual may be recorded.

[0044] Game information 116 is information used to realize game functions used in cleaning. Game information 116 includes AR icons used in the game, such as characters, point calculation rules, and AR icon placement rules. In this embodiment, the cleaning management unit 107 realizes the game functions, but a separate game unit (program) may also realize the game functions.

[0045] Next, a process flow for cleaning with the stick vacuum cleaner 2 in Example 1 will be described. Here, a process flow including a preparation process and a cleaning management process for the above-mentioned cleaning will be described. Figs. 9A and 9B are flowcharts showing a process flow for cleaning in Example 1. Below, the creation of a cleaning area map will be described as a preparation process, and the configuration of the user terminal 1 will be described using the configuration shown in Fig. 3.

[0046] First, in step S1 of FIG. 9A, input unit 109 accepts input from a user and activates a configuration for executing this processing flow. For example, cleaning management program 16 in FIG. 4 is activated. At this time, the user places user terminal 1 in terminal holder 20. Then, in step S2, cleaning management unit 107 determines whether the configuration for executing this processing flow, for example, cleaning management program 16, is being used for the first time. If it is being used for the first time, the process proceeds to step S3. If it is being used for the second or subsequent time, the process proceeds to step S14.

[0047] In step S3, cleaning management unit 107 reads the contents of the terms of use from guidance information 115 and displays them on output unit 110. FIG. 10 is a diagram showing display screen 1101 of output unit 110 that displays the terms of use in Example 1. Display screen 1101 displays the guidance contents of the terms of use. Below the display screen 1101, a checkbox that the user can enter if they confirm the contents and an agreement button that the user can press (designate) if they agree are displayed. When these are entered and pressed, the process transitions to step S4. Below these, text indicating that the user does not agree is displayed. When this is pressed, cleaning management unit 107 terminates the app (cleaning management program 16), that is, this processing flow. When this processing flow is terminated, cleaning management unit 107 may display a pop-up on display screen 1101 to confirm whether to terminate, and require the user to enter consent to the termination.

[0048] Furthermore, in step S4, the cleaning management unit 107 registers the user and the stick vacuum cleaner 2. To do this, the cleaning management unit 107 receives information identifying the user and the stick vacuum cleaner 2 from the user via the input unit 109. The cleaning management unit 107 then registers this as the user and the owned vacuum cleaner in the user management information 112. Furthermore, the cleaning management unit 107 registers this identifying information as the related parties and the vacuum cleaner in the cleaning management information 113. Here, it is desirable that the user and other related parties be recorded separately.

[0049] Note that the stick vacuum cleaner 2, i.e., the owned vacuum cleaner, may be registered using the photographing unit 102. The details of this will be explained below with reference to Figs. 11A to 11C. First, the cleaning management unit 107 starts up the photographing unit 102 and causes an instruction to read the label to be displayed on the display screen 1102a of the output unit 110. That is, the display screen 1102a of the output unit 110 shown in Fig. 11A for registering the stick vacuum cleaner 2 is displayed. The display screen 1102a displays instruction text 1102a-1 instructing the user to read the two-dimensional code on the product nameplate, which is an example of a label.

[0050] Then, according to instruction text 1102a-1, the user moves user terminal 1 and photographs the two-dimensional code recorded on the nameplate with photographing unit 102. In FIG. 11A, the photographed image is displayed in photographing area 1102a-2. Also in FIG. 11A, supplemental text 1102a-3 is linked to information that indicates the position of the product nameplate, which is an example of a sign unit. Then, vacuum cleaner identification unit 103 recognizes product information that identifies the vacuum cleaner from the photographed two-dimensional code.

[0051] Furthermore, when the two-dimensional code in the photographed area 1102a-2 is read and the product information of the vacuum cleaner is recognized, the cleaning management unit 107 causes the output unit 110 to display display screen 1102b. This display screen 1102b is shown in FIG. 11B. Display screen 1102b displays registration confirmation text 1102b-1, which requests confirmation of product information 1102b-2 recognized from the photographed two-dimensional code. Also, above this, product information 1102b-2 recognized from the two-dimensional code is displayed. This product information 1102b-2 can be information that identifies the stick vacuum cleaner 2, such as an individual identification number, as well as the model and product name. Furthermore, below this, an OK button 1102b-3 is displayed, which the user presses if they wish to register product information 1102b-2. On condition that this OK button 1102b-3 is pressed, cleaning management unit 107 registers the displayed product information 1102b-2 in the owned vacuum cleaners of user management information 112 and vacuum cleaners of cleaning management information 113.

[0052] Furthermore, if the product information cannot be recognized because the two-dimensional code in the photographing area 1102a-2 cannot be read, for example, the cleaning management unit 107 causes the output unit 110 to display a display screen 1102c. This display screen 1102c is shown in FIG. 11C. Display screen 1102c displays unconfirmed text 1102c-1, indicating that the target product, i.e., the vacuum cleaner, cannot be confirmed. Below that, an unconfirmed icon 1102c-2, symbolically indicating that the vacuum cleaner cannot be confirmed, is displayed. Below that, a reread button 1102c-3 and an end button 1102-4 are displayed. When the user presses the reread button 1102c-3, the cleaning management unit 107 causes the display screen 1102a to be displayed again, and the photographing unit 102 photographs the two-dimensional code. When the end button 1102-4 is pressed, the cleaning management unit 107 terminates the app (cleaning management program 16), i.e., this processing flow. When this processing flow is to be ended, the cleaning management unit 107 may display a pop-up on the display screen 1101 to confirm whether to end the processing flow, and may require the user to input consent to the end of the processing flow.

[0053] Furthermore, when registering a user, related parties can be registered in addition to the user. Here, the registration of the user and other related parties will be described. In this example, related parties such as family members are registered as players so that cleaning can be done like a game. First, the cleaning management unit 107 displays a display screen 1103 on the output unit 110 for registering a player, which is an example of a related party. An example of this display screen is shown in FIG. 12A. In FIG. 12A, the display screen 1103a displays explanatory text 1103a-1 encouraging registration. In this figure, since this is the first use, a sentence encouraging the registration of the first player to play is displayed. Note that this first player corresponds to the user described above.

[0054] Below that, a player icon area 1103a-2 is displayed, which displays the player icon of the player. In FIG. 12A, the player icon is blank because it has not yet been registered. Below the player icon area 1103a-2, a player name area 1103a-3 is displayed, which displays the player name entered by the user via the input unit 109. In FIG. 12A, "Soujitaro" is displayed as the player name. Below the player name area 1103a-3, text is displayed prompting the user to enter the player name, that is, the user's name. The player name is not limited to a person's name, and may include a family member's attribute such as "father" or an account name for another game.

[0055] Then, below these, a decision button 1103a-4 and a hold button 1103a-5 are displayed. When the decision button 1103a-4 is pressed, the cleaning management unit 107 transitions the display screen 1103a to a display screen 1103b for registering a player icon. At this time, the cleaning management unit 107 may register the input player name as the user in the user management information 112 and as the related person in the cleaning management information 113, or may register the player name together with the registration of the player icon.

[0056] Also, when the hold button 1103a-5 is pressed, the player name is registered without registering the player icon. For this purpose, the cleaning management unit 107 registers the input player name to the user in the user management information 112 and to the related person in the cleaning management information 113.

[0057] Next, the display screen 1103b will be described with reference to FIG. 12B. In FIG. 12B, a player icon type selection button 1103b-6 is superimposed on the same display as the display screen 1103a in FIG. 12A, and the enter button 1103a-4 has been removed. The player icon type selection button 1103b-6 is a button for selecting whether to use a default icon or image data available in the user terminal 1, such as a photograph, as the type of player icon. When the default icon is selected with the player icon type selection button 1103b-6, the cleaning management unit 107 reads out the default icon stored in the memory unit 111. When the photo library is selected with the player icon type selection button 1103b-6, the cleaning management unit 107 reads out image data, such as a photograph, from a so-called photo library stored in the memory unit 111. The cleaning management unit 107 then transitions the display screen 1103b to a display screen 1103c for selecting a player icon.

[0058] Next, the display screen 1103b will be described using FIG. 12C. Here, the same display screen appears whether the default icon or the photo library is selected for the player icon type selection button 1103b-6, and therefore both will be described together using FIG. 12C. In FIG. 12C, the player icon type selection button 1103c-7 is superimposed on the same display as the display screen 1103b of FIG. 12B. The player icon type selection button 1103c-7 displays a group of default icons or image data stored in the storage unit 111. When one of these default icons or image data is selected, the cleaning management unit 107 registers the selected default icon or image data with the user in the user management information 112 and the related person in the cleaning management information 113. Note that, in this registration, information identifying the default icon or image data may be registered instead of the default icon or image data itself. This concludes the description of step S4.

[0059] Furthermore, in step S5, cleaning management unit 107 causes output unit 110 to display instructions for use. FIG. 13 is a diagram showing display screen 1103 of output unit 110 that displays instructions for use in Example 1. Guidance 1104-1 is displayed on display screen 1104. This guidance corresponds to a function of guide information 115. When a function is designated by the user, cleaning management unit 107 displays the corresponding guidance content from guide information 115 within the area of ​​guidance 1104-1. When start button 1104-3 on display screen 1104 is pressed by the user, cleaning management unit 107 activates the designated function. In this processing flow, it is assumed that map creation is selected from guidance 1104-1, and map creation from step S6 onwards is activated. Note that map creation may be activated automatically on the condition that it is the first use.

[0060] Furthermore, the display screen 1103 displays progress information 1104-2 that indicates the progress stage of the display screen 1103. Furthermore, the example of Fig. 13 shows that the guidance 1104-1 is in a "picture story show" format, that is, the content is displayed using multiple still images or screen transitions. In this case, the progress information 1104-2 indicates the progress of the "picture story show."

[0061] Then, a map creation process, which is an example of a preparation process, is executed in steps S6 to S12. First, in step S6, the cleaning area map creation unit 105 causes the output unit 110 to display a display screen 1105 that explains how to create a map. In the first embodiment, two examples of this display screen 1105 are used. These will be described below.

[0062] FIG. 14A is a diagram showing a display screen 1105a (first example) for explaining the creation of a cleaning area map in Example 1. Note that, hereinafter, the cleaning area map will also be simply referred to as a map. The display screen 1105a displays explanatory text 1105a-1 explaining that this screen is a screen for creating a map. Below that, instruction text 1105a-2 is displayed, prompting the user to attach the smartphone (user terminal 1) in order to create a map, i.e., a cleaning area map. That is, the instruction text 1105a-2 instructs the user to attach the user terminal 1 to the terminal holder 20. Below that, an attachment image 1105a-3 is displayed, which is an image showing the user terminal 1 being attached to the terminal holder 20. By visually viewing the attachment image 1105a-3, the user can intuitively understand how to attach the user terminal 1. Note that the attachment image 1105a-3 may be a moving image or a still image, and may also be realized as an animation or illustration in addition to a live-action image.

[0063] In addition, supplemental text 1105a-4 is displayed below installation image 1105a-3, which contains a link to a solution for when the installation location or method is unknown. As shown in FIG. 14A, supplemental text 1105a-4 preferably includes a link to an instruction manual. The instruction manual may be stored in storage unit 111 or on a server of a manufacturer or the like on the Internet. In the former case, the instruction manual may be stored as independent information, or may be stored as part of user management information 112 or guidance information 115.

[0064] Also, below the supplemental text 1105a-4, a skip button 1105a-5 is displayed. When this button is pressed, map creation is skipped and the process proceeds to step S14. Below this, an installation complete button 1105a-6 is displayed. When this button is pressed, the process proceeds to step S7, where the camera, that is, the photographing unit 102, is activated to create a map.

[0065] Next, a second example of the display screen 1105 will be described. Fig. 14B and Fig. 14C are diagrams showing display screens 1105b and 1105c (second example) for explaining the creation of a map in the first embodiment. In this example, the display screen 1105b transitions to the display screen 1105c. First, the display screen 1105b in Fig. 14B is a screen related to the creation of a map, and displays instruction text 1105b-1 prompting the user to create a map.

[0066] Additionally, a creation image showing an image of map creation is displayed below the instruction text 1105b-1. And below that, explanatory text 1105b-3 showing how to create the map is displayed. As a result, the user can understand that by cleaning or moving the vacuum cleaner along the wall of the floor as shown in the instruction text 1105b-1, a map will be created as shown by the solid line in the creation image 1105b-2. In this way, by cleaning or moving the vacuum cleaner along the wall, a cleaning area map showing the perimeter of a unit cleaning area such as a floor will be created.

[0067] The created image 1105b-2 may be a moving image or a still image, and may be realized as a live-action image, animation, illustration, etc. Furthermore, either the created image 1105b-2 or the explanatory text 1105b-3 may be used alone.

[0068] Also, below the explanatory text 1105b-3, a skip button 1105b-4 is displayed. When this button is pressed, map creation is skipped and the screen transitions to step S14. Below this, a back button 1105b-5 and a next button 1105b-6 are displayed. When the back button 1105b-5 is pressed, the screen returns to one of the screens before step S5. When the next button 1105b-6 is pressed, the screen transitions to display screen 1105c of FIG. 14C.

[0069] Next, display screen 1105c displays instruction text 1105c-1, indicating that this screen is a screen for creating a map and prompting the user to attach the smartphone to create the map. Below this, an attachment image 1105c-2 and a skip button 1105c-3, which are the same as attachment image 1105a-3 and skip button 1105a-5 on display screen 1105a in FIG. 14A, are displayed. Furthermore, below skip button 1105c-3, a back button 1105c-4 and a next button 1105c-5 are displayed. Pressing the back button 1105c-4 returns the display screen 1105b in FIG. 14B. Pressing the next button 1105c-5 transitions to step S7, where the camera, i.e., the photographing unit 102, is activated to create the map. This concludes the description of step S6.

[0070] Next, in step S7, the photographing unit 102 is activated and starts photographing in response to an instruction from the cleaning area map creation unit 105. Then, in step S8, the cleaning area map creation unit 105 causes the output unit 110 to display a display screen 1106 for map creation including images photographed by the photographing unit 102. That is, the cleaning area map creation unit 105 provides guidance for creating a cleaning area map. Then, in step S9, the cleaning area map creation unit 105 creates a cleaning area map in response to the user's operation of the stick vacuum cleaner 2 in accordance with the guidance. Steps S8 and S9 will be described below with reference to the display screen 1106 for map creation. In steps S8 and S9, as in step S6, two examples of the display screen 1106 are used. These will be described below.

[0071] First, a first example will be described with reference to FIGS. 15A to 15E. Here, as a prerequisite for creating a cleaning area map, a user is holding a stick vacuum cleaner 2 and is ready to operate it. Then, the cleaning area map creation unit 105 creates a display screen 1106a including an image captured by the image capture unit 102 and displays this on the output unit 110. FIG. 15A shows the display screen 1106a. The display screen 1106a displays operation text 1106a-1 explaining that creation of a cleaning area map can be started and including a prompt to confirm whether the vacuum cleaner suction head is included in the display screen 1106a. Here, the vacuum cleaner suction head refers to the vacuum cleaner head 24 of the stick vacuum cleaner 2. In the example of FIG. 15A, a vacuum cleaner 1106a-5 including the vacuum cleaner head 24 is displayed, allowing the user to continue operating the vacuum cleaner. If the vacuum cleaner 1106a-5 is not displayed, the user adjusts the terminal holder 20 or the like so that the vacuum cleaner 1106a-5 is displayed.

[0072] In addition, an AR information area 1106a-2 is provided below the operation text 1106a-1, and an AR image is displayed therein. This AR information is created by the AR information creation unit 106 using the image captured by the image capture unit 102. This AR information includes the floor surface 1106a-3, wall surface 1106a-4, and vacuum cleaner 1106a-5 to be cleaned. As will be described below with reference to FIG. 15B, it is desirable to display the wall surface 1106a-4 in order to create a cleaning area map by moving the stick vacuum cleaner 2 along the wall. Then, in response to an operation from the user on the input unit 109, such as completion of preparation for creation, the cleaning area map creation unit 105 transitions the display from FIG. 15A to FIG. 15B.

[0073] FIG. 15B shows a display screen 1106b for map creation in Example 1, transitioned from FIG. 15A. Operation text 1106b-1 is displayed on display screen 1106b, explaining that creation of a cleaning area map can be started, and including an instruction to move around a room, which is an example of a unit cleaning area, along a wall (wall surface 1106b-4). The user confirms this to start map creation. In other words, the user moves the stick vacuum cleaner 2 along the wall. At this time, it is desirable that cleaning area map creation unit 105 accepts an input from the user via input unit 109 instructing to start creation.

[0074] 15A, an AR information area 1106b-2 is provided below the operation text 1106b-1. The AR information area 1106b-2 displays AR information based on a captured image including the floor surface 1106a-3, wall surface 1106a-4, and vacuum cleaner 1106a-5 to be cleaned, similar to the display screen 1106a. When conditions such as an input to start creation are met, the cleaning area map creation unit 105 transitions the display from FIG. 15B to FIG. 15C and creates a cleaning area map. The AR information in the AR information area 1106a-2 and the AR information area 1106b-2 may be the image itself captured by the image capture unit 102.

[0075] FIG. 15C shows a display screen 1106c for map creation in Example 1, transitioned from FIG. 15B. The display screen 1106c displays operation text 1106c-1 explaining that a cleaning area map is being created and including operations for completing or canceling map creation. At this time, the cleaning area map creation unit 105 creates the cleaning area map using the cleaned positions identified by the operation status detection unit 104. Here, the cleaned positions indicate the movement trajectory of the stick vacuum cleaner 2, regardless of whether cleaning has been performed or not. The cleaning area map and its creation will be described in detail later.

[0076] Also, an AR information area 1106c-2 is provided below the operation text 1106c-1, similar to the display screens 1106a and 1106b. However, the AR information area 1106c-2 displays AR information including a captured image including the floor surface 1106c-3, wall surface 1106c-4, and vacuum cleaner 1106c-5 to be cleaned, as well as a movement trajectory 1106c-6 of the stick vacuum cleaner 2. Specifically, the movement trajectory 1106c-6 is displayed around the vacuum cleaner 1106a-5 in Fig. 15C.

[0077] This AR information is created by the AR information creation unit 106. In the example of Fig. 15C, it can be seen that the stick vacuum cleaner 2 has moved from bottom to top in the figure. Furthermore, the movement trajectory 1109c-6 may display the boundary between the cleaned position and the uncleaned position using gradation. Then, when the user taps on display screen 1106c while FIG. 15C is displayed, as indicated by operation text 1106c-1, cleaning area map creation unit 105 transitions the display from FIG. 15C to FIG. 15D.

[0078] FIG. 15D shows a display screen 1106d for map creation in Example 1, transitioned from FIG. 15C. This display screen 1106d is a screen for canceling map creation and editing AR information. For this reason, the display screen 1106d has operation buttons (1106d-6 and 1106d-7) added to the display screen 1106c. First, the back button 1106d-6 is a button for canceling map creation. When pressed by the user, the cleaning area map creation unit 105 cancels map creation. Furthermore, the color change button 1106d-7 is a button for changing the color of the AR information, particularly the movement trajectory 1106d-8, to the user's preference. When a color change instruction is input using the color change button 1106d-7, the AR information creation unit 106 changes the color of the movement trajectory 1106d-8. The operation buttons may include buttons other than the back button 1106d-6 and the color change button 1106d-7, such as buttons for changing brightness or shape.

[0079] Furthermore, the color of the AR icon or map icon may be changed using the color change button 1106d-7. To this end, the cleaning management unit 107 changes the color of the AR icon or map icon in response to an operation on the color change button 1106d-7. Furthermore, instead of or in addition to the color change button 1106d-7, a color change button 1106d-7 for changing the form of the AR icon or map icon may be used. In this case, the cleaning management unit 107 changes the form of the AR icon or map icon in response to an operation on the form change button. The form to be changed includes the color (including brightness, saturation, and tone), shape (including size), transparency, blurring, and other effects of each icon.

[0080] Furthermore, when completion of map creation is received from the user via the input unit 109, the cleaning area map creation unit 105 transitions the display from Fig. 15D to Fig. 15E. This transition may be performed depending on the position of the stick vacuum cleaner 2 detected by the operation status detection unit 104. For example, the condition may be that the stick vacuum cleaner 2 has returned to the start position of map creation.

[0081] 15E shows a display screen 1106e for map creation in Example 1, transitioned from FIG. 15D. The display screen 1106e is a screen for ending map creation. An operation text 1106e-1 for confirming the end of map creation is displayed on the display screen 1106e.

[0082] An AR information area 1106e-2 is provided below the operation text 1106e-1. The AR information area 1106e-2 includes a unit cleaning area input area 1106e-3, a created map icon 1106e-4, a continue button 1106e-5, and a done button 1106e-6.

[0083] The unit cleaning area input field 1106e-3 is an area for inputting the name of the unit cleaning area that the created cleaning area map targets. A room or floor can be used as the unit cleaning area, and the name can be easily understood by the user.

[0084] Furthermore, the created map icon 1106e-4 is an icon that schematically shows the created cleaning area map, where the rectangle represented by a thick line in the figure indicates the movement trajectory, and the interior of the rectangle containing this indicates the cleaning area map. Furthermore, the continue button 1106e-5 and the done button 1106e-6 are buttons for accepting from the user whether map creation is to continue or completed, respectively. When the continue button 1106e-5 is pressed, the cleaning area map creation unit 105 transitions to the screen of FIG. 15C or FIG. 15D and continues map creation. When the done button 1106e-6 is pressed, the cleaning area map creation unit 105 ends map creation. In other words, step S9 ends.

[0085] Next, a second example of step S9 will be described with reference to Figs. 15F and 15G. In the second example, as a premise for creating a cleaning area map, a user holds the stick vacuum cleaner 2 and performs cleaning. At this time, the cleaning area map creation unit 105 causes the output unit 110 to display display screen 1106f. This display screen 1106f corresponds to display screen 1106c of Fig. 15C in the first example. Display screen 1106f is shown in Fig. 15F.

[0086] The display screen 1106f displays explanatory text 1106f-1 explaining that a cleaning area map is being created. At this time, as in the first example, the cleaning area map creation unit 105 creates a cleaning area map using the cleaned positions identified by the operation status detection unit 104. Also, as in the first example, an AR information area 1106f-2 is provided below the explanatory text 1106f-1. The AR information area 1106f-2 displays a captured image including the floor surface 1106f-3, wall surface 1106-4, and vacuum cleaner 1106f-5 to be cleaned, as well as a movement trajectory 1106f-7 of the stick vacuum cleaner 2. Furthermore, the AR information area 1106f-2 displays a map-in-progress icon 1106f-6 indicating the creation status of the cleaning area map. The AR information creation unit 106 creates AR information including the captured image, movement trajectory 1106f-7, and map-in-progress icon 1106f-6.

[0087] The map creation icon 1106f-6 may be treated as information separate from the AR information. The map creation icon 1106f-6 includes a line segment indicating the movement trajectory of the stick vacuum cleaner 2 and a circle indicating the current location. Furthermore, a pause button 1106f-8 and a cleaning complete button 1106f-9 are displayed below the display screen 1106f. When these buttons are pressed by the user, the cleaning area map creation unit 105 suspends or terminates creation of the cleaning area map. Here, when creation of the cleaning area map of the unit cleaning area is terminated, that is, when the cleaning complete button 1106f-9 is pressed, the cleaning area map creation unit 105 transitions from FIG. 15F to FIG. 15G.

[0088] 15G shows a display screen 1106g for map creation in Example 1, transitioned from FIG. 15F. The display screen 1106g is a screen for ending map creation. An explanatory text 1106g-1 indicating the end of map creation is displayed on the display screen 1106g.

[0089] In addition, a unit cleaning area input field 1106g-2 is displayed below the explanatory text 1106f-1. This unit cleaning area input field 1106g-2 is an area for inputting the name of the unit cleaning area that the created cleaning area map covers. A room or floor can be used as a unit cleaning area, and the user can input a name that is easy for the user to understand. In addition, a created map display field 1106g-3 is displayed below that. A created map icon 1106g-4 that schematically shows the created cleaning area map is displayed in the created map display field 1106g-3. In the example of FIG. 15G, a cleaning area map has been created for a room (living room) that has eight corners.

[0090] Furthermore, a retake button 1106g-5 and a complete button 1106g-6 are displayed below the created map display area 1106g-3. These are buttons for receiving from the user whether they want to redo map creation or whether they have completed it, respectively. When the retake button 1106g-5 is pressed, the cleaning area map creation unit 105 transitions to the screen of FIG. 15F and creates the map. When the complete button 1106g-6 is pressed, the cleaning area map creation unit 105 ends map creation. In other words, step S9 ends. This concludes the explanation of steps S8 and S9.

[0091] In step S10, the cleaning area map creation unit 105 determines whether map creation for the unit cleaning area for which the map is to be created, exemplified by each room in the house, has been completed. For this determination, for example, a completion instruction input by the user is used. As a result, if there are any rooms left to be created (incomplete), the process proceeds to step S11. If the process has been completed, the process proceeds to step S12.

[0092] In step S11, cleaning area map creation unit 105 accepts the selection of the next room from the user via input unit 109. That is, cleaning area map creation unit 105 accepts a request to create a map of the next room. In step S12, in response to an instruction from cleaning area map creation unit 105, photographing unit 102 ends photographing.

[0093] Furthermore, in step S13, the cleaning area map creation unit 105 registers the created cleaning area map in the cleaning area map information 114. This may be the end of the processing flow, but in the first embodiment, the description will continue assuming that the cleaning management processing is subsequently executed using the created cleaning area map.

[0094] In step S14, the cleaning management unit 107 causes the output unit 110 to display a home screen. First and second examples of this home screen will be described. First, FIGS. 16A and 16B show a first example of a home screen 1107 in the first embodiment. In this example, a screen showing a cleaning history is used as the home screen 1107. FIG. 16A is a diagram showing a home screen 1107a in the first example. The home screen 1107a displays explanatory text 1107a-1 explaining that the home screen 1107a displays the cleaning history. Below this, history image information 1107a-2 is displayed, which schematically shows the cleaning history for each unit cleaning area. In the figure, the white area represents a cleaning area map, within which the movement history is displayed. This history image information 1107a-2 uses floors (floors) as unit cleaning areas, and switching between these can be achieved using tabs. In the example of FIG. 16A, the history information for "1F" to "3F" is displayed overlapping each other. Among these, the cleaning history for "1F" is displayed, and when switching to "2F" using the tab, the display transitions to home screen 1107b shown in Fig. 16B. Fig. 16B will be explained later.

[0095] In addition, a game function selection button 1107a-3 is displayed below the history image information 1107a-2. The game function selection button 1107a-3 is a button for accepting whether or not to use the game function in step S15, which will be described later. Furthermore, below that, history text 1107a-4 is displayed. This represents the cleaning history as text information, and indicates the date and time when the cleaning was performed, the time, and the coverage rate for the unit cleaning area. In addition, a selection button is provided for each history. The cleaning management unit 107 creates a movement history within the cleaning area map of the history image information 1107a-2 according to this selected cleaning history. In the example of FIG. 16A, all cleaning history is selected. By displaying information according to such a selection, the user can intuitively grasp the cleaning history up to the desired period.

[0096] Also, below the history text 1107a-4, a cleaning start button 1107a-5 is displayed. When this cleaning start button 1107a-5 is pressed, the cleaning management process is executed.

[0097] Next, an example in which the home screen is switched using the tabs of the history image information 1107a-2 will be described. When the "2F" tab in the history image information 1107a-2 is selected, the cleaning management unit 107 transitions the screen to FIG. 16B. FIG. 16B is a diagram showing a home screen 1107b transitioned from FIG. 16A. Similar to the home screen 1107a, the home screen 1107b displays explanatory text 1107b-1, history image information 1107b-2, a game function selection button 1107b-3, history text 1107b-4, and a cleaning start button 1107a-5. However, the contents of the history image information 1107b-2 and history text 1107b-4 have been changed from 1F to 2F. In this way, the history image information 1107b-2 and history text 1107b-4 are changed in conjunction with the selection of the tab.

[0098] 16B, the cleaning history for 12 / 25 is not selected. Therefore, the cleaning history for 12 / 30 and 12 / 31 is shown in the history image information 1107b-2. This concludes the explanation of the first example.

[0099] Next, a second example of the home screen will be described with reference to FIG. 16C. In this second example, a cleaning game function is enabled. In this game, acorns are collected by cleaning. Acorns can be collected by virtually sucking in the acorns themselves and other monsters representing dust contained in the AR information.

[0100] For this reason, in FIG. 16C, a game character 1107c-1 is displayed on a home screen 1107c. Below that, collection points 1107c-2 are displayed, which indicate the number of acorns collected by the game function. The number of acorns is an example of game points. Displaying the points in this way motivates cleaning. Note that while it is desirable for the number of acorns to be the number of all participants, it may also be the number of a specific participant, such as a user.

[0101] Furthermore, below the collection point 1107c-2, a start cleaning button 1107c-3 and a display history button 1107c-4 are displayed. When the start cleaning button 1107c-3 is pressed, the cleaning management unit 107 starts the cleaning management process. For example, the cleaning management unit 107 displays the screen of FIG. 17C. This screen will be described later. When the display history button 1107c-4 is pressed, the cleaning management unit 107 displays the cleaning history, such as the history image information and history text of FIGS. 16A and 16B, on the output unit 110. Below these, a cleaning settings button 1107c-5 is displayed. When the cleaning settings button 1107c-5 is pressed, the cleaning management unit 107 displays a screen on the output unit 110 for inputting various settings related to cleaning. Then, in accordance with the input on this screen, the cleaning management unit 107 records the various settings related to cleaning. This concludes the explanation of step S14.

[0102] Next, the cleaning management process is executed from step S15 shown in FIG. 9B. This does not have to be executed consecutively with the map creation process. For example, when the stick vacuum cleaner 2 is started, the cleaning management process is executed using the stored cleaning area map. First, in step S15, the cleaning management unit 107 accepts from the user whether or not to use the game function. For example, an operation on the game function selection button 1107a-3 in FIG. 16A or the game function selection button 1107b-3 in FIG. 16B is accepted. Note that in the first embodiment, step S15 may be omitted, and the game function may be enabled by default.

[0103] Furthermore, in step S16, cleaning management unit 107 accepts a user selection, that is, a selection of a game player, via input unit 109. Furthermore, in step S17, cleaning management unit 107 accepts a selection of a map of a unit cleaning area to be cleaned from the user via input unit 109. Furthermore, in step S18, cleaning management unit 107 causes output unit 110 to display a terminal attachment instruction screen that prompts the user to attach user terminal 1 to terminal holder 20. Then, in step S19, cleaning management unit 107 accepts an input from the user via input unit 109 to confirm that user terminal 1 has been attached.

[0104] If the cleaning management process is executed immediately after the map creation process, steps S18 and S19 can be omitted. Steps S15 to S19 above can be considered as preparatory processing for the cleaning management process. Two examples of this preparatory processing will be described below using display screens.

[0105] 17A is a diagram showing a map selection screen 1108a for accepting map selection in step S17 in the first example of the preparation processing. The map selection screen 1108a is displayed on the output unit 110 by the cleaning management unit 107. The map selection screen 1108a displays explanatory text 1108a-1 explaining that this screen is a screen for selecting a cleaning area map.

[0106] Below that, created map icon 1108a-2 is displayed for each created cleaning area map. The user selects a cleaning area map by instructing (for example, pressing) one of these icons. That is, cleaning management unit 107 selects the corresponding cleaning area map from cleaning area map information 114. Also, below created map icon 1108a-2, decision button 1108a-3 for confirming the selection of the cleaning area map is displayed. When decision button 1108a-3 is pressed, cleaning management unit 107 transitions to step S18, and the screen transitions from FIG. 17A to FIG. 17B.

[0107] 17B is a diagram showing a terminal attachment instruction screen 1108b in step S18 in the first example. Similar to FIG. 14A, the terminal attachment instruction screen 1108b is a display screen for attaching the user terminal 1 to the terminal holder 20. The terminal attachment instruction screen 1108b displays instruction text 1108b-1 that prompts the user to attach the terminal. Below this, an attachment image 1108b-2 is displayed, which is an image showing how the user terminal 1 is attached to the terminal holder 20. The attachment image 1108b-2 may be a moving image or a still image, and may be realized as an animation or illustration in addition to a live action image.

[0108] Furthermore, below installation image 1108b-2, back button 1108b-3 and next button 1108b-4 are displayed. When back button 1108b-3 is pressed, cleaning management unit 107 transitions the screen from FIG. 17B to FIG. 17A. When next button 1108b-4 is pressed, cleaning management unit 107 transitions the processing flow to a process for confirming that user terminal 1 has been installed (step S19). At this time, cleaning management unit 107 desirably causes output unit 110 to display an installation confirmation screen for user terminal 1. This concludes the description of the first example of the preparation processing, and a second example will now be described.

[0109] 17C is a diagram showing a player / map selection screen 1108c for accepting both player selection (user selection) and map selection in steps S16 and S17 in the second example of the preparation process. Instruction text 1108c-1 prompting the user to select a player and a cleaning area map is displayed on the player / map selection screen 1108c.

[0110] Below that, a group of player icons 1108c-2 for selecting a player is displayed. A corresponding player is selected by the user's designation of the group of player icons 1108c-2. In other words, the cleaning management unit 107 identifies the player (stakeholder) corresponding to the selected player or icon.

[0111] Additionally, a map selection area 1108c-3 is displayed below the group of player icons 1108c-2. The map selection area 1108c-3 includes the name of the room that is the unit cleaning area and a created map icon that indicates the room. Then, by designating the triangle mark in the map selection area 1108c-3, the cleaning management unit 107 switches between the room name and the created map icon. As a result, the user can select the desired room. Furthermore, the map selection area 1108c-3b displays the most recent cleaning date and time of the room.

[0112] Furthermore, a back button 1108c-4 and a next button 1108c-5 are displayed below the map selection area 1108c-3. When the back button 1108c-4 is pressed, the cleaning management unit 107 changes the screen from Fig. 17C to Fig. 16C. When the next button 1108c-5 is pressed, the cleaning management unit 107 changes the screen from Fig. 17C to Fig. 17D.

[0113] FIG. 17D is a diagram showing a player / map selection result screen 1108d when a player (user selection) and a map selection are performed in steps S16 and S17 in the second example of the preparation processing. The player / map selection result screen 1108d displays the player icon 1108d-1 and player name 1108d-2 of the selected player. Below that, a unit cleaning area name 1108d-3 indicating the selected unit cleaning area is displayed. As a result, the user can confirm the selected player (Taro) and unit (living room).

[0114] Additionally, below unit cleaning area name 1108d-3, back button 1108d-4 and next button 1108d-5 are displayed. When back button 1108d-4 is pressed, cleaning management unit 107 transitions the display to FIG. 17C. When next button 1108d-5 is pressed, cleaning management unit 107 transitions the processing flow to a process for confirming that user terminal 1 has been installed (step S19). At this time, cleaning management unit 107 desirably displays an installation confirmation screen for user terminal 1 on output unit 110. This concludes the explanation of the second example of the preparation processing. In other words, the explanation of the preparation processing up to step S19 is completed.

[0115] Furthermore, in step S20, the photographing unit 102 is activated and begins photographing in response to an instruction from the cleaning management unit 107. Furthermore, in step S21, the operation status detection unit 104 begins detecting the operation status of the stick vacuum cleaner 2 and identifying the cleaned position using this. Then, in step S22, the cleaning management unit 107 causes the output unit 110 to display cleaning-related information including AR information. Here, the details of step S22 will be described using specific examples of cleaning-related information.

[0116] FIG. 18 is a flowchart showing details of the cleaning-related information display process (step S22) in the first embodiment. In step S2201, cleaning management unit 107 determines whether to use the game function. For example, cleaning management unit 107 can make this determination based on an input to game function selection button 1107a-3 in FIG. 16A or game function selection button 1107b-3 in FIG. 16B. As a result, if the game function is to be used, the process proceeds to step S2202. If the game function is not to be used, the process proceeds to step S2208.

[0117] Then, in step S2202, the cleaning management unit 107 identifies AR icons to be displayed and calculates the number of such icons. Here, the AR icons are included in the AR information and are used when executing the game function. The AR icons include dust character icons, and points can be obtained by virtually sucking these up with the stick vacuum cleaner 2. An example of this is the dust monster 1109a-2 in FIG. 19A.

[0118] The cleaning management unit 107 also calculates the number of AR icons according to the unit cleaning area indicated by the map selected in step S17. For example, the number can be calculated according to at least one of the attributes (room type), area, cleaning frequency, and latest cleaning time of the unit cleaning area. Furthermore, the number of AR icons may be determined according to the user selected in step S16.

[0119] In step SS2203, the cleaning management unit 107 determines the placement of the identified icons in the target unit cleaning area. That is, the placement of the AR icons in the cleaning area map is determined. The placement can be uniform placement in the unit cleaning area, placement of more icons in areas where dust tends to accumulate (near corners, areas far from entrances, etc.), or placement of more icons in areas that have not been cleaned much based on the cleaning history.

[0120] In step S2204, the cleaning management unit 107 creates AR information by superimposing a movement trajectory and an AR icon on the captured image. Here, the captured image is the one captured in step S20. The movement trajectory is the one identified in step S21. The AR icon is the one identified in step S2202. At this time, the history information creation unit 108 uses the movement trajectory to create cleaning records, which are an example of history information.

[0121] In step S2205, the cleaning management unit 107 superimposes the movement trajectory on the corresponding cleaning area map. That is, the movement trajectory can be displayed on the cleaning area map. The placement position of the AR icon may also be superimposed on the cleaning area map. As a result, cleaning-related information is created.

[0122] Then, in step S2206, the cleaning management unit 107 causes the output unit 110 to display cleaning-related information including an operating map icon showing the cleaning area map on which the AR information created in step S2204 and the movement trajectory identified in step S2205 are superimposed. The cleaning-related information may also include points acquired during the cleaning, cleaning history, and various explanations and instructions. As a result, the user visually checks the cleaning-related information and cleans with the stick vacuum cleaner 2 as if sucking in the AR icon. Furthermore, the user can thoroughly clean the unit cleaning area while checking the operating map icon of the cleaning-related information. At this time, the cleaning management unit 107 controls the display on the output unit 110 so that the AR icon appears to be sucked into the stick vacuum cleaner 2. In this way, the cleaning management unit 107 changes the display of the AR icon, such as by erasing it, in response to the user's operation on the stick vacuum cleaner 2.

[0123] In step S2207, the cleaning management unit 107 calculates points for the game function according to the cleaning status and adds them up. For example, when an AR icon is sucked in, the cleaning management unit 107 calculates points according to the AR points using the game information 116 and adds them to the points already acquired by the user. The cleaning management unit 107 may also calculate points according to the movement trajectory, that is, the amount of cleaning. Here, an example of steps S2202 to S2207, which are the processing when the game function is used, will be described using display screens. Specifically, the description will be given using display screens 1109a and 1109b of cleaning-related information in Example 1 shown in FIGS. 19A and 19B.

[0124] First, display screen 1109a is the screen when cleaning has started. An operating map icon 1109a-1 is displayed on display screen 1109a. Because this screen is for when cleaning has started, the operating map icon 1109a-1 does not display a movement trajectory. Also, a dust monster 1109a-2, which is a character representing dust, is displayed as an AR icon. The user can enjoy cleaning by operating the stick vacuum cleaner 2 as if virtually sucking up the dust monster 1109a-2.

[0125] Furthermore, the vacuum cleaner 1109a-3 in the display screen 1109a indicates the vacuum cleaner head of the stick vacuum cleaner 2 included in the captured image. Furthermore, the point display area 1109a-4 indicates the cumulative value of points calculated by the corresponding cleaning, such as when the dust monsters 1109a-2 are virtually sucked in. Here, in this example, when a certain number of dust monsters are sucked in, an acorn can be obtained, and the number of acorns obtained is displayed as points.

[0126] Then, when the user operates the stick vacuum cleaner 2 and the dust monster 1109a-2 is sucked in, the cleaning management unit 107 transitions the display from FIG. 19A to FIG. 19B. The display screen 1109b shown in FIG. 19B is a screen that has been changed from the display screen 1109a by the cleaning management unit 107 in response to the operation and operation of the stick vacuum cleaner 2. Specifically, the dust monster 1109a-2 has been erased from the display screen 1109a, and a movement trajectory has been added to the operating map icon 1109b-1. Furthermore, as the stick vacuum cleaner 2 moves, a movement trajectory 1109b-4 is displayed around the vacuum cleaner 1109b-2. This concludes the description of the specific example of steps S2202 to S2207 in which the game function is used. Returning to FIG. 18, the processing in the case where the game function is not used will be described.

[0127] In step S2208, cleaning management unit 107 creates AR information by superimposing the movement trajectory on the captured image. Here, the captured image is the one captured in step S20. The movement trajectory is the one identified in step S21. In step S2209, cleaning management unit 107 reads a cleaning area map of the unit cleaning area being cleaned from storage unit 111. In step S2209, cleaning management unit 107 superimposes the movement trajectory on the cleaning area map of the unit cleaning area being cleaned. That is, the movement trajectory can be displayed on the cleaning area map. The placement position of the AR icon may also be superimposed on the cleaning area map. At this time, similar to step S2206, history information creation unit 108 uses the movement trajectory to create cleaning results, which is an example of history information.

[0128] Then, in step S2210, cleaning management unit 107 causes output unit 110 to display cleaning-related information including the AR information created in step S2208 and an active map icon showing the cleaning area map with the movement trajectory superimposed in step S2209. The cleaning-related information may also include a cleaning history and various explanations and instructions. As a result, the user can thoroughly clean the unit cleaning area while checking the active map icon in the cleaning-related information. This concludes the explanation of steps S2202 to S2207, which are the processing when the game function is used.

[0129] Next, an example of steps S2208 to S2210, which are the processing when the game function is not used, will be described using display screens. Specifically, the description will be given using display screens 1109c and 1109d of cleaning-related information in Example 1 shown in Figs. 19C and 19D. First, display screen 1109c is a screen during cleaning. Then, display screen 1109c displays cleaning history 1109c-1. In this example, cleaning time and coverage rate, which indicates the proportion of the area that has been cleaned relative to the unit cleaning area, are also displayed. Since these are both 0, it can be understood that this is the display when cleaning has started.

[0130] In addition, instruction text 1109c-2 is displayed below the cleaning history 1109c-1 to help the user terminal 1 take appropriate photos. In addition, the floor surface 1106c-3, wall surface 1106c-4, and vacuum cleaner 1106c-5 to be cleaned are displayed as captured images of cleaning-related information. These are the same as the floor surface 1106a-3, wall surface 1106a-4, and vacuum cleaner 1106a-5 when creating the map.

[0131] Additionally, the cleaning-related information displays a movement trajectory 1109c-8, which is AR information. Here, the movement trajectory 1109c-8 may display the boundary between cleaned positions and uncleaned positions using a gradation. Here, a sign 1109c-7 is reflected in part of the vacuum cleaner 1106c-5. This sign 1109c-7 corresponds to the marker of the instruction text 1109c-2.

[0132] Furthermore, an operating map icon 1109c-5 is displayed on the display screen 1109c. This operating map icon 1109c-5 is similar to the operating map icon 1109a-1 and the operating map icon 1109b-1. However, the operating map icon 1109c-5 includes the current position 1109c-10 of the stick vacuum cleaner 2. This allows the user to intuitively grasp the current cleaning position within the unit cleaning area. Also, a cleaning start button 1109c-9 is displayed on the display screen 1109c, and pressing this causes the cleaning management unit 107 to transition the display from FIG. 19C to FIG. 19D.

[0133] 19D is a screen that has been changed from display screen 1109c by cleaning management unit 107 in response to the operation and performance of stick vacuum cleaner 2. Display screen 1109d displays cleaning history 1109d-1, which is the same as cleaning history 1109c-1. In this cleaning history 1109d-1, the cleaning time and coverage rate have been counted up since cleaning started.

[0134] Similarly to display screen 1109c, display screen 1109d displays, as captured images of the cleaning-related information, a floor surface 1106d-2, a wall surface 1106d-3, and a vacuum cleaner 1109d-7 to be cleaned. Similarly to display screen 1109c, the cleaning-related information displays a movement trajectory 1109d-9, which is AR information. Here, too, movement trajectory 1109d-9 may display the boundary between cleaned positions and uncleaned positions using a gradation.

[0135] Furthermore, the display screen 1109d displays an operating map icon 1109d-4, similar to the display screen 1109c. This operating map icon 1109d-4 includes the current position 1109d-5 and movement trajectory 1109d-6 of the stick vacuum cleaner 2. In other words, the movement trajectory 1109d-6 has been added to the display screen 1109c. This has been added because the cleaning has progressed. This operating map icon 1109d-4 allows the user to intuitively grasp the current cleaning position and cleaning progress status within the unit cleaning area. Also, the display screen 1109d displays a pause button 1109d-10 and a cleaning end button 1109d-11. When the pause button 1109d-10 is pressed, the cleaning management unit 107 temporarily suspends acquisition of the operating status, and when pressed again, resumes this. Furthermore, when the cleaning end button 1109d-11 is pressed, step S22 ends. This concludes the explanation of FIG. 18, and we will return to FIG. 9B and explain step S23 and subsequent steps.

[0136] Furthermore, in step S23, cleaning management unit 107 determines whether cleaning of the corresponding unit cleaning area has ended. To do this, cleaning management unit 107 makes a determination by accepting an input from the user confirming that cleaning has ended, or by making a determination based on the amount or proportion of the movement trajectory. Then, in step S24, cleaning management unit 107 causes output unit 110 to display a display screen indicating that cleaning has ended. Below, an example of the cleaning end process in step S24 will be described using the display screen. Here, the description will be divided into a case where the game function is used (a case where steps S2202 to S2207 are executed) and a case where the game function is not used (a case where steps S2208 to S2210 are executed).

[0137] First, a case where the game function is used will be described. As cleaning progresses as described above in FIG. 19B, cleaning management unit 107 determines that cleaning of the corresponding unit cleaning area has been completed. In this case, cleaning management unit 107 transitions the display from FIG. 19B to FIG. 20A. For example, cleaning management unit 107 can determine that cleaning has been completed if the movement trajectory accounts for a predetermined percentage or more of the area of ​​the cleaning area map. Display screen 1110a in FIG. 20A displays confirmation text 1110a-1 confirming the completion of cleaning. Below that, an active map icon 1110a-2 including the movement trajectory is displayed. This active map icon 1110a-2 is displayed larger than active map icon 1109a-1 and active map icon 1109b-1, and also indicates the possibility of deviation from the actual movement trajectory.

[0138] Furthermore, below this operating map icon 1110a-2, a continue button 1110a-3 and an end button 1110a-4 are displayed. When the continue button 1110a-3 is pressed, the cleaning management unit 107 returns the display to FIG. 19B to continue cleaning. In other words, the process transitions to step S2206. When the end button 1110a-4 is pressed, the cleaning management unit 107 transitions the display from FIG. 20A to FIG. 20B.

[0139] Display screen 1110b shown in Fig. 20B is a screen that indicates that cleaning, that is, the game function, has ended. For this reason, display screen 1110b displays explanatory text 1110b-1 indicating that cleaning has ended. Below that, instruction text 1110b-2 is displayed that prompts the user to remove user terminal 1 from terminal holder 20. Below that, next button 1110b-3 is displayed. When this next button 1110b-3 is pressed, cleaning management unit 107 transitions the display from Fig. 20B to Fig. 20C.

[0140] The display screen 1110c shown in FIG. 20C is a screen that indicates that the cleaning has been completed. For this reason, the display screen 1110c displays evaluation text 1110c-1 indicating the evaluation of the cleaning. Below that, an active map icon 1110c-2 including the movement trajectory is displayed. Furthermore, a points display area 1110c-3 is displayed that indicates the number of acorns collected and the points awarded for sucking up dust monsters. These allow the user to understand the evaluation of the cleaning. Furthermore, a next button 1110c-4 is displayed below the points display area 1110c-3, and pressing this button ends step S24. This concludes the explanation of when the game function is used.

[0141] Next, a case where the game function is not used will be described. As cleaning progresses as described above in FIG. 19D, cleaning management unit 107 determines that cleaning of the corresponding unit cleaning area has been completed. Display screen 1110d shown in FIG. 20D is the screen to which cleaning management unit 107 has changed from display screen 1109d in response to operation of cleaning end button 1109d-11. Display screen 1110d displays explanatory text 1110d-1 including the cleaning status. Below that, an active map icon 1110d-2 including the coverage rate is displayed. Further below that, a next button 1110d-3 is displayed. When this next button 1110d-3 is pressed, step S24 ends. This concludes the description of steps S23 and S24.

[0142] Then, in step S25, cleaning management unit 107 registers various information about the performed cleaning in storage unit 111. For example, cleaning management unit 107 registers the created AR information and cleaning record including the movement trajectory in cleaning management information 113. Cleaning management unit 107 also registers the calculated points in game information 116. History information creation unit 108 also registers the cleaning record created in step S2204 or step S2208 in cleaning management information 113. At this time, the date and time corresponding to cleaning management information 113 are also registered.

[0143] This concludes the description of the first embodiment. In the first embodiment, the user can enjoy cleaning while using the game function, depending on the user's selection. In this way, motivation for cleaning can be ensured. However, the game function is not essential, and it is not necessary to provide the game function. Even in this case, in the first embodiment, cleaning can be performed while understanding the situation, and more appropriate cleaning can be performed. Furthermore, it is not necessary to not use the game function, and it is also possible to always use the game function. [Example]

[0144] In Example 2, a function of recommending a cleaning position is added to Example 1. Hereinafter, Example 2 will be described focusing on this point. First, the configuration and information of Example 2 are the same as those of Example 1, and therefore the description thereof will be omitted. Furthermore, the characteristic process of Example 2 is step S22. Therefore, the corresponding description in Figures 9A and 9B will be omitted, and the details of step S22 will be described.

[0145] FIG. 21 is a flowchart showing details of the cleaning-related information display process (step S22) in the second embodiment. Hereinafter, the second embodiment will be described with reference to FIG. 21. First, in step S2211, the cleaning management unit 107 identifies a recommended area in the unit cleaning area where cleaning is recommended. To do this, the cleaning management unit 107 uses the cleaning history in the cleaning management information 113. For example, an area that is cleaned less frequently based on the cleaning history is set as the recommended area. Here, an area that is cleaned less frequently is identified from a location that is cleaned less frequently than a predetermined threshold, a location that is in the top (for example, 10%) of the unit cleaning areas and is cleaned less frequently, a location where a predetermined period of time has passed since the last cleaning, etc.

[0146] Also, in the same way as in the first embodiment, in step S2201, the cleaning management unit 107 determines whether or not to use the game function. As a result, if it is to be used, the process proceeds to step S2212. If it is not to be used, the process proceeds to step S2219.

[0147] Then, in step S2212, the cleaning management unit 107 identifies AR icons to be displayed and calculates the number of AR icons. In the first embodiment, the number of AR icons is calculated according to the unit cleaning area, but in the second embodiment, the number can be calculated according to at least one of the area of ​​the recommended area identified in step S2211, the cleaning frequency, and the latest cleaning time. In the second embodiment, the attributes of the unit cleaning area (room type), the area, the cleaning frequency, and the latest cleaning time may also be used. Furthermore, the number of AR icons may be determined according to the user, as in the first embodiment.

[0148] Furthermore, in step SS2213, the cleaning management unit 107 determines the arrangement of the identified icons in the target unit cleaning area. That is, the arrangement of the AR icons in the cleaning area map is determined. This arrangement is preferably such that the number (density) of icons per unit area in the recommended area is as large as possible. For example, the cleaning management unit 107 allocates the calculated number of AR icons to the recommended area and the non-recommended area according to the ratio of the area of ​​the recommended area to the unit cleaning area. At this time, a density is determined in advance for the recommended area and the non-recommended area, and the number of AR icons is allocated using this density. The density of the recommended area and the non-recommended area is recorded in the game information 116.

[0149] Furthermore, in step S2214, similar to step S2204 in the first embodiment, the cleaning management unit 107 creates AR information by superimposing the movement trajectory and an AR icon on the captured image. Also, in step S2215, similar to the first embodiment, the cleaning management unit 107 superimposes the movement trajectory on the corresponding cleaning area map. Also, similar to the first embodiment, at this time, the history information creation unit 108 uses the movement trajectory to create cleaning results, which are an example of history information.

[0150] Then, in step S2216, the cleaning management unit 107 creates cleaning-related information by superimposing the recommended area on each of the AR information and the cleaning area map. Here, the cleaning-related information is information including the AR information on which the recommended area is superimposed and an active map icon indicating the recommended area. Note that the recommended area may be superimposed on either the AR information or the active map icon.

[0151] In response to this, in step S2217, cleaning management unit 107 causes the cleaning-related information created in step S2216 to be displayed on output unit 110. An example of this display will be described using cleaning-related information display screens 1111a and 1111b shown in Figures 22A and 22B.

[0152] First, a display screen 1111a in FIG. 22A is a screen displayed when cleaning starts. The display screen 1111a corresponds to the display screen 1109a (FIG. 19A) of the first embodiment. The display screen 1111a has a recommended area 1111a-6 and a recommended area 1111a-7 added to the display screen 1109a. The recommended area 1111a-6 is superimposed on the cleaning target (such as the floor) of the captured image in the AR information, and is displayed to be distinguished from other areas (non-recommended areas). As a result, it is easy to understand that the recommended area 1111a-6 is an area that needs to be cleaned. Note that the boundary between the recommended area 1111a-6 and the other areas may be displayed using a gradation. The recommended area 1111a-7 is superimposed on the active map icon, and indicates the recommended area within the entire unit cleaning area. As a result, the user can grasp the areas that need to be cleaned from a bird's-eye view.

[0153] 22B is a display screen transitioned from the display screen 1111a of FIG. 22A. That is, the display screen 1111b of FIG. 22B is a display screen corresponding to the display screen 1109b (FIG. 19B) of the first embodiment. Therefore, the operating map icon 1111b-1 to the movement trajectory 1111b-4 of the display screen 1111b are the same as the operating map icon 1109b-1 to the movement trajectory 1109b-4 of the display screen 1109b, respectively. Furthermore, a recommended area 1111b-5 and a recommended area 1111b-6 have been added to the display screen 1111b.

[0154] Here, the recommended area 1111b-5 is superimposed on the cleaning target (such as the floor) in the captured image in the AR information, and is displayed to be distinguished from other areas (non-recommended areas). As a result, it is easy to understand that the recommended area 1111b-5 is an area that needs to be cleaned highly. The boundary between the recommended area 1111b-5 and other areas may be displayed using a gradation. Furthermore, the recommended area 1111b-6 is superimposed on the active map icon, and indicates the recommended area within the entire unit cleaning area. As a result, the user can grasp the areas that need to be cleaned highly from a bird's-eye view. This concludes the explanation of step S2217, and we will return to FIG. 21 and explain the processing from step S2218 onwards.

[0155] In step S2218, similar to step S2207 in the first embodiment, the cleaning management unit 107 calculates points for the game function according to the cleaning status. For example, when an AR icon is sucked in, the cleaning management unit 107 calculates points according to the AR points using the game information 116. Here, more AR icons (for example, dust monsters) are arranged in the recommended area. Therefore, by giving priority to cleaning the recommended area, the user can obtain more points. As a result, it is possible to encourage the user to give priority to cleaning the recommended area. This concludes the description of steps S2212 to S2218, which are the processing when the game function is used.

[0156] Returning to FIG. 21, the process from step S2219 onwards, which is the process when the game function is not used, will be described.

[0157] In step S2219, the cleaning management unit 107 creates AR information by superimposing the recommended area and the movement trajectory on the captured image. Also, in step S2220, similar to step S2209 in the first embodiment, the cleaning management unit 107 superimposes the movement trajectory on the cleaning area map of the unit cleaning area being cleaned. At this time, similar to step S2214, the history information creation unit 108 uses the movement trajectory to create cleaning results, which is an example of history information.

[0158] Then, in step S2221, the cleaning management unit 107 displays cleaning-related information, including the AR information created in step S2219 and the operating map icon showing the cleaning area map on which the movement trajectory is superimposed in step S2210, on the output unit 110. Note that the cleaning-related information may also include a cleaning history and various explanations and instructions.

[0159] An example of this display will be described using cleaning-related information display screens 1112a and 1112b shown in Figures 23A and 23B. First, display screen 1112a in Figure 23A is a display screen corresponding to display screen 1109c in Figure 19C of Example 1. Therefore, cleaning history 1112a-1 to current location 1112a-10 on display screen 1112a are the same as cleaning history 1109c-1 to current location 1109c-10 on display screen 1109c, respectively. Furthermore, recommended area 1112a-11 and recommended area 1112a-12 have been added to display screen 1112a.

[0160] The recommended area 1112a-11 is superimposed on the cleaning target (such as the floor) in the captured image in the AR information and is displayed to be distinguished from other areas (non-recommended areas). As a result, it is easy to understand that the recommended area 1112a-11 is an area that is highly in need of cleaning. Note that the boundary between the recommended area 1112a-11 and other areas may be displayed using a gradation.

[0161] Furthermore, the recommended area 1112a-12 is superimposed on the operating map icon 1112a-5, and indicates the recommended area within the entire unit cleaning area, allowing the user to grasp, from a bird's-eye view, the areas that are most in need of cleaning.

[0162] 23B is a display screen transitioned from display screen 1112a in FIG. 23A. That is, display screen 1112b in FIG. 23B is a display screen corresponding to display screen 1109d (FIG. 19D) of Example 1. Therefore, cleaning history 1112a-1 to cleaning end button 1112ad-11 on display screen 1112b are the same as cleaning history 1109d-1 to cleaning end button 1109d-11 on display screen 1109d, respectively. Furthermore, recommended area 1112b-12 and recommended area 1112b-13 have been added to display screen 1112b.

[0163] Here, the recommended area 1112b-12 is superimposed on the cleaning target (such as the floor) in the captured image in the AR information, and is displayed to be distinguished from other areas (non-recommended areas). As a result, it is easy to understand that the recommended area 1112b-12 is an area that needs to be cleaned. The boundary between the recommended area 1112b-12 and other areas may be displayed using a gradation. Furthermore, the recommended area 1112b-13 is superimposed on the active map icon, and indicates the recommended area within the entire unit cleaning area. As a result, the user can grasp the areas that need to be cleaned from a bird's-eye view.

[0164] In the second embodiment, the recommended area is displayed on the display screen during cleaning, but may also be displayed on the display screens 1110a to 1110d in the process of ending cleaning. This concludes the description of FIG. 21, that is, the description of the second embodiment. In this embodiment, the recommended area is displayed, so that the user can easily grasp the areas that are most in need of cleaning. Also, in the second embodiment, the game function does not have to be disabled, and the game function may always be used. Furthermore, superimposing the recommended area on the AR information is not essential and may be omitted. In particular, the display of the operating map icon may express the density of the AR icon by a shade of color or the like. [Example]

[0165] In Examples 1 and 2, the cleaning area presentation process is performed by the user terminal 1 alone, but in Example 3, various functions are realized in cooperation with a cleaning management device 3 realized by a server or the like. Fig. 24 is a system configuration diagram of the cleaning management system in Example 3. In Fig. 24, the cleaning management system has a user terminal 1 held by a terminal holder 20 of a stick vacuum cleaner 2, and a cleaning management device 3, which are connected via a network 5. In this case, the user terminal 1 may be connected to the network 5 via a router 4, or may be connected via another device, or may be connected to the network 5 directly.

[0166] In the third embodiment, among the functions of the user terminal 1 described in the first and second embodiments, the functions and configurations executed by the cleaning management device 3 can be omitted. For example, at least one of the cleaner identification unit 103, the operation status detection unit 104, the cleaning area map creation unit 105, the AR information creation unit 106, the cleaning management unit 107, and the history information creation unit 108 can be omitted. Furthermore, at least one of the information stored in the memory unit 111 can be omitted.

[0167] Furthermore, the user terminal 1 and the cleaning management device 3 may be connected to the user terminal 1-a used by the relevant person described in the first embodiment via the router 4 and the network 5. The user terminal 1-a may be located in the same house as the user terminal 1 or outside the house, and may be connected to the user terminal 1 and the cleaning management device 3 via the router 4 or the network 5, and the cleaning results including points can be shared with the user terminal 1. The user terminal 1 and the stick vacuum cleaner 2 are the same as those in the first embodiment. The user terminal 1-a may have the same functions as the user terminal 1, or may be configured such that the information stored in the vacuum cleaner identification unit 103 to the cleaning management unit 107, the history information creation unit 108, and the memory unit 111 is omitted.

[0168] Next, a description will be given of the cleaning management device 3, which is a characteristic configuration of Example 3. The cleaning management device 3 is used by manufacturers and distributors of stick vacuum cleaners 2, and these distributors manage users as members. Next, a description will be given of the configuration of the cleaning management device 3.

[0169] First, the functional blocks of cleaning management device 3 will be described with reference to Fig. 24. Cleaning management device 3 has communication unit 301, cleaner identification unit 302, operation status detection unit 303, cleaning area map creation unit 304, AR information creation unit 305, cleaning management unit 306, history information creation unit 307, and storage unit 308. First, communication unit 301 connects to each user terminal and router 4 via network 5.

[0170] Furthermore, cleaner identification unit 302 to history information creation unit 307 execute the same processes as cleaner identification unit 103 to history information creation unit 108 in Examples 1 and 2, respectively. Storage unit 308 stores user management information 310, cleaning management information 311, cleaning area map information 312, guidance information 313, and game information 314. Each of these pieces of information will be described later.

[0171] Next, we will explain the hardware configuration of one implementation example of cleaning management device 3. Cleaning management device 3 can be realized by a computer such as a server or a cloud system. Fig. 25 is a hardware configuration diagram of cleaning management device 3 in Example 3. In Fig. 25, cleaning management device 3 has processing device 31, communication device 32, memory 33, and secondary storage device 34, which are connected to each other via a communication path.

[0172] First, the processing device 31 can be realized by a processor such as a CPU, and executes calculations in accordance with a cleaning management program 35 stored in a secondary storage device 34, which will be described later. The communication device 32 corresponds to the communication unit 301 in Fig. 24, connects to the network 5, and communicates with other devices such as the user terminal 1.

[0173] 24. Memory 33 and secondary storage device 34 store cleaning management program 35 and information used in processing by processing device 31. Secondary storage device 34 can be implemented as a so-called storage. Secondary storage device 34 stores cleaning management program 35, user management information 310, cleaning management information 311, cleaning area map information 312, guidance information 313, and game information 314. Secondary storage device 34 may be implemented as various storage media such as an external hard disk drive (HDD), solid state drive (SSD), or memory card, or may be implemented as a device separate from cleaning management device 3, such as a file server.

[0174] Cleaning management program 35 is composed of a vacuum cleaner identification module 351, an operating status detection module 352, a cleaning area map creation module 353, an AR information creation module 354, a cleaning management module 355, and a history information creation module 356, each for its respective functions. Note that each of these modules may be realized as an individual program or a partial combination. In particular, AR information creation module 354 may be realized as a program with an augmented reality function separate from the cleaning management program. Furthermore, cleaning management program 35 may be realized as one function of an integrated management application that manages multiple home appliances.

[0175] Here, the configuration shown in FIG. 24, which performs the same functions as each module, is as follows. Vacuum cleaner identification module 351: Vacuum cleaner identification unit 302 Operation status detection module 352: Operation status detection unit 303 Cleaning area map creation module 353: Cleaning area map creation unit 304 AR information creation module 354: AR information creation unit 305 Cleaning management module 355: Cleaning management section 306 History information creation module 356: History information creation unit 307 Therefore, the processing device 31 executes the processing of the vacuum cleaner identification unit 302, the operation status detection unit 303, the cleaning area map creation unit 304, the AR information creation unit 305, the cleaning management unit 306 and the history information creation unit 307 in accordance with the cleaning management program 35.

[0176] Here, cleaning management program 35 executes the same processes as cleaning management program 16 of user terminal 1. Note that cleaning management program 35 can be stored in secondary storage device 34 or other storage media.

[0177] Next, the user management information 310, cleaning management information 311, cleaning area map information 312, guidance information 313, and game information 314 will be described. FIG. 26 is a diagram showing the user management information 310 used in the third embodiment. The user management information 310 is information about the user of the vacuum cleaner and related parties. It has the same items as the user management information 112 in the first and second embodiments, but in comparison, it handles information about multiple users. For example, information about other family members such as user B is added.

[0178] 27 is a diagram showing cleaning management information 311 used in Example 3. The cleaning management information 311 is information for managing the cleaning results of multiple users using vacuum cleaners. The cleaning management information 311 has the same items as the cleaning management information 113 of Examples 1 and 2, but in comparison, it handles multiple stick vacuum cleaners. For example, information on stick vacuum cleaner x has been added.

[0179] FIG. 28 is a diagram showing cleaning area map information 312 used in Example 3. The cleaning area map information 312 is map information that defines the layout of a usage location, such as a house or an office, where multiple vacuum cleaners, such as the stick vacuum cleaner 2 and the stick vacuum cleaner x, are used, that is, each unit cleaning area that makes up the usage location. The cleaning area map information 312 has the same items as the cleaning area map information 114 of Examples 1 and 2, but in comparison, it handles information on multiple users (usage locations). For example, information on user B has been added.

[0180] 29 is a diagram showing guidance information 313 used in the third embodiment. The guidance information 313 is information showing various kinds of guidance, such as terms of use and guidance on how to use the device, including user operations, for executing the cleaning management process and preparation process in the third embodiment. The guidance information 313 has the same items as the guidance information 115 in the first and second embodiments, but in comparison, it handles a larger number of stick vacuum cleaners. For example, information on stick vacuum cleaner x has been added.

[0181] Next, a process flow for cleaning with a stick vacuum cleaner 2 in Example 3 will be described. Here, as in Examples 1 and 2, a process flow including a map creation process and a cleaning management process, which are examples of preparation processes for cleaning, will be described. Fig. 30 is a flowchart showing the process flow for cleaning in Example 3. Below, the configuration of the user terminal 1 will be described using the configuration shown in Fig. 3, and the configuration of the cleaning management device 3 will be described using the configuration shown in Fig. 24.

[0182] First, in steps S101 and S102, the user terminal 1 executes the same processes as steps S1 and S2 of the first embodiment. If it is determined in step S102 that this is the first use, the process proceeds to step S103, and if it is determined that this is the second or subsequent use, the process proceeds to step S111. Then, in step S103, the cleaning management unit 107 transmits a registration request for the user or related person to the cleaning management device 3 via the communication unit 101. This registration request includes the user / related person, address, contact information, and owned devices from the user management information 310.

[0183] In step S301, the communication unit 301 of the cleaning management apparatus 3 receives the transmitted registration request.

[0184] In step S302, the cleaning management unit 306 transmits the terms of use to the user terminal 1 via the communication unit 301. Here, the cleaning management unit 306 searches the guidance information 313 for the terms of use corresponding to the owned device for which registration is requested, and transmits them.

[0185] Furthermore, in step S104, the communication unit 101 of the user terminal 1 receives the terms of use, and the cleaning management unit 107 displays them on the output unit 110. The display screen at this time is as described in the first embodiment. The display screen in the third embodiment is the same as in the first and second embodiments. Then, when consent to the terms of use is received from the user via the input unit 109, in step S105 the cleaning management unit 107 transmits a confirmation notice to the cleaning management device 3 indicating that confirmation has been confirmed.

[0186] In response to this, when the communication unit 301 of the cleaning management device 3 receives the confirmation notification, in step S303 the cleaning management unit 306 performs registration processing in response to the registration request. Specifically, the cleaning management unit 306 registers the user / related person, address, contact information, and owned devices included in the registration request in the user management information 310.

[0187] Furthermore, in step S304, the cleaning management unit 306 determines whether the user management information 310 and cleaning management information 311 for the stick vacuum cleaner 2 have already been registered by another user terminal or another related party. If they have already been registered, the process proceeds to step S305. If they have not yet been registered, the process proceeds to step S306. Note that when proceeding to step S305, it is desirable for the cleaning management unit 306 to add the user / related party information registered in step S303 to the registered record. For this reason, step S303 may be executed after this step.

[0188] In step S305, the cleaning management unit 306 transmits the usage instructions to the user terminal 1 via the communication unit 301. To do this, the cleaning management unit 306 searches for the functions and guidance content of the corresponding stick vacuum cleaner 2 from the guidance information 313. The cleaning management unit 306 then transmits these as the usage instructions via the communication unit 301.

[0189] Then, when the communication unit 101 of the user terminal 1 receives the usage instructions, in step S110, the cleaning management unit 107 causes the output unit 110 to display the usage instructions. This display screen is the same as in step S5 of the first embodiment. Also, in step S109, the cleaning management unit 107 causes the output unit 110 to display a home screen, similar to step S14 of the first embodiment.

[0190] In step S306, the cleaning management unit 306 transmits instructions for use and a map creation instruction to the user terminal 1 via the communication unit 301.

[0191] Then, when the communication unit 101 of the user terminal 1 receives the usage instruction and map creation instruction, in response to this, in step S106, the cleaning management unit 107 causes the output unit 110 to display a usage instruction and map creation explanation screen. That is, the same processes as in steps S5 and S6 of the first embodiment are executed.

[0192] Furthermore, in step S107, a map creation process is executed. That is, the processes described in steps S6 to S12 of FIG. 9A in the first embodiment are executed. Then, in step S108, the cleaning area map creation unit 105 transmits a registration request including the created cleaning area map to the cleaning management device 3 via the communication unit 101. Note that the created cleaning area map may be stored in the storage unit 111 of the user terminal 1, and steps S108 to S308 may be omitted.

[0193] Furthermore, when the communication unit 301 of the cleaning management device 3 receives a registration request transmitted from the user terminal 1, in step S307 the cleaning management unit 306 registers the accepted cleaning area map in the cleaning area map information 312. Then, in step S308, the cleaning management unit 306 transmits a registration notification to the user terminal 1 via the communication unit 301 indicating that the cleaning area map has been registered.

[0194] Then, when the communication unit 101 of the user terminal 1 receives the registration notification, step S109 is executed. Furthermore, in step S111, a cleaning management process is executed. That is, the processes described in steps S15 to S24 of FIG. 9B in the first embodiment are executed. Of course, step S22 in this cleaning management process can be realized by the display process of cleaning-related information shown in FIG. 18 or FIG. 21.

[0195] Furthermore, in step S112, cleaning management unit 107 sends a data update request including various information resulting from the cleaning execution to cleaning management device 3 via communication unit 101. This information includes, for example, cleaning results and calculated points. Then, in step S309, communication unit 301 of cleaning management device 3 accepts the data update request. In response to this, in step S310, cleaning management unit 306 registers the various information included in the data update request in storage unit 308.

[0196] This concludes the explanation of the flowchart in FIG. 30. However, in addition to the processing described here, related information may be sent to the user terminal 1-a of the relevant party. The related information includes the cleaning record and calculated points included in the data update request. Furthermore, various display screens in the map creation processing of step S107 and the cleaning management processing of step S11 may be shared between the user terminal 1 and the user terminal 1-a. Furthermore, the user terminal 1 and the user terminal 1-a can access their own information among the user management information 310, cleaning management information 311, cleaning area map information 312, and guidance information 313 stored in the memory unit 308 of the cleaning management device 3.

[0197] This concludes the explanation of the third embodiment. According to the third embodiment, the above-mentioned process is executed in cooperation with the user terminal 1 such as a smartphone and the cleaning management device 3 such as a server. This allows appropriate resources to be used, and efficient processing can be achieved. Furthermore, the execution entity of each step shown in FIG. 30 is merely an example, and some of the steps may be changed to be executed by another device (the user terminal 1 or the cleaning management device 3).

[0198] The user terminal 1 and cleaning management device 3 in Examples 1 to 3 are examples of information processing devices for realizing the cleaning management system. Therefore, the cleaning management system may be configured to include either the user terminal 1 or the cleaning management device 3. [Example]

[0199] Next, a fourth embodiment relating to the structure of the terminal holder 20 will be described. Fig. 31 is a diagram schematically illustrating a side view of the stick vacuum cleaner 2 in the fourth embodiment when the terminal holder 20 is attached thereto. As also described in Fig. 1, the stick vacuum cleaner 2 has a vacuum cleaner body 21 and a vacuum cleaner head 24, which are connected via an extension tube 23. The vacuum cleaner body 21 is provided with a handle 22 that is held by the user. The lower part of the vacuum cleaner body 21 has a tubular or annular connector 211 for connecting to the extension tube 23. The stick vacuum cleaner 2 in the fourth embodiment has a communication function and other information processing functions, and may be linked to a user terminal 1, a cleaning management device 3, etc.

[0200] Here, the terminal holder 20 is detachably installed on the vacuum cleaner main body 21, more preferably on the connection part 211. For example, the terminal holder 20 is configured in a ring-like or tubular shape and is fixed via an adapter or the like that wraps around the connection part 211. The terminal holder 20 may also be attached to a part other than the extension tube 23 or the connection part 211. Furthermore, when the extension tube 23 of the stick vacuum cleaner 2 is removed and a replacement nozzle (tube) is provided, the terminal holder 20 may be connected to the nozzle. The replacement nozzle is a nozzle that is shorter than the extension tube 23, and includes a brush nozzle and a tapered nozzle. Furthermore, it is desirable for the replacement nozzle to have a cableless structure in which a cable for controlling the vacuum cleaner head 24 is omitted.

[0201] Next, Fig. 32 is an enlarged view of Fig. 31 showing the state in which the terminal holder 20 in Example 4 is attached. In Fig. 32, the terminal holder 20 is fixed to the connection part 211 via the attachment part 200. At this time, it is desirable to position the holding part 20-1 of the terminal holder 20 so that it is at an angle of approximately 100° from the direction of the connection part 211 and the extension tube 23. However, this angle can be finely adjusted by moving the holding part 20-1 or the arm part 20-2, which will be described later. This allows the imaging part 102 of the user terminal 1 to capture images of the cleaner head 24 and the floor surface to be cleaned.

[0202] Here, the terminal holder 20 has at least a holding part 20-1, an arm part 20-2 and a base 20-3, and is attached to the stick vacuum cleaner 2. This attachment is preferably performed via an attachment part 200 that the terminal holder 20 has.

[0203] First, the holding unit 20-1 securely holds the user terminal 1 by clamping it, for example. The base 20-3 and the attachment unit 200 are connected, and the terminal holder 20 is fixedly attached to the stick vacuum cleaner 2. The arm unit 20-2 connects the holding unit 20-1 and the base 20-3. In the fourth embodiment, the base 20-3 or the terminal holder 20 and the attachment unit 200 are configured separately. However, these may be configured as an integrated unit, or may be configured as an integrated unit or separable unit. In other words, the terminal holder 20 of the fourth embodiment may be configured with the holding unit 20-1, the arm unit 20-2, and the base 20-3, or may be configured with the attachment unit 200 added to these. This allows the terminal holder 20 to be shared with multiple vacuum cleaners, especially vacuum cleaners with different specifications such as size. As described above, in the fourth embodiment, the terminal holder 20 having the holding unit 20-1 is detachably attached to the vacuum cleaner body of the stick vacuum cleaner 2.

[0204] Next, an example of the structure of the mounting portion 200 according to the fourth embodiment will be described with reference to Fig. 33 and Fig. 34. Fig. 33 is a diagram showing a first example of the mounting portion 200 according to the fourth embodiment. Here, Fig. 33 is a view seen from the direction AA shown in Fig. 31. The same applies to Fig. 34.

[0205] In Fig. 33, the lower part of the terminal holder 20 is connected to the mounting part 200. The mounting part 200 has an upper mounting part 200-1, a lower mounting part 200-2a, and a lower mounting part 200-2b, and its cross section is configured to be annular. Here, the lower mounting part 200-2a and the lower mounting part 200-2b move in the direction of the arrows (left and right), and together with the upper mounting part 200-1, they sandwich the connection part 211. For this reason, the upper mounting part 200-1 and the lower mounting part 200-2a and the lower mounting part 200-2b are movably connected by pins.

[0206] The lower mounting portion 200-2a and the lower mounting portion 200-2b are fixedly connected with screws or nails, and a force is generated in the left direction in the figure. This causes the mounting portion 200 to tighten the connection portion 211. As a result, the terminal holder 20 connected to the mounting portion 200 is fixed to the connection portion 211.

[0207] Next, Fig. 34 is a diagram showing a second example of the attachment portion 200 in the fourth embodiment. In Fig. 34 as well, the attachment portion 200 is connected to the lower part of the terminal holder 20. This attachment portion 200 has an upper attachment portion 200-3, a lower attachment portion 200-4a, and a lower attachment portion 200-4b, and its cross section is configured to be annular. Here, the lower attachment portion 200-4a and the lower attachment portion 200-4b move in the direction of the arrows (left and right), and together with the upper attachment portion 200-3, they sandwich the connection portion 211.

[0208] Here, the upper mounting portion 200-3 is connected to the lower mounting portion 200-4a and the lower mounting portion 200-4b via springs that generate force in the tightening direction. In this way, the mounting portion 200 tightens the connection portion 211. As a result, the terminal holder 20 connected to the mounting portion 200 is fixed to the connection portion 211.

[0209] With the configuration of the fourth embodiment described above, the user terminal 1 can be fixedly held in the terminal holder 20. As a result, the user of the stick vacuum cleaner 2 can use the user terminal 1 and benefit from various information processing including information processing related to cleaning. [Example]

[0210] Next, a fifth embodiment, which is different from the fourth embodiment and relates to the structure of the terminal holder 20, will be described. The fifth embodiment shows a structure in which the terminal holder 20 is connected to a handy brush 25 that can be attached to and detached from the stick vacuum cleaner 2. The handy brush 25 is a type of accessory, and is tubular, but is a shorter pipe than the extension tube 23. The handy brush 25 may also be called a shelf brush, a round brush, a brush nozzle, or the like. Various nozzles without a brush may also be used. Such nozzles include a crevice nozzle and a water-absorbing nozzle. The stick vacuum cleaner 2 of the fifth embodiment has a communication function and other information processing functions, and may be linked to a user terminal 1, a cleaning management device 3, or the like.

[0211] FIG. 35 is a diagram schematically illustrating a side view of the stick vacuum cleaner 2 in Example 5 when the terminal holder 20 is attached thereto. As described in FIG. 1 , the stick vacuum cleaner 2 has a vacuum cleaner body 21 and a vacuum cleaner head 24, which are connected via an extension tube 23. The vacuum cleaner body 21 is provided with a handle 22 that is held by a user. A handy brush 25 is attached to the bottom of the vacuum cleaner body 21. The extension tube 23 is connected to the vacuum cleaner body via the handy brush 25. Note that "via the handy brush 25" includes the following aspects: (1) The handy brush 25 and the extension tube 23 may each be connected to the vacuum cleaner body 21, and the extension tube 23 may extend into the hollow portion of the handy brush 25; or (2) The extension tube 23 may be connected to the extension tube 23, and the handy brush 25 may be connected to the vacuum cleaner body 21.

[0212] Here, the connection between the terminal holder 20 and the handy brush 25 will be described. Fig. 36 is an external view showing the state in which the terminal holder 20 and the handy brush 25 are connected in Example 5. In Fig. 36, the terminal holder 20 is installed so as to be detachable from the handy brush 25. For this purpose, the terminal holder 20 is connected to the handy brush 25 via the attachment part 20-4, which is fixed (locked) and released (released) by the elastic force of a spring provided in the attachment part 20-4. Note that, in addition to the spring, the terminal holder 20 and the handy brush 25 may be fixed by a claw such as a so-called side release buckle, or may be fixed by generating frictional force by a groove provided in the handy brush 25.

[0213] Next, Fig. 37 is an enlarged view of Fig. 35 showing the state in which the terminal holder 20 in Example 5 is attached. In Fig. 37, the terminal holder 20 is fixed to the vacuum cleaner body 21 via the handy brush 25. At this time, it is desirable to position the holding part 20-1 of the terminal holder 20 so that it is at an angle of approximately 100° from the direction of the handy brush 25 and the extension tube 23. However, this angle can be finely adjusted by moving the holding part 20-1 or the arm part 20-2. This allows the image capturing unit 102 of the user terminal 1 to capture images of the vacuum cleaner head 24 and the floor surface to be cleaned.

[0214] Here, the terminal holder 20 has at least a holding part 20-1, an arm part 20-2, and an attachment part 20-4, and is attached to the stick vacuum cleaner 2 via the handy brush 25. In this way, in the fifth embodiment, an accessory such as the handy brush 25 is used to physically connect the user terminal 1 to the stick vacuum cleaner 2, so that the accessory can be used efficiently. [Example]

[0215] Next, a sixth embodiment will be described regarding the structure of the terminal holder 20. In the sixth embodiment, the holding portion 20-1, the arm portion 20-2, and the base 20-3 of the terminal holder 20 in the fourth and fifth embodiments are integrated together. In the sixth embodiment, the terminal holder 20-10 is referred to as the terminal holder 20-10.

[0216] First, Fig. 38 is a perspective view showing the terminal holder 20-10 in Example 6 connected to the handy brush 25. In Fig. 38, the terminal holder 20-10 is provided with upper ribs 20-11a, 20-11b and a lower rib 20-13 to hold the user terminal 1. The upper ribs 20-11a, 20-11b and the lower rib 20-13 make it difficult for the user terminal 1 placed on the terminal holder 20-10 to move toward the front right in the drawing, making it difficult for the user terminal 1 to come off the terminal holder 20-10. Note that it is sufficient to provide at least one of the upper ribs 20-11a, 20-11b and the lower rib 20-13.

[0217] Additionally, receiving portions 20-12a and 20-12b are provided inside the upper ribs 20-11a and 20-11b of the terminal holder 20-10 (receiving portion 20-12b is not shown due to a blind spot). These receiving portions 20-12a and 20-12b accommodate both sides of the user terminal 1. For this reason, receiving portions 20-12a and 20-12b are implemented as grooves or gaps in the thickness direction of the user terminal 1. This makes it less likely that the terminal holder 20-10 will interfere with switches (buttons) provided on the side of the user terminal 1. In other words, when the user terminal 1 is placed on the terminal holder 20-10, it is possible to prevent or suppress the switches from being pressed accidentally. For this reason, it is desirable to make the width of the grooves or gaps in receiving portions 20-12a and 20-12b slightly shorter than the thickness of the user terminal 1 (for example, the thickness of a typical smartphone). This allows the front and back surfaces of the user terminal 1 to be fixed with these, and the switches can be placed in the grooves or gaps and out of contact with the terminal holder 20-10.

[0218] Furthermore, the grooves and gaps in receiving portion 20-12a and receiving portion 20-12 make it difficult for user terminal 1 to come off terminal holder 20-10. Note that it is sufficient to provide either receiving portion 20-12a or 20-12b. Furthermore, it is also possible to provide either upper ribs 20-11a, 20-11b and lower rib 20-13 or receiving portions 20-12a and 20-12b.

[0219] The terminal holder 20-10 is connected to the handy brush 25 below (only a portion of the handy brush 25 is shown in the figure). The configuration of the terminal holder 20-10 will be described in detail below. Figure 39A is a side view showing the terminal holder 20-10 in Example 6 connected to the handy brush 25. The upper rib 20-11b and the lower rib 20-13 can be seen on the side of the terminal holder 20-10. In particular, the lower rib 20-13 extends upward as circled in the figure, which makes it difficult for the user terminal 1 to come off. Next, a top view (direction A) and a bottom view (direction B) of the terminal holder 20-10 will be described.

[0220] Fig. 39B is a top view showing the terminal holder 20-10 in Example 6 connected to the handy brush 25. In Fig. 39B, the upper rib 20-11a and the upper rib 20-11b each extend toward the center of the terminal holder 20-10 as circled in the figure, which makes it difficult for the user terminal 1 to come off.

[0221] Furthermore, receiving portions 20-12a and 20-12b are configured as grooves into which both ends of the user terminal 1 fit, preventing or inhibiting switch malfunction. Furthermore, receiving portions 20-12a and 20-12b are tapered (shaped like the letter "V" in the figure), meaning that receiving portions 20-12a and 20-12b are located approximately in the center of the "V" shape. As a result, the user terminal 1 can be more securely attached to receiving portions 20-12a and 20-12b, which are located in the center. Therefore, switches on the side of the user terminal 1 are positioned in the spaces of the grooves and gaps, preventing malfunction.

[0222] Furthermore, the terminal holder 20-10 is provided with an opening / closing section 20-14 realized by a hinge structure near the upper rib 20-11a. This opening / closing section 20-14 acts as an axis, allowing one end of the terminal holder 20-10 to open (rotate) in the direction of the arrow in the figure. As a result, the user terminal 1 can be attached (placed) or removed from the terminal holder 20-10. In this way, the rotation of one end of the terminal holder 20-10 allows the user terminal 1 to be easily attached or removed with a simple structure.

[0223] FIG. 39C is a bottom view showing the terminal holder 20-10 in Example 6 connected to the handy brush 25. In FIG. 39C, the terminal holder 20-10 is provided with a circular protrusion 20-15 (Ω-shaped within the dashed line in the figure). The circular protrusion 20-15 is made of an elastic material such as metal, and is fixedly attached to the handy brush 25 by its elastic force. Furthermore, in Example 6, a structure is provided for fixedly connecting the terminal holder 20-10 and the handy brush 25. This will be explained next.

[0224] 40A is a rear view showing the state in which the terminal holder 20-10 in Example 6 is connected to the handy brush 25. At the location shown in the CC cross section of this rear view, a claw portion is provided on the terminal holder 20-10, and by combining with a protrusion of the handy brush 25, a fixed installation is achieved.

[0225] Fig. 40B is a cross-sectional view taken along CC in Fig. 40A in Example 6. In Fig. 40B, the terminal holder 20-10 is provided with a claw portion 20-16. In contrast, the handy brush 25 is formed with a protrusion 25-1. The claw portion 20-16 and the protrusion 25-1 are interlocked with each other, preventing the claw portion 20-16 from easily coming off the protrusion. As a result, the handy brush 25 is fixedly attached to the terminal holder 20-10.

[0226] In the sixth embodiment described above, the terminal holder 20-10 is attached to the handy brush 25, but the present invention is not limited to this. For example, the terminal holder 20-10 may be attached to the vacuum cleaner body 21 or the extension tube 23. Furthermore, each of the above-described configurations may be provided in the terminal holder 20 as in the fourth or fifth embodiment. [Example]

[0227] Next, a seventh embodiment will be described, which shows a modified example of the map creation process. In the first to third embodiments, the cleaning area map is created by moving the stick vacuum cleaner 2 to which the user terminal 1 is attached. In the seventh embodiment, an example in which the cleaning area map is created by operating the user terminal 1 will be described.

[0228] FIG. 41 is a diagram for explaining the map creation process in Example 7. More specifically, FIG. 41 shows a display screen for explaining the operation of the user terminal 1 in creating a cleaning area map. Before explaining FIG. 41, the premise of Example 7 will first be explained. Example 7 uses the above-mentioned marker unit. However, the marker unit may be provided in a location other than the cleaner head 24. For example, it may be provided in the charging base 26, or more preferably in the base member 29. Furthermore, it is desirable to use a so-called two-dimensional code as the marker unit. Below, the map creation process of Example 7 will be explained with reference to FIG. 41.

[0229] First, the cleaning area map creation unit 105 causes the output unit 110 to display a display screen for starting the creation of a map. Unlike in the first to third embodiments, this display screen does not prompt the user to attach the user terminal 1, but rather causes the user to read a two-dimensional code, which is an example of a marker, as shown in FIG. 41(a). If the marker is successfully read, the cleaning area map creation unit 105 causes the output unit 110 to display proof of this (for example, a white marker). It is desirable to display the content displayed in the case of success, as shown in FIG. 41(a).

[0230] Next, in response to a user's operation or the like, the cleaning area map creation unit 105 causes the output unit 110 to display a display screen for explaining the creation of the map shown in Fig. 41(b). Here, when creating the map, the cleaning area map creation unit 105 causes the output unit 110 to display an AR image created by the AR information creation unit 106 by superimposing a marker on the captured image. Therefore, the cleaning area map creation unit 105 displays an explanation instructing the user to virtually place the marker of the AR image in a corner of a room (an example of a unit cleaning area) as shown in Fig. 41(b).

[0231] Next, in response to a user's operation or the like, the cleaning area map creation unit 105 causes the output unit 110 to display a display screen for explaining the creation of a map, as shown in FIG. 41(c). FIG. 41(c) shows the operation for creating a cleaning area map that combines multiple rooms. Here, it displays information about moving the location, that is, the room, and placing a marker. The middle section of FIG. 41(c) displays an image of the cleaning area map being created. It also displays information about how to include the area under an obstacle, such as a table, in the cleaning area without placing a marker. Furthermore, FIG. 41(c) displays information about how to create a cleaning area map that combines multiple rooms by tapping around the room in a certain direction (for example, clockwise or counterclockwise).

[0232] Then, in response to a user's operation or the like, the cleaning area map creation unit 105 causes the output unit 110 to display a display screen for explaining the creation of the map shown in FIG. 41(d). FIG. 41(d) displays that the cleaning area map is completed when the complete button is pressed. Then, when FIG. 41(d) is pressed, the cleaning area map creation unit 105 starts creating the cleaning area map. To this end, the photographing unit 102 is started and begins photographing in response to an instruction from the cleaning area map creation unit 105. Then, during the map creation process, the cleaning area map creation unit 105 displays information in accordance with the operation content shown in FIG. 41 described above. For example, the display of the markers that are AR information in FIGS. 41(a) and (b) and the image of the cleaning area map that is being created or has been created in FIGS. 41(c) and (d) are displayed. [Example]

[0233] Next, Example 8 is an example of a display form of AR information including AR icons and cleaning-related information, and an example of dynamically changing the display form. Thus, Example 8, like Examples 1 to 3, performs a display process of cleaning-related information including AR information. For example, in a specific example of Example 8, the recommended area of ​​Example 2 is displayed. However, like Examples 1 and 3, the display of the recommended area can be omitted, and Example 8 can be combined with any of Examples 1 to 3. Note that Example 8 will be described using substantially the same configuration and information as Examples 1 and 2. Therefore, the following description will focus on the differences, particularly the processing flow and display screen.

[0234] Fig. 42 is a flowchart for explaining the details of the display process in Example 8. The process up to this flowchart is executed in the same way as up to S20 in Fig. 9B. Then, in step S41, similar to step S21 in Fig. 9B, the operational status detection unit 104 detects the operational status of the stick vacuum cleaner 2. However, in step S41, the operational status detected includes the stick vacuum cleaner 2 (particularly the position of the cleaner head 24), the position that has been cleaned, the movement speed, the amount of dust collected, the remaining charge (or the amount of charge), and whether the cleaner head has been replaced. Furthermore, the cleaning performance history can also be used as the operational status.

[0235] In step S42, the cleaning management unit 107 creates display-related determination information based on the operating status. The creation of this determination information includes the calculation and creation of the following: Specify the AR icon to display. The distance between the AR icon and the stick vacuum cleaner 2 (particularly the vacuum cleaner head 24). The number of contacts between the AR icon and the stick vacuum cleaner 2 (specifically, the vacuum cleaner head 24). The distance of the movement trajectory of the stick vacuum cleaner (particularly the vacuum cleaner head 24). The area shown by this movement trajectory. The coverage rate of the recommended area by the movement trajectory. The amount of exercise and calories burned by cleaning. Power consumption. Furthermore, the operating status itself, such as the remaining charge, may be used as the determination information.

[0236] Furthermore, in step S43, the cleaning management unit 107 specifies the display format according to the determination information. To this end, the cleaning management unit 107 determines whether or not to display a predetermined AR icon and whether or not to change the display format of the AR icon, etc. Then, the cleaning management unit 107 specifies the AR icon to be displayed and the display format to be changed. In step S43, the AR icons to be displayed, their display formats, the number of icons, the movement trajectory and its display format, the cleaning history and its display format, etc. are specified. The determination in step S43 includes using a threshold value stored in advance in the storage unit 111. For example, if the distance between the AR icon and the stick vacuum cleaner 2 is within the distance threshold value, the display format of the AR icon is changed. Note that specifying the display format in Example 8 includes specifying the display format (format such as color and size) and whether or not to display a predetermined display object (such as an AR icon).

[0237] In response to this, in step S44, the cleaning management unit 107 causes the output unit 110 to display the display format determined in step S43. In this way, in the second embodiment, various displays are executed on the user terminal 1 for the user depending on the cleaning status. As a result, the user's motivation for cleaning can be improved. Below, specific examples of the eighth embodiment will be described, but these can be implemented in combination with each other. Furthermore, at least a part of steps S42 to S44 may be executed by the AR information creation unit 106, or may be executed in cooperation between the cleaning management unit 107 and the AR information creation unit 106.

[0238] <Example 1> In Specific Example 1, the position of the vacuum cleaner head 24 is detected as the position of the stick vacuum cleaner 2, and a dust monster, which is a type of AR icon, is displayed according to the detected position. Therefore, Specific Example 1 is premised on the game functions of Examples 1 to 3. Furthermore, in Specific Example 1, the display format of the dust monster is changed according to the positional relationship between the position of the vacuum cleaner head 24 and the dust monster. Details thereof will be explained below.

[0239] 43 is a flowchart showing details of the display process in specific example 1 of Example 8. In step S20, the photographing unit 102 is activated and starts photographing in response to an instruction from the cleaning management unit 107. In addition, in step S21, the operating status detection unit 104 starts detecting the operating status of the stick vacuum cleaner 2. These are the same processes as in Example 1, but as described above, the detection of the operating status in Example 8 is not limited to the cleaned position, whereas in specific example 1, the cleaned position, that is, the position of the vacuum cleaner head 24, which is an example of the position of the stick vacuum cleaner 2, is used. Here, the cleaning management unit 107 uses the detection and identification of the cleaning position by the operating status detection unit 104 as determination information.

[0240] Also, in step S401, the cleaning management unit 107 specifies, as the display format, a screen of the dust monster and the cleaner head 24 of the captured image taken by the image capture unit 102. Then, the cleaning management unit 107 causes the dust monster and the cleaner head 24 to be displayed on a screen of the output unit 110. This display is similar to the display screen 1109a of FIG. 19A in the first embodiment. That is, the dust monster 1109a-2 and the cleaner 1109a-3 are displayed on the same screen on the display screen 1109a. Here, the cleaner 1109a-3 in FIG. 19A includes the cleaner head 24 of the stick cleaner 2 included in the captured image.

[0241] Furthermore, the dust monster 1109a-2 is displayed to allow the user to acquire points in the game function. Therefore, the dust monster 1109a-2 acts as an inducement factor for cleaning, that is, as a motivation for pointing the vacuum cleaner head 24 at it. By displaying the dust monster 1109a-2 and the vacuum cleaner head 24 on the same screen, the user will naturally move the vacuum cleaner head 24 towards the dust monster 1109a-2 to clean. Note that although Specific Example 1 may end at step S401, it is desirable to execute the processing from step S402 onward in order to further increase motivation for cleaning.

[0242] In step S402, the cleaning management unit 107 calculates the virtual distance between the dust monster 1109a-2 and the cleaner head 24, i.e., the distance on the display screen, as determination information. To do this, the cleaning management unit 107 calculates the position of the cleaner head 24 (vacuum cleaner 1109a-3) on the display screen based on the position where cleaning has been completed. Then, the cleaning management unit 107 calculates the distance between the dust monster 1109a-2 and the cleaner head 24 on the display screen. This calculation is performed continuously while this flowchart is being executed.

[0243] In step S403, the cleaning management unit 107 determines whether the calculated interval is equal to or less than the interval threshold stored in the memory unit 111. In other words, it is determined whether the user's operation of the stick vacuum cleaner 2 has caused the cleaner head 24 to approach the interval threshold or less relative to the dust monster 1109a-2. If the result shows that the interval is equal to or less than the interval threshold (YES), the process proceeds to step S404. If the result shows that the interval is not equal to or less than the interval threshold (NO), the process proceeds to step S405.

[0244] In step S404, the cleaning management unit 107 changes the expression of the dust monster as a display format. In specific example 1, the dust monster 1109a-2 in Fig. 19A has a normal expression (for example, a poker face), but this expression is changed to a troubled expression (troubled face). The cleaning management unit 107 then displays the dust monster 1113a-2 with the changed expression and the vacuum cleaner 1113a-3 including the vacuum cleaner head 24 on one display screen 1113a of the output unit 110.

[0245] 44A is a diagram showing a display screen 1113a in specific example 1 of working example 8. In Fig. 44A, the display screen 1113a displays an operating map icon 1113a-1, a dust monster 1113a-2, a vacuum cleaner 1113a-3, and a point display area 1113a-4, similar to Fig. 19A. First, the operating map icon 1113a-1 shows the cleaned area, including the cleaning position and movement trajectory, according to the progress of cleaning, compared to the operating map icon 1109a-1.

[0246] Furthermore, the dust monster 1113a-2's facial expression has changed from that of the dust monster 1109a-2 due to the vacuum cleaner head 24 indicated by the vacuum cleaner 1113a-3 approaching within the distance threshold. In the example of FIG. 44A, the dust monster's facial expression has changed to a troubled one. However, the changes to the dust monster 1113a-2 are not limited to this, and may include a panicked expression, an aggressive expression, a change in color (warning color), a flashing display, a change in size, and so on. Furthermore, these changes may be combined. Note that, like the point display area 1109a-4, the point display area 1113a-4 shows the accumulated points calculated by the dust monster 1113a-2 virtually sucking in dust, etc., during the cleaning. Here, in this example, if a certain number of dust monsters are sucked in, an item called an acorn can be obtained, and the number of items obtained is displayed as points.

[0247] In step S405, the cleaning management unit 107 continues displaying the dust monsters while maintaining their facial expressions, which are the display format of the dust monsters. At this time, the distance between the dust monsters 1109a-2 and the vacuum cleaner 1109a-3, etc., is changed as appropriate in accordance with the movement of the vacuum cleaner head 24. Then, the process returns to step S403 and continues.

[0248] Furthermore, in step S406, it is determined whether the cleaner 1113a-3 and the dust monster 1113a-2 have come into contact on the display screen based on the distance calculated in step S402. In step S406, as in steps S402 and S403, the position of the cleaner head 24 (cleaner 1109a-3) on the display screen is calculated based on the cleaning position. Then, based on this position, the cleaning management unit 107 determines whether the cleaner head 24 has come into contact with the dust monster 1113a-2, that is, whether the distance between them has become zero.

[0249] As a result, if there is contact (YES), the process proceeds to step S407, whereas if there is no contact (NO), the process proceeds to step S408.

[0250] Also, in step S407, cleaning management unit 107 changes the display format of the dust monster so that it appears as if it is being sucked into cleaner head 24, and causes output unit 110 to display this. At this time, the dust monster changes its shape over time, as if it is being sucked into the suction port of cleaner head 24, and is eventually erased. As a result, cleaning management unit 107 ultimately causes output unit 110 to display a display screen from which the dust monster has been erased, as shown in FIG. 19B.

[0251] In step S408, the cleaning management unit 107 continues displaying the dust monster 1113a-2 with the troubled face as its display format. At this time, the distance between the dust monster 1113a-2 and the vacuum cleaner 1113a-3, etc., is changed as appropriate in accordance with the movement of the vacuum cleaner head 24. Then, the process returns to step S406 and continues. However, if this distance becomes greater than or equal to the distance threshold, it is desirable for the cleaning management unit 107 to return the dust monster's facial expression to the normal expression shown in FIG. 19A. In this case, the process returns to step S403.

[0252] In step S409, the cleaning management unit 107 determines whether the number of contacts between the dust monster and the cleaner head 24 is equal to or greater than the contact threshold value previously stored in the storage unit 111. If the number of contacts is equal to or greater than the contact threshold value (YES), the process proceeds to step S410. If the number of contacts is not equal to or greater than the contact threshold value (NO), the process proceeds to step S411.

[0253] Furthermore, in step S410, the cleaning management unit 107 adds, for example, 1 to the points, i.e., the number of items indicated by acorns, and displays them in the point display area 1109a-4 of FIG. 19A or the point display area 1113a-4 of FIG. 44A. For example, as in the point display area 1109b-3 of FIG. 19B, the number of items may be displayed numerically, or may indicate that all dust monsters have been sucked up, such as MAX. Then, in response to the user's operation of the stick vacuum cleaner 2, the process returns to step S401 again. The cleaning management unit 107 records the added number of items (points) in the user management information 112. As a result, the points in the user management information 112 are updated. Furthermore, the size of the point display area 1109b-3 itself may be increased or the color may be changed according to the added number of items.

[0254] In step S411, the cleaning management unit 107 continues the process while maintaining the number of points in the point display area 1109a-4 and the point display area 1113a-4, i.e., the number of items. In this case, too, the process returns to step S401 again in response to the user's operation on the stick vacuum cleaner 2.

[0255] As described above, in steps S402 to S410 of specific example 1, the display mode is specified and changed depending on the proximity situation. Note that specific example 1 may also be executed as follows. The determination information in step S21 may be an image or display of the cleaner head 24, or whether a preset display position of a dust monster is included within the display screen. Alternatively, the determination may be made based on whether a predetermined time has elapsed since the start of cleaning.

[0256] Furthermore, specific example 1 also includes the following modified example. In the above, when the number of contacts is equal to or greater than the contact threshold, the number of items is incremented, but an icon other than the dust monster is displayed. This may be performed instead of step S410, or may be performed in conjunction with it. This modified example will be described below.

[0257] In step S410, the cleaning management unit 107 causes the output unit 110 to display a dust monster other than the dust monster. That is, a dust monster is displayed in place of the sucked-in dust monster. FIG. 44B is a diagram showing a display screen 1113b in a specific example 1 (variant) of the eighth embodiment. In FIG. 44B, n dust monsters 1113b-1 to 1113b-n are displayed on the display screen 114a. In addition to the dust monsters 1113b-1 to 1113b-n, the display screen 1113b also displays an operating map icon 1113b-3, a vacuum cleaner 1113b-4, and a point display area 1113b-5. These are similar to the operating map icon 1113a-1, the vacuum cleaner 1113a-3, and the point display area 1113a-4 described above. However, the operating map icon 1113a-1 may be a graph showing the cleaning amount instead of a map.

[0258] These trash monsters 1113b-1 to 1113b-n appear when the player inhales the dust monster above the contact threshold, so their appearance probability is lower than that of the dust monster. The trash monsters can be treated as so-called rare characters. Furthermore, the points per trash monster 1113b-1 to 1113b-n, i.e., the number of items, will be greater than that of the trash monsters. Alternatively, it may be possible to obtain other, more valuable items.

[0259] Here, it is desirable that the number of dust monsters 1113b-1 to 1113b-n displayed on the display screen 1113b, which is one screen, is greater than the number of dust monsters, but this is not limited to this. Also, it is more desirable that n be about 2 to 4. It is also possible to display multiple dust monsters on one screen.

[0260] Furthermore, the change in display mode according to the proximity state in steps S402 to S410 may be applied to the dust monster. This will be described below with reference to FIG. 44C. First, if it is determined in steps S402 to S403 that the distance between the dust monster and the cleaner head is equal to or less than the distance threshold value, the process proceeds to step S404. In step S404, the cleaning management unit 107 causes the output unit 110 to display the display screen 1113c of FIG. 44C. That is, the dust monster is trembling uncontrollably, and its expression has changed to one of distress.

[0261] Furthermore, as a result of operating the stick vacuum cleaner 2, the cleaner head 24 approaches the dust monster and comes into contact with it. In this case, the cleaning management unit 107 causes the output unit 110 to display display screen 1113d in FIG. 44C. That is, the display transitions from display screen 1113c to display screen 1114d. As a result, in step S406, the cleaning management unit 107 determines that the cleaner head 24 has come into contact with the dust monster.

[0262] Then, in step S407, the cleaning management unit 107 causes the output unit 110 to display the display screen 1113e in which the contacted dust monster has been erased. At this time, points (number of items) are added to the point display area 1113b-5, and the cleaning management unit 107 records this result in the user management information 112. As a result, the points in the user management information 112 are updated. Furthermore, as described above, the size of the point display area on the display screen 1113e itself may be increased or the color may be changed according to the number of items added.

[0263] <Example 2> Next, Specific Example 2 realizes a display in a display mode that corresponds to the cleaning operation speed. To this end, Specific Example 2 describes an example in which the display mode is specified according to the movement speed of the stick vacuum cleaner 2 or the user terminal 1. Here, the processing content will be described using FIG. 42. First, in step S41, the operation status detection unit 104 detects the movement speed of the user terminal 1 or the cleaner head 24. To this end, an acceleration sensor or a speed sensor may be provided in the cleaner head 24 or the user terminal 1, and the operation status detection unit 104 may acquire this detection result. Alternatively, the movement speed may be detected by performing image analysis on images captured by the imaging unit 102. Furthermore, the operation status detection unit 104 may detect the position that has been cleaned for each hour.

[0264] Furthermore, in step S42, the cleaning management unit 107 uses the movement speed detected by the operation status detection unit 104 as determination information. When the moved position is used, the cleaning management unit 107 calculates the movement speed from the change in the cleaned position over time.

[0265] Furthermore, in step S43, cleaning management unit 107 determines which of the speed information stored in advance in storage unit 111 corresponds to the movement speed identified in step S42. Here, speed information is information that associates a display format with each movement speed (range). As a result, cleaning management unit 107 identifies a display format that corresponds to the identified movement speed. This display format includes the display format of the surrounding display area and the movement trajectory.

[0266] Then, in step S44, cleaning management unit 107 causes output unit 110 to display the surrounding display area and the movement trajectory in the display format determined in step S43. In step S44, at least one of the surrounding display area and the movement trajectory may be displayed in a display format according to the movement speed.

[0267] First, an example of the peripheral display area will be described with reference to FIG. 45. FIG. 45 is a diagram illustrating a display screen 1113a including a peripheral display area in specific example 1 of working example 8. FIG. 45 illustrates a display screen 1113a based on FIG. 44A. Therefore, the display screen 1113a of FIG. 45 displays a peripheral display area 1113s-5 in addition to an operating map icon 1113a-1, a dust monster 1113a-2, a vacuum cleaner 1113a-3, and a point display area 1113a-4. This is an area displayed along the outer edge of the display screen 1113a. Then, depending on the moving speed, either the color or the width of the peripheral display area 1113s-5 is changed. For example, the faster the moving speed, the redder the color. Furthermore, the wider the width, the faster the moving speed.

[0268] Next, an example of a movement trajectory will be described using FIG. 19B of the first embodiment. In FIG. 19B, the color, length, and width of each line segment of the movement trajectory 1109b-4 are changed according to the movement speed. For example, the color is made redder as the movement speed increases. Furthermore, the length of the line segments is made longer as the movement speed increases. Furthermore, the width of the line segments is made wider as the movement speed increases. Note that although FIG. 19B displays the movement trajectory 1109b-4 as a collection of multiple line segments, it may also be displayed as a surface. In this case, the color of the movement trajectory 1109b-4 and the length of the line segments are changed according to the movement speed. This change is the same as in the above example. This concludes the description of the second embodiment.

[0269] <Example 3> Next, in Specific Example 3, it is possible to realize a display in a display format according to the cleaning volume. For this purpose, in Specific Example 3, an example will be described in which the display format is specified according to the cleaning volume, such as the driving time of the stick vacuum cleaner 2, the moving distance, the cleaning area, the coverage rate of a predetermined area such as a unit cleaning area, etc., as the determination information. In Specific Example 3, the processing content will also be described using FIG. 42.

[0270] First, in step S41, the operational status detection unit 104 detects at least one of the following. The time elapsed since the start of cleaning for Stick Vacuum Cleaner 2. Cleaned positions by time for stick vacuum cleaner 2 (vacuum cleaner head 24).

[0271] The elapsed time is measured by implementing the operation status detection unit 104 as a timer. The cleaned position can be detected in the same way as in the first and second specific examples.

[0272] In step S42, the cleaning management unit 107 identifies the cleaning amount, which is the determination information, from the operation status detected by the operation status detection unit 104 as follows.

[0273] The cleaning management unit 107 specifies the elapsed time as the operating time of the stick vacuum cleaner 2. The cleaning management unit 107 also calculates the travel distance of the stick vacuum cleaner 2 by accumulating the amount of change (interval) in the cleaned position for each hour. The cleaning management unit 107 also calculates the cleaning area of ​​the stick vacuum cleaner 2 from the travel distance based on the cleaned position for each hour and the width of the cleaner head 24. Furthermore, the cleaning management unit 107 calculates the coverage rate from the cleaning area and the area of ​​a predetermined area such as a unit cleaning area. Here, the area of ​​a predetermined area such as a unit cleaning area can be specified from the cleaning area map information 114.

[0274] Furthermore, in step S43, cleaning management unit 107 determines to which of the amount information stored in advance in storage unit 111 the cleaning amount identified in step S42 corresponds. Here, amount information is information that associates a display format with each cleaning amount (range). Furthermore, as a result of this, cleaning management unit 107 identifies a display format that corresponds to the identified cleaning amount. This display format includes a display of the number of icons (points) that have been added. In other words, the number of icons increases as the cleaning amount increases. This can be achieved in the same way as in specific example 1. Note that cleaning management unit 107 records the number of items (points) that have been added in the points of user management information 112.

[0275] Then, in step S44, cleaning management unit 107 causes output unit 110 to display the surrounding display area and movement trajectory in the display format determined in step S43. In step S44, for example, the display is as shown in point display area 1109b-3 in FIG. 19B. Furthermore, the size of point display area 1109b-3 itself may be increased according to the number of added items. This concludes the explanation of specific example 3.

[0276] <Example 4> Next, a specific example 4 will be described, in which a recommended area is displayed as a display format similar to the second embodiment. In this case, cleaning results are used to create the recommended area, but the amount of dust collected and the remaining charge (or the amount of charge) may also be used. Details of this will be described below with reference to FIG. 43.

[0277] First, in step S41, the operating status detection unit 104 acquires the recommended area identified by the cleaning management unit 107 when starting cleaning, etc. At this time, it is desirable that the recommended area be identified using the previous cleaning record. Note that starting cleaning includes any one of steps S15 to S20 in FIG. 9B. Also, in step S42, as in specific example 1, the cleaning management unit 107 uses the detection and identification of the cleaning position by the operating status detection unit 104 as determination information.

[0278] Then, in step S43, cleaning management unit 107 creates a display screen including the created recommended area. Then, in step S44, cleaning management unit 107 displays this on output unit 110. The display screen is similar to that shown in FIG. 22A or FIG. 23A in the second embodiment.

[0279] In addition, in specific example 4, the recommended area can be identified using the dust collection amount and the remaining charge. These examples are explained below. In either case, the user terminal 1 can acquire the dust collection amount and the remaining charge from the stick vacuum cleaner 2 or the charging base 26 using a communication function or the like. First, an example in which the amount of collected dust is used will be described. In order to use the amount of collected dust, the stick vacuum cleaner 2 is provided with a dust collection sensor.

[0280] In this example, the cleaning management unit 107 calculates the amount of collected dust for each position using the results of detection by the dust collection sensor.

[0281] Here, the dust collection sensor can detect the weight of the dust collection chamber or the amount of dust sucked in per hour. Even when the former is used, the amount of dust collected per hour can be determined by calculating the amount of change in the weight of the dust collection chamber per hour. Then, the cleaning management unit 107 compares the date and time in the cleaning management information 113 with the amount of dust collected per hour to calculate the amount of dust collected for each coordinate of the cleaned position in the cleaning management information 113.

[0282] As a result, the cleaning management unit 107 identifies a recommended area based on the dust collection amount for each coordinate. For example, the recommended area is created by connecting coordinates that indicate the highest dust collection amounts within a unit cleaning area. As described above, in this example, the cleaning management unit 107 identifies a recommended area using the cleaning position information in the cleaning management information 113 and the dust collection amount detected by the dust collection sensor. As a result, it becomes possible to give priority to cleaning areas with a lot of dust.

[0283] When the dust collection amount is used, the display format, such as the color of the recommended area, may be changed depending on the dust collection amount for each position. For example, the position with a larger dust collection amount may be displayed in a more noticeable color such as red, a darker color, or a three-dimensional display or contour line display according to the dust amount.

[0284] Next, an example using the remaining charge will be described. When specifying the recommended area, the operating status detection unit 104 acquires the remaining charge of the stick vacuum cleaner 2 from the stick vacuum cleaner 2. Then, the cleaning management unit 107 determines whether the recommended area of ​​Example 2 or the recommended area specified based on the above-mentioned dust collection amount can be cleaned with the acquired remaining charge. For example, the cleaning area per unit charge is stored in the memory unit 111, and the cleaning management unit 107 can make this determination by comparing this with the area of ​​the recommended area.

[0285] If the result of this determination is that cleaning is possible, the recommended area is used as is. On the other hand, if cleaning is not possible, the cleaning management unit 107 narrows the recommended area to create a new recommended area. In this case, the original recommended area may be narrowed evenly, or positions with lower dust collection amounts or cleaning frequencies may be deleted to create a new recommended area.

[0286] In this example, the recommended area in Example 2 and the recommended area specified based on the dust collection amount described above are used, but their use can be omitted. That is, the cleaning management unit 107 calculates the cleanable area from the remaining charge, and sets the range of the area calculated from the current position of the stick vacuum cleaner 2 as the recommended area, or sets a unit cleaning area (room, etc.) with a relatively high coverage rate that can be cleaned in the calculated area as the recommended area. Note that a high coverage rate includes being able to clean the entire unit cleaning area. Also, a priority area (e.g., a floor) can be recorded in the cleaning area map information 114, and the priority area can be set as the recommended area. Furthermore, the charge amount may be used instead of the remaining charge amount. In this case, the user terminal 1 acquires the charge amount from the stick vacuum cleaner 2 or the charging stand 26, and the cleaning management unit 107 estimates the remaining charge amount from the charging time and the charge amount.

[0287] Furthermore, in specific example 4, it is desirable to output a response when the recommended area has been cleaned. To this end, cleaning management unit 107 compares the recommended area with the history information (cleaning results in cleaning management information 113) created by history information creation unit 108. As a result, if it is determined that an area equal to or greater than a predetermined standard has been cleaned, cleaning management unit 107 outputs words of praise. For example, it outputs words or phrases indicating a successful cleaning, such as "nice cleaning." In this case, cleaning management unit 107 may output text or images via output unit 110, or may output audio via a speaker provided in the user terminal.

[0288] <Example 5> Next, a fifth concrete example of displaying health information as a display format will be described with reference to FIG. 43. First, in step S41, the operating status detection unit 104 acquires the history information (cleaning record in the cleaning management information 113) created by the history information creation unit 108 when, for example, cleaning has been completed. At this time, it is desirable that the history information be identified using the most recent cleaning record. Note that it is desirable that the completion determination in step S23 of FIG. 9B be used to indicate when cleaning has been completed. Furthermore, in step S42, the cleaning management unit 107 determines that the cleaning completion display (step S24) in FIG. 9B has started to be displayed as determination information.

[0289] Then, in step S43, cleaning management unit 107 creates a display screen including the created recommended area. At this time, cleaning management unit 107 calculates the distance traveled during the most recent cleaning from the history information. Then, cleaning management unit 107 estimates the user's walking distance from the calculated distance traveled. In response to this, cleaning management unit 107 creates health information including the user's calorie expenditure and / or exercise amount according to the walking distance.

[0290] For this reason, it is desirable to record physical information including the user's weight in user management information 112. Then, cleaning management unit 107 can calculate the calories burned and / or the amount of exercise using the walking distance and physical information. As a result, health information related to the user's health and exercise can be created. Note that cleaning management unit 107 may be linked to a health management function (health management app) provided in user terminal 1. For example, a pedometer in the health management function may be used to calculate the calories burned and / or the amount of exercise during cleaning, or the health management function may use history information to output the calories burned and / or the amount of exercise during cleaning separately from other activities.

[0291] <Example 6> Next, a sixth concrete example of displaying the area to be cleaned will be described. This area to be cleaned includes a movement trajectory, a movement history (hereinafter simply referred to as a movement trajectory), and a cleaning area map. Here, various variations regarding the movement trajectory will be described.

[0292] (1) First, an example of managing the movement trajectory for each accessory of the stick vacuum cleaner 2 will be described. For this purpose, the cleaning performance of the cleaning management information 113 is recorded for each accessory. Here, the accessory refers to an accessory that has a suction port, such as a brush nozzle, a tapered nozzle, or the cleaner head 24.

[0293] For this purpose, the stick vacuum cleaner 2 may be provided with an identification device for identifying accessories, or the accessories may be identified in response to a user's instruction via the user terminal 1. Furthermore, the accessories may be identified based on image processing of the image captured by the image capturing unit 102. As a result, when the movement trajectory is displayed in Figures 19B, 19D, 20C, 20D, etc., the display will be in accordance with the attached accessories.

[0294] (2) Next, cleaning records in the cleaning management information 113 for identifying a movement trajectory are created in accordance with the movement of the user terminal 1. The cleaning management unit 107 detects the position of the user terminal 1 at each time using a position sensor, acceleration sensor, or the like that the user terminal 1 has, and records this as cleaning records in the cleaning management information 113. As a result, a movement trajectory is created in accordance with the movement of the user terminal 1. At this time, the user terminal 1 does not need to be placed in the terminal holder 20, and may be held in the user's pocket, for example.

[0295] (3) Next, the display of the cleaning area map will be described. In the created map icon 1106g-4 of FIG. 15G, a location that is difficult to clean, such as the back of a television or a table, is accepted, and the cleaning area map creation unit 105 identifies this as an area excluded from cleaning. Then, the cleaning area map creation unit 105 creates a cleaning area map that distinguishes the area excluded from cleaning from other areas. As a result, a cleaning area map including the area excluded from cleaning can be displayed in the created map icon 1106g-4. Furthermore, in order to create such a cleaning area map, the user terminal 1 may detect a three-dimensional position and designate this as the area excluded from cleaning.

[0296] Furthermore, by creating such a cleaning area map, a cleaning area map including the cleaning excluded area can be displayed during cleaning. For example, this can be done in Figures 19A, 19B, 19C, 19D, 20A, etc. Furthermore, when displaying a history such as that in Figures 16A and 16B, the cleaning excluded area can also be included in the display. Note that the cleaning area map may be displayed without including the cleaning excluded area.

[0297] (4) Naturally, the above-mentioned movement trajectory can be superimposed on each display screen of FIGS. 44A to 45 as history information. Furthermore, this movement trajectory itself may be recorded in the cleaning management information 113. At this time, the cleaning management unit 107 may compare the movement trajectory of the cleaning currently being performed with the movement trajectory that is the history information, and change the display format to display an uncleaned area indicating the difference on the output unit 110 (for example, by changing the color of the uncleaned area). Then, when the stick vacuum cleaner 2 (cleaner head 24) moves through this uncleaned area, the cleaning management unit 107 changes the display format, such as the color of the area that has been moved, and displays it.

[0298] As a result, cleaning can be performed by referring to past cleanings. Even more preferably, the cleaning management unit 107 may provide guidance (navigation) for cleaning by providing guidance information that changes the display format of the history information or uncleaned areas as time passes. As a result, it is possible to prevent missing or overlooking cleaning tasks.

[0299] This concludes the description of each specific example of the eighth embodiment. In the eighth embodiment, common functions that can be applied to all of the embodiments (each specific example) may be provided. The common functions will be described below.

[0300] First, the display of the cleaning area map, i.e., the display format of the active map icon, can be changed. That is, the cleaning management unit 107 controls whether or not the active map icon is displayed, and changes its size. To achieve this, the cleaning management unit 107 accepts a display instruction from the user and changes the display format of the active map icon in response to this display instruction.

[0301] Here, an example of controlling whether or not to display a display format will be described with reference to FIG. 46. FIG. 46 is a diagram showing an example of controlling whether or not to display an operating map icon 1114a-1 in Example 8. In FIG. 46, the operating map icon 1114a-1 is displayed on the display screen 1114a, but the cleaning management unit 107 erases the operating map icon 1114a-1 in accordance with an instruction to erase it. As a result, the cleaning management unit 107 displays the display screen 1114 on the output unit 110. Conversely, the cleaning management unit 107 can also display the display screen 1114a on the display screen 1114b by receiving an instruction to display the operating map icon.

[0302] The erase instruction or display instruction may be implemented by tapping the screen. The cleaning management unit 107 also enlarges or reduces the display of the active map icon 1114a-1 in response to a zoom-in / zoom-out instruction. This zoom-in / zoom-out instruction can be implemented by a pinch gesture. While AR icons are not shown in FIG. 46, AR icons (such as dust monsters) can be displayed on the display screen 1114a or the display screen 1114b. Such a change in display format can also be implemented in other embodiments. For example, this can be implemented on the display screens shown in FIGS. 19A, 19B, 19C, 19D, and 20A. Furthermore, this can also be implemented on the display screens shown in FIGS. 16A and 16B.

[0303] Also, as a common function, a function of displaying a drive instruction display screen that instructs the stick vacuum cleaner 2 to drive when starting cleaning will be described. More preferably, before displaying the display screens for cleaning in Example 8 such as the display screen 1113a in FIG. 44A to the display screen 1113a in FIG. 45, the cleaning management unit 107 displays a drive instruction display screen as shown in FIG. 47. Also, the drive instruction display screen may be displayed before displaying cleaning-related information (steps S2206, S2210, S2217, S2221). In Example 8, two examples of the drive instruction display screen are shown in FIGS. 47(a) and (b).

[0304] These Figs. 47(a) and (b) have in common that they display text instructing the user to turn on the vacuum cleaner, i.e., the stick vacuum cleaner 2. In Fig. 47(a), the numbers count down from "3," "2," and "1" as time passes in a countdown format. Furthermore, Fig. 47(a) also displays a game character for the game function. Furthermore, in Fig. 47(b), there is no countdown, and a drive instruction display screen is displayed on a single screen. Note that in Figs. 47(a) and (b), an AR icon such as a dust monster may be displayed on the display screen.

[0305] As a common function, the display screen (cleaning end display screen) for the cleaning end display (step S24) when cleaning is ended will be described. In the eighth embodiment, the display screens shown in Figs. 20A to 20D as described in the first embodiment are also displayed. Furthermore, as the cleaning end display, a display screen including a cleaning history as shown in Fig. 48 may be displayed. The display screen 1115a in Fig. 48 includes a cleaning end map icon 1115a-1. The cleaning end map icon 1115a-1 shows the cleaning history of the cleaning that has been performed.

[0306] Furthermore, although display screen 1110c in FIG. 20C is shown as a display screen for the cleaning end display showing points (number of icons), this is not limiting. A display screen different from display screen 1110c is shown in FIG. 49. FIG. 49 is a diagram showing display screen 1116a for the cleaning end display including points (number of icons) in Example 8. In FIG. 49, display screen 1116a displays acquired item display area 1116a-1, acquired item count display area 1116a-2, and game character 1116a-3.

[0307] First, the acquired item display area 1116a-1 displays items (acorns) acquired through cleaning. The acquired item count display area 1116a-2 displays the number of items acquired through cleaning, i.e., the number of items (30) displayed in the acquired item display area 1116a-1. Furthermore, the game character 1116a-3 displays the character used by the user in the game function.

[0308] In addition, in each of the embodiments including the eighth embodiment, the image capturing unit 102 of the user terminal 1 is used. However, the image capturing unit 102 may be configured as a device separate from the user terminal 1, such as a so-called wearable camera (regardless of type, such as a wristwatch type, eyeglass type, or type worn on the body). In this case, the image capturing unit and the user terminal 1 can communicate with each other.

[0309] This concludes the description of the eighth embodiment. According to the eighth embodiment, it is possible to improve motivation for cleaning and also to display in a display format that corresponds to the cleaning status. [Example]

[0310] Next, a ninth embodiment will be described, showing an example of cleaning support, such as creating a cleaning area map using a two-dimensional code and cleaning management processing. The ninth embodiment can be realized by using the same configuration and information as the first embodiment. For example, the functional block shown in FIG. 3 can be realized by the hardware shown in FIG. 4. Therefore, the processing of the ninth embodiment can be executed using the configuration of FIG. 3 below. That is, the processing of the ninth embodiment can be executed according to the cleaning management program 16.

[0311] 3 to 8, any configurations and information not mentioned below may be omitted. For example, the operating status detection unit 104 and the game information 116 may be omitted. Hereinafter, the ninth embodiment will be described, focusing on the differences from the above-mentioned embodiments.

[0312] Fig. 50A is a flowchart showing the process flow of the map creation process in Example 9. Here, steps up to step S8 are executed in the same manner as in Fig. 9A in Example 1, but these are not essential and other processes may be performed, or these may be omitted.

[0313] First, in Fig. 50A, similarly to Fig. 9A, the process transitions from step S6 to step S7. Also, in step S7, the photographing unit 102 is activated and begins photographing in response to an instruction from the cleaning area map creation unit 105. Then, in step S8, the cleaning area map creation unit 105 causes the output unit 110 to display a display screen 1106 for map creation including images photographed by the photographing unit 102. At this time, it is assumed that the photographing unit 102 is facing the cleaning target, such as the floor surface.

[0314] In step S1001, the cleaning area map creation unit 105 reads the recorded content of the two-dimensional code captured by the image capture unit 102 in response to a user's operation. The two-dimensional code is used to identify the position of the stick vacuum cleaner 2 or the user terminal 1. The two-dimensional code can be provided on the charging base 26, more preferably on the base member 29. The two-dimensional code may be provided horizontally or vertically with respect to the floor surface. The two-dimensional code may also be provided diagonally with respect to the floor surface. If the two-dimensional code is provided vertically or diagonally, the cleaning area map creation unit 105 and the cleaning management unit 107 will detect a plane when they operate.

[0315] A two-dimensional code may be provided on the floor surface to be cleaned. The two-dimensional code is an example of the above-mentioned marking section, and a barcode, RFID, AR marker, etc. may also be used.

[0316] The two-dimensional code may store any information as long as it can be used to estimate the relative position. For example, the two-dimensional code may store information indicating the operation mode of the stick vacuum cleaner 2. For example, the cleaning target (room, store, etc.) may be recorded. In this case, the stick vacuum cleaner 2 can be operated in normal mode if in a room, and in demo mode if in a store. Furthermore, the two-dimensional code may store identification information for the stick vacuum cleaner 2 and the two-dimensional code itself.

[0317] In step S1002, cleaning area map creation unit 105 detects the coordinates and direction of the cleaning target, for example, the floor surface, using the results of reading the two-dimensional code. Here, cleaning area map creation unit 105 displays on output unit 110 that the two-dimensional code has been read and that map creation processing can begin. In response, cleaning area map creation unit 105 accepts an instruction from the user via input unit 109 to start map creation.

[0318] In response to this, in step S1003, the cleaning area map creation unit 105 starts plane detection for the cleaning target using the image captured by the image capture unit 102. Here, the plane detection method will be explained. First, as a premise, as described above, the user terminal 1 is pointed in the direction of the floor surface that is the cleaning target. Then, the cleaning area map creation unit 105 acquires feature points from the image captured by the image capture unit 102 and detects a plane by connecting feature points that are at approximately the same height.

[0319] Here, plane detection requires about one second to acquire multiple feature points. Therefore, when the cleaning area map creation unit 105 receives an instruction to start map creation, it performs a countdown and displays it on the output unit 110, while the plane detection process is carried out in the background. This intuitively reduces the time the user has to wait for plane detection.

[0320] Then, in step S1004, cleaning area map creation unit 105 determines whether the plane detection in step S1003 has been completed. As a result, if it has been completed (YES), the process proceeds to step S1005. On the other hand, if it has not been completed (NO), the plane detection in step S1003 continues.

[0321] Furthermore, when a plane is detected, in step S1005, the cleaning area map creation unit 105 displays an alignment for creating a cleaning area map on the output unit 110. More specifically, an alignment that serves as a reference for creating a cleaning area map, such as a designated position (to be described later), is superimposed on the cleaning target, such as the photographed floor surface. Note that this step can be omitted. Then, as described in FIG. 9A, a cleaning area map is created in step S9. Note that an example of this step may be realized in Example 10 (to be described later). The cleaning area map created above can be identified by relative coordinates with the coordinates of the two-dimensional code. When Example 10 is used, the cleaning area map can be identified by the relative position between the coordinates of the two-dimensional code and the designated position. Note that the cleaning area map created in this manner is stored in the memory unit 111 by the cleaning area map creation unit 105 as cleaning area map information 114.

[0322] Next, the cleaning management process in Example 10 will be described. Fig. 50B and Fig. 50C are flowcharts showing the processing flow of the cleaning management process in Example 9. Fig. 50B is a flowchart when cleaning is performed in normal mode, and Fig. 50C is a flowchart when cleaning is performed in simple mode (easy mode). Note that these cleaning management processes can be replaced with step S22 (particularly step S2205 and step S2210) of the cleaning management process in Fig. 9B. The cleaning management process in Example 9 will be described below using each flowchart.

[0323] First, in step S1011 of Fig. 50B, the cleaning management unit 107 photographs the cleaning area using the photographing unit 102. As described in Fig. 50A, at this time, the user terminal 1 faces the cleaning area. Then, in step S1012, the cleaning management unit 107 causes the output unit 110 to display a display screen 1106 for creating a map including the image photographed by the photographing unit 102.

[0324] Also, in step S1013, similar to step S1001, cleaning management unit 107 reads the recorded content of the two-dimensional code captured by imaging unit 102 in response to a user's operation. Also, in step S1014, similar to step S1002, cleaning area map creation unit 105 detects the coordinates and direction of the cleaning target, for example, the floor surface, using the read result of the two-dimensional code.

[0325] Also, in step S1015, similar to step S1003, cleaning management unit 107 starts plane detection for the cleaning target using the image captured by image capture unit 102. Then, in step S1016, similar to step S1004, cleaning management unit 107 determines whether plane detection in step S1015 is complete. As a result, if completed (YES), the process proceeds to step S1017. If not completed (NO), plane detection in step S1015 continues.

[0326] Then, in step S1017, cleaning management unit 107 reads out the cleaning area map created by cleaning area map creation unit 105 and displays it on output unit 110. At this time, cleaning management unit 107 may display the cleaning area map superimposed on the photographed cleaning area. Alternatively, it may omit displaying the cleaning area map and display the content photographed by photographing unit 102. In response to this, the user operates stick vacuum cleaner 2 to start cleaning.

[0327] When cleaning starts (for example, when the cleaning button is turned on), in step S1018, the cleaning management unit 107 draws and displays the movement trajectory of the cleaner head 24 of the stick vacuum cleaner 2 on the output unit 110. It is desirable that this movement trajectory be displayed superimposed on the photographed cleaning area. Specific examples of drawing the activation trajectory will be described after the description of the contents of the simple mode.

[0328] Next, the cleaning management process in simple mode will be described with reference to Fig. 50C. Here, normal mode supports cleaning using the created cleaning area map, and steps S1011 and S1012 are the same processes as in normal mode in Fig. 50B.

[0329] Furthermore, steps S1013 and S1014 in normal mode are skipped, and in step S1015, cleaning management unit 107 starts plane detection for the cleaning target using the image captured by image capture unit 102. Then, in step S1016, cleaning management unit 107 determines whether plane detection in step S1015 is complete. As a result, if it is completed (YES), the process proceeds to step S1017. If it is not completed (NO), plane detection in step S1015 continues.

[0330] Then, when cleaning starts, step S1017 is skipped and in step S1018 the cleaning management unit 107 draws and displays the movement trajectory of the cleaner head 24 of the stick vacuum cleaner 2 on the output unit 110. This completes the description of the cleaning management process in simple mode.

[0331] <Trajectory drawing> Next, a specific example of trajectory drawing (step S1018) in the cleaning management process of FIGS. 50B and 50C will be described. This can also be replaced with step S2205 or step S2210 in FIG. 18 described above. In this example, the movement trajectory of the cleaner head 24 is displayed as an ellipse. In this example, the movement trajectory is drawn only when the cleaner head 24 (suction nozzle) is in contact with a cleaning target such as a floor. In other words, when the cleaner head 24 is lifted from the cleaning target, drawing of the movement trajectory is suppressed. However, either one of these steps may be omitted. Details thereof will be described below.

[0332] First, FIG. 51 is a diagram illustrating the concept of trajectory drawing in Example 9. In Example 9, it is assumed that the user terminal 1, in other words, its image capturing unit 102, is facing the floor surface 510 to be cleaned. For this purpose, it is assumed that the user terminal 1 is installed in the terminal holder 20. Then, the cleaning management unit 107 draws a virtual ray 511 (straight line) from the user terminal 1 (image capturing unit 102) to the floor surface 510 and determines the intersection point of this ray 511 with the floor surface. Then, using this result, it draws a movement trajectory. Specifically, the cleaning management unit 107 (1) displays an elliptical movement trajectory including this intersection point on the output unit 110 of the user terminal 1, and (2) draws the movement trajectory when the length of the ray 511, that is, the distance from the user terminal 1 to the floor surface 510, is within a certain distance. With regard to (1), it is more preferable that the intersection point be near the center of the cleaner head 24. Regarding (2), in other words, when the distance from the user terminal 1 to the floor surface 510 is equal to or greater than a certain distance, the drawing of the movement trajectory is suppressed.

[0333] For example, a stick vacuum cleaner 2 with a user terminal 1 installed may tilt parallel to the floor surface. In this case, the cleaner head 24 of the stick vacuum cleaner 2 will tend to point upward in response to the tilt. If the cleaner head 24 tilts beyond a certain level, the suction nozzle of the cleaner head 24 will come partially off the floor surface. Furthermore, if the stick vacuum cleaner 2 with a user terminal 1 installed moves above the floor surface by more than a certain amount, the suction nozzle will come off the floor. In other words, the stick vacuum cleaner 2 will be unable to clean or will not be able to clean sufficiently (hereinafter referred to as a situation where cleaning is not possible). In this way, (2) can detect the lifting of the cleaner head 24 (suction nozzle), etc.

[0334] As described above, when the length of Ray 511 (the distance between the intersection and the user terminal 1) exceeds a predetermined distance, the cleaner head 24 (suction nozzle) moves away from the floor surface. Therefore, in the ninth embodiment, when it is detected that the length of Ray 511 (the distance between the intersection and the user terminal 1) exceeds a predetermined distance, it is determined that cleaning is not possible and the drawing of the movement trajectory is suppressed. In other words, when the length of Ray 511 (the distance between the intersection and the user terminal 1) is within a predetermined distance, the movement trajectory is drawn in a limited manner.

[0335] The following describes in detail (1) drawing the movement trajectory and (2) detecting the lifting of the cleaner head 24 (suction nozzle). (1) Fig. 52 is a diagram showing a display screen 1106h on which a movement trajectory is drawn in Example 9. In Fig. 52, the display screen 1106h constituting the output unit 110 of the user terminal 1 displays the floor surface 1106h-1, wall surface 1106h-2, and vacuum cleaner 1106h-3 to be cleaned. These are photographed by the photographing unit 102, and the cleaning management unit 107 displays them. The cleaning management unit 107 then draws a movement trajectory 1106h-4 around the cleaner head of the vacuum cleaner 1106h-3.

[0336] Here, movement trajectory 1106h-4 is formed around intersection 512 in Figure 51, i.e., the cleaner head on display screen 1106h, and its size is approximately the same as the cleaner head on display screen 1106h. In this case, the major axis of the ellipse is approximately the same as the width of the cleaner head, and the minor axis is approximately the same as the height of the cleaner head. The shape of the movement trajectory also includes so-called ovals, i.e., egg shapes, oval shapes, rounded rectangles, etc.

[0337] In FIG. 52, the cleaner 1106h-3 has not yet moved, so a single shape is shown as the movement trajectory 1106h-4, but as the cleaner 1106h-3 moves, the movement trajectory 1106h-4 is formed in a continuous manner.

[0338] Next, the process for drawing a movement trajectory will be described. First, the cleaning management unit 107 calculates the intersection (intersection position) of the floor surface and the ray 511 extended from the user terminal 1 (image capture unit 102). The cleaning management unit 107 also obtains the yaw angle of the user terminal 1 (image capture unit 102) relative to the floor surface. To do this, the cleaning management unit 107 estimates the direction in which the cleaner head 24 (suction nozzle) is facing from the orientation of the user terminal 1 and uses this. The cleaning management unit 107 then displays the movement trajectory on the output unit 110 so that it is positioned at the calculated intersection position as an ellipse object with the orientation of the user terminal 1 at the yaw angle. In other words, the cleaner head 24 (suction nozzle) and the ellipse object are displayed so that they are positioned so that their length directions coincide. As a result, a movement trajectory 1106h-4 can be displayed as shown in FIG. 52.

[0339] (2) Next, a process for detecting lifting of the vacuum cleaner head 24 (suction nozzle) will be described. First, the cleaning management unit 107 calculates the intersection position of the floor surface and the ray 511 extended from the user terminal 1 (photographing unit 102), as in (1). The cleaning management unit 107 also calculates the distance from the intersection position to the user terminal 1 (photographing unit 102). If the calculated distance is within a predetermined distance (reference distance) stored in the memory unit 111, the cleaning management unit 107 draws the movement trajectory 1106h-4. If the calculated distance is longer than the predetermined distance, the cleaning management unit 107 suppresses drawing of the movement trajectory 1106h-4. Here, the predetermined distance can be a distance corresponding to the length of the extension tube 23 of the stick vacuum cleaner 2. For example, 87 cm can be used. Therefore, if a brush nozzle or the like is used, a different distance can be used as the reference. In this case, the cleaning management unit 107 may select the reference constant distance depending on image processing of the image captured by the photographing unit 102, a user's designation of the extension tube, or the like. Furthermore, the cleaning management unit 107 may calculate the distance from the intersection position at the start of cleaning and use this as a fixed distance (reference).

[0340] This concludes the explanation of the ninth embodiment. However, the game function may be used in the ninth embodiment as in the first embodiment, or may be omitted. The distance being longer than a certain distance also includes a case where the intersection position cannot be calculated. This can be used when the stick vacuum cleaner 2 is tilted, the user terminal 1 is not facing the floor, and the suction nozzle is also facing away from the floor. [Example]

[0341] Next, a tenth embodiment will show an example of creating a cleaning area map. The tenth embodiment can also be realized by using the same configuration and information as the first embodiment. For example, the functional block shown in FIG. 3 can be realized by the hardware shown in FIG. 4. Therefore, the processing of the tenth embodiment can be executed using the configuration of FIG. 3 below. In other words, the processing of the tenth embodiment can also be executed according to the cleaning management program 16. The following description will focus on the differences from the above-mentioned embodiments.

[0342] In the tenth embodiment, the cleaning area map creation unit 105 creates a cleaning area map in the form of a polygon, which is an example of an area formed by line segments connecting positions (designated positions) designated by the user via the input unit 109. For example, FIG. 53 shows how a cleaning area map 501-1 indicated by a rectangle is created on the display screen of the user terminal 1 by designating positions designated by the user in the order from "1" to "4." In this way, by connecting the designated positions "1" to "4" in the designated order with line segments, which are an example of line segments, a cleaning area map formed by a polygon with these as corners is created. Note that the areas in the tenth embodiment include not only polygons but also various closed shapes such as ellipses and crescents.

[0343] On the other hand, if the line segments that connect the specified positions and form a polygon intersect (self-intersect), a shape like 501-2 in Figure 53 will result (the black squares in the figure indicate self-intersections). When self-intersections like this occur, it becomes difficult for the user to clean. This is because multiple cleaning areas are created, and the user ends up moving outside the cleaning area. Furthermore, when multiple self-intersections occur, as in 501-3 in Figure 53, it becomes impossible or difficult to determine which areas are within the cleaning area. For example, in the figure, it is difficult to determine whether 501-3a and 501-3b are within or outside the cleaning area.

[0344] Therefore, in the tenth embodiment, the cleaning area map creation unit 105 determines whether the line segments constituting the polygon self-intersect. This determination includes a final line segment determination as to whether the final line segment candidate connecting the last specified final position candidate and the first specified starting position self-intersects with another line segment. In the case of 501-1 in FIG. 53, it is determined whether the line segment "4"-"1" self-intersects with another line segment.

[0345] In the tenth embodiment, the cleaning area map creation unit 105 controls the execution of an end instruction to complete the cleaning area map and a final line segment determination in cooperation with each other. For example, the cleaning area map creation unit 105 executes the final line segment determination on the condition that an end instruction is received. Furthermore, if the final line segment determination determines that the final line segment candidate self-intersects with another line segment, the cleaning area map creation unit 105 disables the acceptance of the end instruction. Specific examples showing these details will be described below. In the tenth embodiment, it is assumed that these designated positions are designated on the display screen 1116 of the user terminal 1, and a cleaning area map such as that shown in FIG. 53 is displayed. In the tenth embodiment, the two-dimensional code of the ninth embodiment can be used to identify each designated position and the position of the cleaning area map identified thereby using relative coordinates with the installation position of the two-dimensional code, but a coordinate system other than that of the ninth embodiment may be used in the tenth embodiment.

[0346] <Example 1> Specific Example 1 is an example in which the final line segment determination is performed on the condition that an end instruction has been received. The processing flow of Specific Example 1 will be described below. Fig. 54 is a flowchart showing the cleaning area map creation processing in Specific Example 1. Here, the processing flow shown in Fig. 54 can be realized as the map creation processing of step S9 in Example 9 or Fig. 9A, but may also be realized as processing independent of the processing in Fig. 9A.

[0347] First, in step S901, cleaning area map creation unit 105 accepts a designation of a designated position from the user via input unit 109. At this time, cleaning area map creation unit 105 is assumed to be displaying on output unit 110 the cleaning target, such as the floor, in the room photographed by photographing unit 102, as in step S7. Then, the user designates a designated position in relation to the output display content. For example, when using user terminal 1 shown in FIG. 4, touch panel 12 accepts the designated position by tapping on touch panel 12.

[0348] Here, the cleaning area map creation unit 105 identifies the coordinates of the floor surface corresponding to the position specified in step S901 in the captured image of the cleaning target displayed on the output unit 110. These coordinates correspond to the coordinates of the cleaning area map information 114 in Fig. 7. Note that these coordinates can be treated as relative coordinates between the installation position of the two-dimensional code and the specified position using the two-dimensional code of the ninth embodiment.

[0349] In step S902, the cleaning area map creation unit 105 determines whether the specified position in step S901 is the first specification in creating the cleaning area map. If it is the first specification, the process proceeds to step S903, and if it is the second or subsequent specification, the process proceeds to step S904.

[0350] In step S903, the cleaning area map creation unit 105 sets the coordinates identified in step S901 as the starting point. This starting point indicates the coordinate "1" shown in Fig. 53, that is, the initial position when creating the cleaning area map. Then, the process returns to step S901, and the cleaning area map creation unit 105 accepts the next specified position.

[0351] In step S904, the cleaning area map creation unit 105 creates a newest line segment, which is a line segment connecting the most recently designated position and the designated position immediately before that. In other words, the line segment connecting consecutive designated positions is set as the newest line segment. Here, this line segment is preferably a straight line, but may also be a curved line.

[0352] In step S905, cleaning area map creation unit 105 determines whether an end instruction has been received from the user via input unit 109 regarding the input of the designated position. As a result, if an end instruction has been received (YES), the process proceeds to step S906. If an end instruction has not been received (NO), the process returns to step S901 and the input of the next designated position is received. Note that whether an end instruction has not been received can also be determined by whether the next designated position has been input.

[0353] In step S906, the cleaning area map creation unit 105 creates a final position candidate, which is the most recently accepted specified position, and a final line segment candidate, which is a line segment connecting the starting point. In step S907, the cleaning area map creation unit 105 determines whether either the final line segment candidate or the latest line segment created in step S904 self-intersects. As a result, if any line segment self-intersects (YES), the process proceeds to step S908. If none of these line segments self-intersects (NO), the process proceeds to step S909.

[0354] Furthermore, in step S908, the cleaning area map creation unit 105 displays an alert to the effect that a self-intersection has occurred on the output unit 110. Here, the content of the alert can be expected to be something like "An invalid point has been specified."

[0355] Then, in step S909, cleaning area map creation unit 105 creates a polygon made up of line segments connecting the specified positions as a cleaning area map. As a result, the final line segment candidate created in step S906 becomes the final line segment. At this time, cleaning area map creation unit 105 stores the created cleaning area map in memory unit 111 as cleaning area map information 114. At this time, the coordinates of the cleaning area map information 114 can be set as each specified position.

[0356] This concludes the explanation of Specific Example 1, but in step S908, the following guidance display may be performed. The cleaning area map creation unit 105 displays the most recently specified position on the self-intersecting line segment on the output unit 110. As a result, the user can confirm the specified position that caused the self-intersection. Furthermore, if the cleaning area map creation unit 105 determines that the final line segment candidate self-intersects, it displays guidance information on the output unit 110 indicating a final position candidate where no self-intersection occurs.

[0357] <Example 2> Next, specific example 2 will be described, which shows another example of executing the final line segment determination on the condition that an end instruction has been received. FIG. 55 is a flowchart showing the cleaning area map creation process in specific example 2. Here, the processing flow shown in FIG. 55 can be realized as the map creation process of step S9 in embodiment 9 or FIG. 9A, similar to FIG. 54, but it may also be realized as a process independent of the process in FIG. 9A. Furthermore, the same reference numerals are used for the steps in FIG. 55 as for the processes similar to those in FIG. 54. Here, when similar processes are performed using different information, they are treated as similar processes.

[0358] 55, steps S901 to S904 are executed in the same manner as in Specific Example 1. Also, in step S911, cleaning area map creation unit 105 determines whether the created latest line segment self-intersects with another line segment. As a result, if there is a self-intersection (YES), the process proceeds to step S908. Also, if there is no self-intersection (NO), the process proceeds to step S905.

[0359] In step S905, the cleaning area map creation unit 105 determines whether an end instruction has been received, similar to Specific Example 1. Then, similar to Specific Example 1, if an end instruction has been received (YES), the process proceeds to step S906, and if an end instruction has not been received (NO), the process returns to step S901 and receives input of the next specified position.

[0360] In step S906, the cleaning area map creation unit 105 creates a final line segment candidate, as in Specific Example 1. In step S912, the cleaning area map creation unit 105 determines whether the final line segment candidate self-intersects with another line segment. As a result, if there is a self-intersection (YES), the process proceeds to step S909. If there is no self-intersection (NO), the process proceeds to step S908. In step S908, the cleaning area map creation unit 105 displays an alert on the output unit 110 indicating that a self-intersection has occurred, as in Specific Example 1. Here, the same content may be displayed when transitioning from step S911 and when transitioning from step S912, or different content may be displayed. In the latter case, the designated positions that caused the self-intersection (the most recent designated position, the final designated position candidate) may be displayed.

[0361] Then, in step S909, the cleaning area map creating unit 105 creates a cleaning area map, similar to Specific Example 1, and ends this processing flow. This concludes the description of Specific Example 2.

[0362] <Example 3> Specific Example 3 is an example in which acceptance of an end instruction is disabled when the final line segment candidate self-intersects with another line segment. FIG. 56 is a flowchart showing the process of creating a cleaning area map in Specific Example 3. Here, the process flow shown in FIG. 56 can be realized as the map creation process of step S9 in Example 9 and FIG. 9A, similar to FIGS. 54 and 55, but it may also be realized as a process independent of the process in FIG. 9A. Furthermore, the same reference numerals are used for the steps in FIG. 56 to indicate processes that are the same as those in FIGS. 54 and 55. Here, when similar processes are performed using different information, they are treated as similar processes.

[0363] 56, steps S901 to S904 are executed in the same manner as in specific examples 1 and 2. In step S911, cleaning area map creation unit 105 determines whether the created latest line segment self-intersects with another line segment. As a result, if the self-intersects (YES), the process proceeds to step S908. If the self-intersects (NO), the process proceeds to step S909.

[0364] In step S911, the cleaning area map creation unit 105 determines whether the latest line segment self-intersects with another line segment, as in Example 2. As a result, if the latest line segment self-intersects with another line segment (YES), the process proceeds to step S906. If the latest line segment does not self-intersect with another line segment (NO), the process proceeds to step S909.

[0365] In step S906, the cleaning area map creation unit 105 creates a final line segment candidate, similar to specific examples 1 and 2. In step S912, the cleaning area map creation unit 105 determines whether the final line segment candidate self-intersects with other line segments, similar to specific example 2. As a result, if the final line segment candidate self-intersects with another line segment (YES), the process proceeds to step S913. If the final line segment candidate does not self-intersect with another line segment (NO), the process proceeds to step S905.

[0366] Furthermore, in step S913, cleaning area map creation unit 105 invalidates the end instruction. Invalidating the end instruction includes refusing the end instruction from the user. As an example of this invalidation, the end instruction button on touch panel 12, which is an example of input unit 109, is deactivated so that input from the user to this button cannot be accepted. Then, the process proceeds to step S908.

[0367] Then, in step S908, the cleaning area map creation unit 105 displays an alert on the output unit 110. Here, in step S908 of specific example 3, the display content may be the same, i.e., "self-intersection has occurred," when transitioning from step S911 and when transitioning from step S913, but it is preferable to change the display content. In the latter case, when transitioning from step S913, a self-intersection has also occurred, so it is preferable to display a message that the end instruction button has been disabled. In this case, the cleaning area map creation unit 105 may further display a method for canceling the disablement on the output unit 110. The method for canceling the disablement can be realized by displaying guidance information that indicates final position candidates where no self-intersection occurs, as described in specific example 1.

[0368] Also, in step S905, the cleaning area map creation unit 105 determines whether an end instruction has been received, similar to specific examples 1 and 2. Then, similar to specific examples 1 and 2, if an end instruction has been received (YES), the process proceeds to step S909, where the cleaning area map creation process is executed. Also, similar to specific examples 1 and 2, if an end instruction has not been received (NO), the process returns to step S901, where the input of the next specified position is received. This concludes the explanation of example 10. [Example]

[0369] Next, as Example 11, a process for correcting distortion or distortion (hereinafter simply referred to as distortion) of a cleaning area map will be described. If a cleaning area map is displayed as is using the granularity (size) of unit areas currently used for various detections in creating a cleaning area map, there is a risk that the shape will be distorted. In other words, misalignment may occur in the horizontal or vertical directions of the screen. For example, FIG. 57 shows distortion of a cleaning area map 1106i-1 in Example 11. In FIG. 57, each side of the cleaning area map 1106i-1 is tilted with respect to each side of the display screen 1106i. In other words, distortion occurs in the cleaning area map 1106i-1.

[0370] Therefore, in the eleventh embodiment, the cleaning area map creation unit 105 corrects the distortion (tilt) by making the granularity of the unit area coarser, that is, larger, than that of the coordinate system used at the time of detection. Fig. 58 is a diagram showing the downward tilt (distortion) of the top side of the cleaning area map 1106i-1 in the eleventh embodiment. In Fig. 58, an upper side 583 is formed by connecting the end points 581 and 582. These are spaced apart by 0.01 mm in the AR space. 2 Here, the specified positions in the tenth embodiment can be used as the end points 581 and 582, for example.

[0371] For such cleaning area map 1106i-1, the slope can be made gentler by making the unit areas coarser. Hereinafter, as shown in FIG. 59A, for example, the slope can be made gentler by making the end points and the lines connecting them into coarser unit areas (portions filled with vertical lines) than the rectangles shown in the figure. Here, cleaning area map creation unit 105 corrects the top edge by making the unit areas coarser than the coordinate system used during detection and filling in each rectangle (unit rectangle) through which top edge 583 passes.

[0372] The process for this purpose will be described in detail below. First, coordinate system information consisting of a plurality of grids for the floor surface to be cleaned is stored in advance in storage unit 111. This coordinate system information is shown by arranging unit areas in a matrix, as shown in Figures 58 and 59A.

[0373] Then, the cleaning area map creation unit 105 identifies a ray from the specified position (a position in a virtual space having a certain height) specified by the user toward the floor. As explained in the tenth embodiment, a cleaning area map represented by a polygon is constructed by the line segments connecting the specified positions. Therefore, the specified positions indicate the corners (vertices) of the cleaning area map, which is a polygon.

[0374] The cleaning area map creation unit 105 also identifies the intersection of the identified Ray and the floor surface to be cleaned. The cleaning area map creation unit 105 also acquires the intersection coordinates of the identified intersection, and identifies the grid closest to the intersection coordinates from the stored coordinate system information.

[0375] The cleaning area map creation unit 105 also places corner objects in the identified grid. Corner objects are parts that become corners (vertices) of the cleaning area map, which is represented by a polygon. The cleaning area map creation unit 105 then places edge objects in the specified order so as to connect the corner objects (the filled-in parts in FIG. 59A). As a result, the cleaning area map creation unit 105 can create a grid map 1106i-1a, which is a modified cleaning area map as shown in FIG. 59B, and display it on the display screen 1106i. Hereinafter, such a modified cleaning area map will be referred to as a "grid map."

[0376] In the eleventh embodiment, a grid map may be used. Alternatively, or in addition to this, a "dot map" may be created. As shown in FIG. 59B, the grid map 1106i-1a has a slight distortion (tilt). Therefore, the cleaning area map creation unit 105 further corrects the grid map 1106-1i so that each side is aligned horizontally and vertically. If the tilt is greater than a predetermined value, the correction may be performed to create a step. As a result, the cleaning area map creation unit 105 can create a dot map 1106i-1b as shown in FIG. 59C, which is displayed on the display screen 1106i. Comparing the cleaning area map 1106i-1 with the corrected grid map 1106i-1a and dot map 1106i-1b, the distortion (tilt) is reduced or eliminated, and the sides of the cleaning area map are thicker.

[0377] The cleaning area map creation unit 105 may correct distortion of the cleaning area map 1106i-1 as follows. FIG. 60 is another diagram showing the correction of the downward right slope (distortion) of the top side of the cleaning area map 1106i-1 in Example 11. In this example, the cleaning area map creation unit 105 corrects the top side 583a by moving the end points 581a and 582a to the center of the unit area in a coordinate system in which the unit area is coarser than the coordinate system used during detection. In a more desirable mode, the cleaning area map creation unit 105 increases the thickness of the top side 583a. This makes the slope more gentle.

[0378] This concludes the explanation of Example 11, but Example 11 can be applied to map creation in Example 1 as well as map creation in Example 9. Furthermore, the coarse unit area in Example 11 is a square, and one side of the square is preferably equal to or greater than the width of the cleaner head 24, and preferably a value in which a predetermined value is added to the width. For example, if the width of the cleaner head 24 is 25 cm, one side of the unit area can be 30 cm.

[0379] Furthermore, when cleaning (cleaning management processing) is performed with the stick vacuum cleaner 2, the cleaning management unit 107 preferably displays at least one of the cleaning area map 1106i-1, the grid map 1106i-1a, and the dot map 1106i-1b on the output unit 110. The display will be described below.

[0380] FIG. 61A is a diagram showing a display screen 1106j of cleaning-related information in Example 11. In FIG. 61A, a cleaning area map 1106i-1, a grid map 1106i-1a, and a dot map 1106i-1b are displayed as cleaning-related information, along with a vacuum cleaner 1106j-2 and a movement trajectory 1106j-3, which are captured by the image capture unit 102. In this figure, the cleaning area map 1106i-1, the grid map 1106i-1a, and the dot map 1106i-1b are displayed from left to right slightly above the center of the display screen 1106j, and each map also shows the cleaned area (circled in the figure) as the cleaning status. This allows the user to intuitively grasp the cleaning status. The movement trajectory 1106j-3 may be elliptical as shown in Example 9, or may be another shape.

[0381] Here, the display screen that displays the above-mentioned cleaning area map may also display the movement status, such as the speed, of cleaner head 24. An example of this is shown in Fig. 61B. Fig. 61B is a diagram showing a display screen 1106k for cleaning-related information in Example 11. In Fig. 61B, the suction tip lift, speed, and acceleration are further displayed as the movement status of cleaner head 24 as cleaning-related information.

[0382] 61B, in addition to the display contents of Fig. 61A, cleaning management unit 107 displays suction nozzle floating warning lamp 1106j-4, speed warning lamp 1106j-5, and acceleration warning lamp 1106j-6. When it is detected that cleaner head 24 (suction nozzle) has left the floor surface as described in Example 9, cleaning management unit 107 changes the color of the display to red or the like, changes the intensity, or turns on the lamp.

[0383] Furthermore, when the speed of the cleaner head 24 (cleaner 1106j-2) exceeds a predetermined speed such as 0.8 m / s, the cleaning management unit 107 changes the color of the display to red or the like, changes the intensity, or turns on the speed warning lamp 1106j-5. Furthermore, when the acceleration of the cleaner head 24 (cleaner 1106j-2) exceeds 1.4 m / s, the speed warning lamp 1106j-6 is turned on. 2 When the predetermined speed is exceeded, cleaning management unit 107 changes the color of the display to red or the like, changes the intensity, or lights up the display. The technology can be used. Note that, with regard to each of these warning lamps (1106j-4 to 1106j-6), a configuration in which any one of them is provided may be provided, or a display indicating which of the "suction nozzle float," "speed," and "acceleration" warning lamps is provided near each warning lamp may be provided. Furthermore, on the display screen 1106k, display of at least one of the cleaning area map 1106i-1, the grid map 1106i-1a, the dot map 1106i-1b, and the movement trajectory 1106-k3 may be omitted. Note that these warning lamps may be displayed on the output unit 110 by the cleaning area map creation unit 105 during the map creation process. [Example]

[0384] Example 12 illustrates adjustment of a movement trajectory such as in Example 1 or Example 9. This adjustment includes changing the shape and shifting the position. Details thereof will be described below with reference to Figs. 62A to 62D. Fig. 62A is a diagram showing a display screen 1106l (part 1) for adjusting a movement trajectory in Example 12. On the display screen 1106l in this figure, the cleaning management unit 107 displays an adjustment button 1106l-1, a reset button 1106l-2, a vacuum cleaner 1106l-3, and a movement trajectory 1106l-4a.

[0385] The cleaning management unit 107 shifts the display position of the drawn movement trajectory 1106l-4a up, down, left, or right in response to a user's specification using the adjustment button 1106l-1. This is because the movement trajectory 1106l-4a is displayed superimposed on the image captured by the image capturing unit 102, and the display position may shift from the vacuum cleaner 116l-3. Therefore, the display position of the drawn movement trajectory 1106l-4a can be adjusted using the adjustment button 1106l-1. The amount of this shift (adjustment) can be set to 2.5 cm, 5 cm, 10 cm, 15 cm, 20 cm, etc. each time the button is pressed. Alternatively, the shift may be set to a fixed amount.

[0386] Furthermore, when the reset button 1106l-2 is pressed, the cleaning management unit 107 returns the shifted display position of the movement trajectory 1106l-4a to its original position. Note that in Fig. 62A, the shape of the movement trajectory 1106l-4a is displayed as a rectangle. However, the shape of the movement trajectory is not limited to a rectangle, and it can be displayed as a polygon including a quadrangle or the ellipse described above.

[0387] Fig. 62B is a diagram showing a display screen 1106l (part 2) for adjusting a movement trajectory in Example 12, and shows an example in which a movement trajectory 1106l-4b is displayed in an elliptical shape. The rest is the same as Fig. 62A. As a result, the user can display the movement trajectory in a shape that is easy for the user to understand.

[0388] Furthermore, the display effect of the movement trajectory can be changed. For example, a blurring effect, a glowing effect, or a change in transparency can be applied. Fig. 62C is a diagram showing a display screen 1106l (part 3) for adjusting the movement trajectory in the twelfth embodiment, and shows an example in which a blurring effect is added to the movement trajectory 1106l-4c and displayed.

[0389] In addition, in the twelfth embodiment, the size of the movement trajectory and its drawing start position can also be adjusted. Fig. 62D is a diagram showing a display screen 1106l (part 4) for adjusting the movement trajectory in the twelfth embodiment, and shows an example of a screen for adjusting the size of the movement trajectory and its drawing start position. In Fig. 62D, a movement trajectory size adjustment button 1106l-5 and a drawing start position adjustment button 1106l-6 are added to Figs. 62A to 62C.

[0390] When the user presses the movement trajectory size adjustment button 1106l-5, the cleaning management unit 107 changes the size of the movement trajectory. Furthermore, when the drawing start position adjustment button 1106l-6 is pressed, the cleaning management unit 107 moves the drawing start position of the movement trajectory. For example, it moves as indicated by the cross marks in the figure. Note that these cross marks may be displayed on the display screen 1106l, or their display may be omitted. Note that the drawing start position indicates the intersection point of Ray 511 in FIG. 52 with the floor surface. Furthermore, although not shown, the color and shade of the movement trajectory may also be adjustable. [Example]

[0391] Example 13 is an example of display processing in cleaning management processing using a cleaning area map created during cleaning. Fig. 63 is a diagram showing a display screen 1106m during cleaning management processing in Example 13. In Fig. 63, the cleaning management unit 107 displays a minimap area 1106m-1, a real map 1106m-2, and a real trajectory 1106m-3 on the display screen 1106m.

[0392] First, a minimap including a mini-track showing the movement track is displayed in the minimap area 1106m-1. The minimap is a kind of thumbnail image that is a reduced version of the created cleaning area map. However, the mini-track is also drawn in accordance with the cleaning (movement) of the stick vacuum cleaner 2.

[0393] Furthermore, the real map 1106m-2 shows an image in which the boundaries (edges) of the created cleaning area map are superimposed on the floor surface photographed by the photographing unit 102. The real map 1106m-2 shows a grid map or dot map, and each rectangle in the figure indicates a unit area (grid). Furthermore, grids in different colors indicate grids at specified positions or grids at vertices. Furthermore, the real trajectory 1106m-3 shows a movement trajectory superimposed on the floor surface photographed by the photographing unit 102. By using the display screen 1106m as described above, the user can intuitively grasp the cleaning status because the cleaning area map and movement trajectory are virtually displayed in the photographed image.

[0394] Next, a process for displaying such a display screen 1106m will be described. In the thirteenth embodiment, the above display is realized by virtual photography using a virtual camera. The idea is to photograph an object, which is a grid placed on the floor, with a virtual camera virtually placed above. That is, the object placed on the floor is photographed with a virtual camera placed above, and the image is displayed in the mini-map area 1106m-1 at the top right of the display screen 1106m.

[0395] Fig. 64A is a diagram for explaining the placement of a virtual camera in Example 13. In Fig. 64A, cleaning management unit 107 sets the average of corners 641 (plurality of coordinates) of the room specified by the user in cleaning area map 642 as the central coordinate 643 of the room, that is, cleaning area map 642. Then, cleaning management unit 107 virtually places a virtual camera facing downward above central coordinate 643. Here, cleaning management unit 107 sets the height of the virtual camera so that the corner of the room farthest from central coordinate 643 fits within the angle of view.

[0396] FIG. 64B is a diagram for explaining the positioning of the virtual camera in the height direction in the thirteenth embodiment. As described above, the height of the virtual camera 644 needs to capture the entire cleaning area map 642, that is, the farthest corner. Therefore, when the angle of view θ (for example, 60 degrees) is set, the cleaning management unit 107 calculates the height h as h = L / Tan(θ / 2). Here, L indicates the distance from the center coordinate 643 to the corner 641 (the farthest corner). The cleaning management unit 107 displays the results of the image captured by the virtual camera 644 on the display screen 1106m.

[0397] Here, the actual trajectory 1106m-3 shown in FIG. 63 is displayed in detail, and the trajectory on the minimap is displayed as a grid. For this reason, the cleaning management unit 107 displays the actual trajectory object for the content captured by the capture unit 102, and does not display the grid trajectory object on the minimap. In other words, the capture unit 102 acquires the actual trajectory in a limited manner. Furthermore, the cleaning management unit 107 does not display the actual trajectory object for the content captured by the virtual camera 644, and displays the grid trajectory object. In other words, the virtual camera 644 acquires the grid trajectory in a limited manner. As a result, a display such as that shown in FIG. 63 is possible.

[0398] Furthermore, on the display screen 1106m in the thirteenth embodiment, there is a risk that the actual trajectory 1106m-3 may go beyond the real map 1106m-2. This situation is shown in FIG. 65A. This may occur when the stick-type vacuum cleaner 2, such as the cleaner head 24, goes beyond the area indicated by the created cleaning area map due to a user operation, or when the drawing position of the actual trajectory 1106m-3 or the real map 1106m-2 is shifted. In the thirteenth embodiment, the actual trajectory 1106m-3 is adjusted in such a case. This adjustment will be explained below. First, the cleaning management unit 107 determines whether the actual trajectory 645 has protruded outside the cleaning area map. This determination will be explained using FIG. 65B. FIG. 65B schematically shows an actual trajectory 645 protruding outside the cleaning area map 642. In such a case, the cleaning management unit 107 counts the number of intersections between line segments 646 from the origin of the coordinate system to each actual trajectory 645 and the sides (boundaries) of the cleaning area map 642. If the number of intersections is an even number, the cleaning management unit 107 determines that the corresponding actual trajectory 645 has protruded outside the cleaning area map 642.

[0399] Then, if one or more actual trajectories 645 protrude beyond the cleaning area map 642, the cleaning management unit 107 performs a trajectory adjustment process on each of the actual trajectories 645. Specifically, the cleaning management unit 107 moves the coordinates of each actual trajectory 645 toward the center coordinate 643 of the cleaning area map 642. For example, as shown in FIG. 65C , the actual trajectories 645 are shifted toward the center coordinate 643 by a predetermined percentage (for example, 1%) in both the x and y axis directions. The cleaning management unit 107 repeats this process until the actual trajectories 645 fit within the cleaning area map 642. In this way, when the actual trajectory 645 protrudes beyond the cleaning area map 642, the actual trajectory 645 can be moved toward the center of the cleaning area map 642 to prevent it from protruding, and it is possible to prevent the trajectory from being drawn outside the cleaning area map.

[0400] Here, each actual trajectory 645 is shifted, but adjustment, i.e., shifting, may be limited to the actual trajectory that protrudes. Furthermore, the cleaning management unit 107 may suppress drawing when the actual trajectory protrudes, or may suppress drawing of the actual trajectory 645 that protrudes in a limited manner. Furthermore, in the above example, the actual trajectories 645 are uniformly shifted by a predetermined percentage, but the actual trajectories 645 farther from the central coordinate 643 may be shifted by a greater amount.

[0401] Furthermore, the adjustment of the actual trajectory (movement trajectory) also includes the following aspects: When the directions of cleaning area map 642 and actual trajectory 645 do not match and diverge, cleaning management unit 107 adjusts the direction of actual trajectory 645 to match the direction of cleaning area map 642 according to the content read from the two-dimensional code. The direction of cleaning area map 642 can be, for example, any side of cleaning area map 642.

[0402] Also, if the scale of cleaning area map 642 and actual trajectory 645 do not match, cleaning management unit 107 makes adjustments according to the content read from the two-dimensional code. Furthermore, if there is a discrepancy between actual trajectory 645 and the real space, that is, the content captured by imaging unit 102, cleaning management unit 107 makes adjustments using the processes described in Figures 65A to 65C.

[0403] Furthermore, if cleaning area map 642 is not drawn correctly on display screen 1106m, cleaning management unit 107 displays a message urging the user to move stick vacuum cleaner 2 by tracing the sides of cleaning area map 642 (actual map 1106m-2 in FIG. 65A). In response to such movement, cleaning management unit 107 adjusts the sides of cleaning area map 642 to straight lines. Furthermore, cleaning management unit 107 receives a specification from the user regarding a corner of cleaning area map 642, and adjusts the angle to 90 degrees.

[0404] Furthermore, if the self-position of the stick vacuum cleaner 2 deviates, the cleaning management unit 107 resets the position recognition once and corrects the position according to the content read from the two-dimensional code. In this case, the origin of the coordinate system is recorded in the two-dimensional code. The cleaning management unit 107 may also suppress drawing of the actual trajectory 645. Furthermore, the cleaning management unit 107 may correct the deviation using the detection results of a gyro sensor or acceleration sensor provided in the user terminal 1. Furthermore, the cleaning management unit 107 may arrange items such as those described in Example 8 on the display screen 1106m and perform the cleaning management process without drawing an actual trajectory.

[0405] In the thirteenth embodiment, the cleaning management unit 107 displays the minimap as a dot or grid, and displays the movement trajectory and cleaning area map in more detail than the minimap in the AR space (other than the minimap area 1106m-1 in FIG. 63). As a result, the user can roughly check which areas have been cleaned on the minimap, and can check in more detail which areas have been cleaned or not, that is, the cleaning status, between ARs.

[0406] This concludes the description of each embodiment of the present invention, but the present invention is not limited to these embodiments. In particular, the user terminal 1 may be realized as a wearable terminal such as so-called AR goggles, smart glasses, or a smart watch. In this case, the wearable terminal does not need to be attached to the stick vacuum cleaner 2. Furthermore, a main body (e.g., a smartphone) separate from the wearable terminal may be provided. In this case, the main body may create cleaning-related information and send it to the wearable terminal, or the wearable terminal may create it. Furthermore, at least two of the embodiments may be combined. Furthermore, the display screens described in embodiments 1 and 2 may be shared with the relevant parties. Sharing may be performed with the manufacturer's engineers or service personnel. Furthermore, the present invention also includes the following various modifications.

[0407] <Modification 1: Minimap adjustments> The cleaning management unit 107 rotates the minimap of the minimap area 1106m-1 displayed on the display screen 1106m in Figures 63 and 65A so that it aligns with each side of the display screen 1106m. In other words, if the horizontal and vertical directions of the minimap and the display screen 1106m do not match, the cleaning management unit 107 rotates the minimap so that they match.

[0408] Furthermore, the minimap may be rotated manually. That is, the cleaning management unit 107 displays a rotation button on the display screen 1106m, accepts operation of the rotation button by the user, and rotates the minimap accordingly.

[0409] <Variation 2: Use of depth information> The cleaning area map information 114 includes depth information, which is used to control the display content. First, depth information is information that indicates the height at a position (coordinate) on a plane. Therefore, by using the depth information, various objects with height, such as chairs and walls, can be identified.

[0410] Therefore, the cleaning management unit 107 uses the depth information to make the object transparent if there is an object in front of the movement trajectory, cleaning area map, or other object, and displays the movement trajectory or cleaning area map. As a result, it is possible to reduce the sense of incongruity that occurs when an object behind an object such as a wall is displayed. Note that when the position of the movement trajectory is the same as the floor height, it is desirable for the cleaning management unit 107 to set the position (height) at which the movement trajectory is drawn higher than the original height to prevent the movement trajectory from being hidden or revealed by the floor surface.

[0411] <Modification 3: Recognition of the cleaner head 24> When drawing a movement trajectory, the cleaning management unit 107 specifies its reference position. For example, the coordinates slightly below the center of the user terminal 1 are set as the drawing start position of the movement trajectory. To achieve this, the cleaning management unit 107 performs image recognition of the cleaner head 24 and draws the movement trajectory from its center position. This reduces the sense of incongruity regarding the position where the movement trajectory is drawn. As a result, movement trajectories can be drawn for a variety of vacuum cleaners from different manufacturers.

[0412] <Variation 4: Use of game functions> In the first and second embodiments, the use and non-use of the game function are controlled. Similar control is possible in the other embodiments and modifications. The former operation can be called the game mode, and the latter operation can be called the normal mode.

[0413] In addition to the above-described modifications, the following display examples are also included in the present invention.

[0414] <Display example 1: Switching between game mode and normal mode> Fig. 66 is a diagram showing a display screen 1106n for switching (selecting) between the game mode and the normal mode. The display screen 1106n-1 and the display screen 1106n-2 in Fig. 66 are displayed by the cleaning management unit 107 in step S2201 in Fig. 18 of the first embodiment, for example.

[0415] On the display screen 1106n-1, "No Score" is selected, so in the example of FIG. 18, the process transitions to step S2208. That is, the stick vacuum cleaner 2 operates in normal mode. On the display screen 1106n-2, "With Score" is selected, so in the example of FIG. 18, the process transitions to step S2202. That is, the stick vacuum cleaner 2 operates in game mode.

[0416] <Display example 2: Map creation process> Next, as display example 2, the display screen 1106o during the map creation process will be described. Figs. 67A to 67C show the screen transitions of the display screen 1106o during the map creation process. First, when the map creation process is started, the display screen 1106o-1 of Fig. 67A is displayed. Here, when the user selects "Start cleaning", the display screen transitions to display screen 1106o-2 or display screen 1106o-3. These correspond to display screen 1106n-1 and display screen 1106n-2 of Fig. 66.

[0417] When a mode is selected on display screen 1106o-2 or 1106o-3, the display screen transitions to display screen 1106o-4. This display screen 1106o-4 prompts the user to create a cleaning area map, and then transitions to display screen 1106o-5 in Fig. 67B. This transition may occur after a certain time has elapsed since display screen 1106o-4 was displayed, or in response to a user's instruction, such as a tap.

[0418] Additionally, display screen 1106o-5 displays an explanation that the two-dimensional code needs to be read in order to create a cleaning area map. Here, the two-dimensional code is shown attached to charging stand 26. Then, in response to an operation from the user, photographing unit 102 is activated and begins photographing, and the display screen changes to display screen 1106o-6. Display screen 1106o-6 displays the photographed content and a prompt to read the two-dimensional code.

[0419] In response to this, when the user points the user terminal 1 (photographing unit 102) at the two-dimensional code, a display like that shown on display screen 1106o-7 appears. Here, the user is prompted to acquire the two-dimensional code. Then, when the two-dimensional code is read by the user's operation, the display screen transitions to display screen 1106o-8. Display screen 1106o-8 displays a prompt to input the corner of the room as the designated position (center of the white circle). When this designated position is input, the display transitions to display screen 1106o-9 of Figure 67C, which displays a "marker" to indicate the designated position in Example 9 and prompts the user to input the designated position.

[0420] <Display example 3: Cleaning after map creation process> Next, as display example 3, a display screen 1106p for cleaning after the map creation process will be described. FIGS. 68A to 68C show screen transitions of the display screen 1106p for cleaning after the map creation process. First, when the map creation process is completed or when a cleaning instruction is received from the user after completion, display screen 1106p-1 is displayed. Display screen 1106p-1 is similar to display screen 1106n-2 in FIG. 66, and is a screen for selecting between game mode and normal mode for cleaning. Then, when an input is made on this display screen 1106p-1 that cleaning will be performed, the screen transitions to display screen 1106p-2.

[0421] Display screen 1106p-2 is a display screen for prompting the user to install the user terminal 1 in the terminal holder 20. When the "Start" button is selected on this display screen 1106p-2, the display screen transitions to display screen 1106p-3. Note that display screen 1106p-2 may be displayed as display screen 1106p-4, visually displaying the installation of the user terminal 1 so that it is easy to intuitively understand.

[0422] When the user points the user terminal 1 (photographing unit 102) at the two-dimensional code, a display such as that shown on the display screen 1106p-3 appears, similar to the display screen 1106o-7.

[0423] Next, display screens 1106p-5 to 1106p-7 in Figure 68B display information for performing cleaning (cleaning management process). These display screens 1106p-5 to 1106p-7 count down, and the character's appearance changes in accordance with this countdown. Then, when the cleaning management process starts, the display screen transitions to display screen 1106p-8. This display screen 1106p-8 is a display screen for cleaning-related information, which corresponds to display screen 1106j and display screen 1109k in Example 11. Furthermore, display screen 1106p-8 shows cleaning in game mode, and displays acquired items in the lower left corner.

[0424] Furthermore, when cleaning is completed, display screen 1106p-9 in FIG. 68C is displayed. This display screen 1106p-9 allows the user to confirm whether or not to end the cleaning management process (cleaning). As a result, when the user selects the "End" button, the display screen transitions to display screen 1106p-10. Display screen 1106p-10 displays that cleaning is completed and how to remove user terminal 1 from terminal holder 20. Then, when the "View Results" button is selected on display screen 1106p-10, the display screen transitions to display screen 1106p-11.

[0425] Display screen 1106p-11 displays information to guide the user to the details of the cleaning that has been performed. Tapping on this display screen 1106p-11 transitions to display screen 1106p-12. Display screen 1106p-12 then displays the evaluation (Nice) of the cleaning details and the points (number of icons) that have been acquired. [Explanation of symbols]

[0426] 1...user terminal, 11...camera, 12...touch panel, 13...processing device, 14...communication device, 15...storage device, 16...cleaning management program, 161...vacuum cleaner identification module, 162...operation status detection module, 163...cleaning area map creation module, 164...AR information creation module, 165...cleaning management module, 166...history information creation module, 101...communication unit, 102...photography unit, 103...vacuum cleaner identification unit, 104...operation status detection unit, 105...cleaning area map creation unit, 106...AR information creation unit, 107...cleaning management unit, 108...history information creation unit, 109...input unit, 110...output unit, 111...storage unit, 112...user management information, 113...cleaning management information, 114...cleaning area map information, 115...guidance information, 116...game information, 2...stick Type vacuum cleaner, 20... terminal holder, 20-1... holding part, 20-2... arm part, 20-3... base, 20-4... attachment part, 21... vacuum cleaner main body, 22... handle, 23... extension tube, 24... vacuum cleaner head, 25... handy brush, 26... charging base, 27... holder member, 28... stand member, 29... base member, 3... cleaning management device, 31... processing device, 32... communication device, 33... memory, 34... secondary storage device, 301... communication unit, 302... vacuum cleaner identification unit, 303... operation status detection unit, 304... cleaning area map creation unit, 305... AR information creation unit, 306... cleaning management unit, 307... history information creation unit, 308... storage unit, 310... user management information, 311... cleaning management information, 312... cleaning area map information, 313... guidance information, 314... game information, 4... router, 5... network

Claims

1. A user terminal, which is a computer attached to a terminal holder of the vacuum cleaner, an input unit that accepts a plurality of designated positions from a user; a cleaning management program that functions as a cleaning area map creation unit that creates a cleaning area map using line segments connecting the specified positions, The cleaning area map creation unit is a cleaning management program that controls the execution of an end instruction to complete the cleaning area map and a determination of self-intersections between the final line segment identified in response to the end instruction and other line segments in cooperation with each other.

2. The cleaning management program according to claim 1 , the input unit accepts the termination instruction, The cleaning area map creation unit creating a final line segment candidate connecting a final position candidate among the specified positions and a starting point among the plurality of specified positions; determining whether the final line segment candidate and any of the latest line segments created by successive designated positions self-intersect with each other; A cleaning management program that creates a polygon made up of line segments connecting the plurality of designated positions as the cleaning area map when the final line segment candidate and any of the plurality of latest line segments do not self-intersect.

3. The cleaning management program according to claim 1 , The cleaning area map creation unit Create multiple latest line segments by specifying consecutive positions, determining whether each of the plurality of latest line segments self-intersects; the input unit accepts the end instruction when each of the plurality of latest line segments does not self-intersect; The cleaning area map creation unit creating a final line segment candidate connecting a final position candidate among the specified positions and a starting point among the plurality of specified positions; determining whether the final line segment candidate and any of the plurality of latest line segments self-intersect; A cleaning management program that creates a polygon made up of line segments connecting the plurality of designated positions as the cleaning area map when the final line segment candidate and any of the plurality of latest line segments do not self-intersect.

4. The cleaning management program according to claim 1 , The cleaning area map creation unit creating a plurality of latest line segments from consecutive designated positions, and determining whether each of the plurality of latest line segments intersects with itself; a cleaning management program that invalidates the end instruction when each of the plurality of latest line segments self-intersects;

5. The cleaning management program according to claim 1 , The cleaning management program causes the user terminal to further function as a cleaning management unit that displays, on an output unit, a movement locus of a cleaner head of the vacuum cleaner in an elliptical shape.

6. The cleaning management program according to claim 1 , A cleaning management program in which the cleaning area map creation unit creates the cleaning area map using the line segments as unit areas larger than the unit areas used when creating the cleaning area map.

7. The cleaning management program according to claim 6, A cleaning management program in which a unit area larger than the unit area used when creating the cleaning area map is equal to or larger than the width of a cleaner head of the vacuum cleaner.

8. The cleaning management program according to claim 1 , The cleaning management program further causes the user terminal to function as a cleaning management unit that determines that the cleaning head of the vacuum cleaner is floating above the cleaning object when the distance between the user terminal and the intersection of the cleaning object of the vacuum cleaner is equal to or greater than a reference distance.

9. The cleaning management program according to claim 8, The reference distance is determined according to an extension tube of the vacuum cleaner.

10. The cleaning management program according to claim 9, The cleaning management program, wherein the cleaning management unit calculates the distance between the user terminal and the intersection of the cleaning target of the vacuum cleaner at the start of cleaning, and uses the calculated distance as the reference distance.

Images